On January 15 the Publish team started rolling out the new Instagram profile connection flow. In this analysis we will measure the effect that the new flow has on 1) the rate at which all Instagram profiles are connected to Publish and 2) the rate at which Instagram Business profiles are connected to Publish.
Overall, the changes had a positive effect on the proportion of users that were able to successfully connect Instagram accounts to Buffer.
We’ll collect all of the Instagram connection attempt events with the SQL query below. A connection attempt is considered to be “successful” if the user was able to connect an Instagram account to Buffer at some point after the attempt.
# connect to BigQuery con <- dbConnect( bigrquery::bigquery(), project = "buffer-data" ) # define sql query sql <- " select ca.original_timestamp as attempted_at , ca.user_id as account_id , ca.channel , ca.channel_type , case when u.features like '%reminders_flow%' then true else false end as has_new_flow , count(distinct case when p.service_type = 'profile' then p.id else null end) ig_profiles , count(distinct case when p.service_type = 'business' then p.id else null end) biz_profiles from segment_publish_server.channel_connection_attempted ca inner join dbt_buffer.core_accounts a on ca.user_id = a.id left join atlas_publish_buffer.users u on u._id = a.publish_user_id left join dbt_buffer.publish_profiles p on ca.user_id = p.account_id and p.service = 'instagram' and p.created_at >= ca.original_timestamp where ca.channel = 'instagram' and ca.original_timestamp >= '2020-01-15' group by 1,2,3,4,5 " # query bigquery events <- dbGetQuery(con, sql)
# connect to BigQuery con <- dbConnect( bigrquery::bigquery(), project = "buffer-data" ) # define sql query sql <- " select timestamp_trunc(p.created_at, week) as week , count(distinct p.id) as profiles from dbt_buffer.publish_profiles p where p.created_at > '2019-01-01' and p.service = 'instagram' and p.service_type = 'business' group by 1 " # query bigquery profiles <- dbGetQuery(con, sql)
Overall Connection Rate
The plot below suggests that the new flow has had a significant, positive impact on the proportion of users that were able to connect an Instagram account, personal or business.
Business Profile Connection Rate
The plot below suggests that there was an improvement in the Business connection rate of around ~10%, which is very significant given that the connection rate for the control group was below 15%.
Total Number of Business Profiles Added
Let’s plot the total number of Instagram Business profiles added to Buffer by week.
It looks like the overall number of Instagram Business profiles being added to Publish is increasing, which is a good thing.