On January 15 the Publish team started rolling out the new Instagram profile connection flow. In this analysis we will measure the effect that the new flow has on 1) the rate at which all Instagram profiles are connected to Publish and 2) the rate at which Instagram Business profiles are connected to Publish.
Overall, the changes had a positive effect on the proportion of users that were able to successfully connect Instagram accounts to Buffer.
Data Collection
We’ll collect all of the Instagram connection attempt events with the SQL query below. A connection attempt is considered to be “successful” if the user was able to connect an Instagram account to Buffer at some point after the attempt.
# connect to BigQuery
con <- dbConnect(
bigrquery::bigquery(),
project = "buffer-data"
)
# define sql query
sql <- "
select
ca.original_timestamp as attempted_at
, ca.user_id as account_id
, ca.channel
, ca.channel_type
, case when u.features like '%reminders_flow%' then true else false end as has_new_flow
, count(distinct case when p.service_type = 'profile' then p.id else null end) ig_profiles
, count(distinct case when p.service_type = 'business' then p.id else null end) biz_profiles
from segment_publish_server.channel_connection_attempted ca
inner join dbt_buffer.core_accounts a
on ca.user_id = a.id
left join atlas_publish_buffer.users u
on u._id = a.publish_user_id
left join dbt_buffer.publish_profiles p
on ca.user_id = p.account_id
and p.service = 'instagram'
and p.created_at >= ca.original_timestamp
where ca.channel = 'instagram'
and ca.original_timestamp >= '2020-01-15'
group by 1,2,3,4,5
"
# query bigquery
events <- dbGetQuery(con, sql)# connect to BigQuery
con <- dbConnect(
bigrquery::bigquery(),
project = "buffer-data"
)
# define sql query
sql <- "
select
timestamp_trunc(p.created_at, week) as week
, count(distinct p.id) as profiles
from dbt_buffer.publish_profiles p
where p.created_at > '2019-01-01'
and p.service = 'instagram'
and p.service_type = 'business'
group by 1
"
# query bigquery
profiles <- dbGetQuery(con, sql)Overall Connection Rate
The plot below suggests that the new flow has had a significant, positive impact on the proportion of users that were able to connect an Instagram account, personal or business.
Business Profile Connection Rate
The plot below suggests that there was an improvement in the Business connection rate of around ~10%, which is very significant given that the connection rate for the control group was below 15%.
Total Number of Business Profiles Added
Let’s plot the total number of Instagram Business profiles added to Buffer by week.
It looks like the overall number of Instagram Business profiles being added to Publish is increasing, which is a good thing.