On July 12 we made a change to improve the experience for Business trialists that did not have any profiles connected. This was called the “blank state”.
In this analysis we’ll check if the fix had any impact on the proportion of business trialists that were able to connect a profile and schedule a post.
So far, there isn’t much evidence to suggest that the change has made a large impact on the number of trialists connecting profiles. Interestingly though, there is some evidence that the change has had a positive effect on the proportion of trialists that create posts.
Let’s look at Business trialists that started trials on the same day they signed up for Publish. A couple of important choices:
- This only includes trialists that started a trial within 12 hours of signing up.
- This only includes profiles that were connected within 6 hours of starting the trial.
- This only includes posts created within 12 hours of starting the trial.
# connect to BigQuery con <- dbConnect( bigrquery::bigquery(), project = "buffer-data", dataset = "dbt_buffer" ) # define sql query sql <- " select t.id as trial_id , u.id as user_id , date(trial_start_at) as trial_date , t.customer_id , t.converted , count(distinct p.id) as profiles , count(distinct up.id) as updates from stripe_trials t inner join publish_users u on t.customer_id = u.stripe_customer_id and timestamp_diff(t.trial_start_at, u.created_at, hour) <= 12 left join publish_profiles p on p.user_id = u.id and timestamp_diff(p.created_at, t.trial_start_at, hour) <= 6 left join publish_updates up on p.id = up.profile_id and up.created_at > t.trial_start_at and timestamp_diff(up.created_at, t.trial_start_at, hour) <= 12 where date(t.trial_start_at) >= '2019-04-01' and t.plan_id in ('business_v2_small_monthly', 'business_v2_small_yearly', 'business_v2_medium_monthly', 'business_v2_medium_yearly', 'premium_v1_monthly', 'premium_v1_yearly', 'business_v2_large_monthly', 'business_v2_large_yearly') group by 1, 2, 3, 4, 5 " # collect data trials <- dbGetQuery(con, sql)
There are around 16 thousand trials in this dataset.
Effect on Profile Connections
Let’s plot the trials over time.
We can see that there are somewhere around 150 business trial starts per day. Let’s view the number of trialists that were able to connect a profile.
It’s hard to tell what is happening here, so let’s plot the proportion of trialists that connected a profile.
It looks like around 60% of Business trialists connect profiles within 6 hours of starting a trial. There doesn’t appear to have been a large effect on the proportion of trialists that are connecting profiles just yet.
Effect on Posts Sent
Let’s plot the proportion of trialists that create posts in Publish.
It does look like the proportion of trialists that create posts within 12 hours of starting the trial has increased since the change was made, although it is still below 20%.