Which Features are Used Most by Pro Trialists?

Data Collection

We need to find the events in the actions_taken table that correspond to each of these actions. First we’ll collect all of the trialists that were on a Pro trial in the past year, then join the relevant updates, profiles, and actions_taken records. The SQL query used is a big one, so the code has been hidden.

There are over 58 thousand Pro trial records to work with. The trials started after August 1, 2018 and before August 2, 2019.

Data Tidying

Next we’ll create a couple new fields that tell us some more information about the trialist.

# determine if started at signup
trials <- trials %>% 
  mutate(started_at_signup = trial_start == signup_date,
         used_first_comment = updates_with_first_comment > 0,
         used_calendar = calendar_actions > 0,
         did_view_posts = viewed_posts > 0,
         shared_update = updates > 0,
         added_profile = profiles > 0,
         used_extension = extension_updates > 0,
         used_mobile = mobile_updates > 0)

Exploratory Analysis

Let’s explore some of these metrics to get a better idea of how people are making use of the Pro trial.

I suspect that usage of some of these features is correlated. Let’s see if that isn’t true by looking at a correlation matrix.

The number of profiles is most strongly correlated with the number of updates created, which makes sense. The number of updates created is most strongly correlated with the number of posts created from the extension and mobile apps – that also makes sense. Calendar actions are correlated with viewing post analytics. Let’s remove extension updates and mobile updates to see if we can uncover anything else.

The number of profiles one has is correlated with updates, viewing post analytics, and to a smaller extent viewing the calendar. Viewing the calendar is correlated with viewing post analytics.

Now we’ll look at each feature individually. We’ll start with the number of profiles trialists had. Note that they could have added these profiles before the trial started, but not after it ended.

We see here that most trialists added between 0 and 5 profiles. Let’s see how this correlates with conversion rates.

As the title states, there is a strong positive correlation between the number of profiles added and the trial conversion rate. The relationship is roughly linear between adding the second and sixth profile. After the sixth profile, the returns in terms of conversion rate diminish to nothing.

Now let’s look at the distribution of the number of updates sent. Because this distribution is more continuous and long-tailed, we will “bin” the number of updates into buckets.

We can see here that most trialists didn’t create any updates. The next biggest bucket is the 11-50 update range, followed by the 2-5 range. Let’s see how these are related to the conversion rate.

We can see again that there is a strong positive correlation between updates created and the conversion rate. The correlation weakens after 100 updates have been created. It also strengthens once trialists create their second update. Let’s take the same approach for posts created from the extension.

Ok, most trialists didn’t make full use of the extension. Did usage correlate with a higher probability of converting?

There is a positive correlation here. Now let’s look at mobile app usage.

Again, relatively few users are taking advantage of the mobile apps.

Now let’s see how many Pro trialists view post analytics.

We can see here that most people did not look at the Posts tab in Analytics. Let’s see if there is any correlation between viewing the Posts tab and converting.

There is a positive correlation between viewing post analytics and converting, especially when the tab is viewed more than once. I have a sneaking suspicion that viewing post analytics is correlated with sending updates.

Let’s see how many trialists viewed the Week and Month views.

Very few people viewed the week or month views. Let’s see how this correlates with conversion.

Again there is a strong correlation between viewing the calendar and converting.