How the Premium Plan Affected Small Business MRR

This will be a quick analysis of how the Premium plan’s introduction affected Small Business MRR. We introduced the Premium plan on March 21, 2019.

# list premium plans
prem_plans <- "Premium Business v1 - Monthly - Monthly,Premium Business v1 - Yearly - Yearly"

# list small business plans
smb_plans <- "Small Business v2 - Monthly,Small Business v2 - Yearly"

# get premium MRR
premium <- get_mrr_metrics(metric = "mrr",
                           start_date = "2018-01-01",
                           end_date = "2019-08-21",
                           interval = "day",
                           plans = prem_plans) %>% mutate(plan = "Premium")

## [1] "Getting data from: https://api.chartmogul.com/v1/metrics/mrr"

# get small business MRR
small <- get_mrr_metrics(metric = "mrr", 
                           start_date = "2018-01-01", 
                           end_date = "2019-08-21",
                           interval = "day",
                           plans = smb_plans) %>% mutate(plan = "Small Business")

## [1] "Getting data from: https://api.chartmogul.com/v1/metrics/mrr"

# join datasets
all <- premium %>% 
  rbind(small) %>% 
  mutate(date = as.Date(date),
         mrr = mrr / 100)

Now let’s plot the combination of MRR.

We see that there doesn’t appear to have been a huge impact on MRR.

library(CausalImpact)

# run analysis
impact <- CausalImpact(mrr_ts, pre.period, post.period, model.args = list(niter = 5000))

# plot impact
plot(impact) +
  labs(title = "Impact on Premium and Small Business MRR",
       subtitle = "There is not evidence of any significant impact") +
  buffer_theme()

summary(impact)

## Posterior inference {CausalImpact}
## 
##                          Average        Cumulative     
## Actual                   208            31778          
## Prediction (s.d.)        241 (111)      36834 (17038)  
## 95% CI                   [21, 458]      [3265, 70047]  
##                                                        
## Absolute effect (s.d.)   -33 (111)      -5055 (17038)  
## 95% CI                   [-250, 186]    [-38269, 28513]
##                                                        
## Relative effect (s.d.)   -14% (46%)     -14% (46%)     
## 95% CI                   [-104%, 77%]   [-104%, 77%]   
## 
## Posterior tail-area probability p:   0.37688
## Posterior prob. of a causal effect:  62%
## 
## For more details, type: summary(impact, "report")

There is no evidence of a significant positive or negative impact on MRR.