In the course of an A/B experiment, the correct calculation of a sample size is one of the key success ingredients. Yet, sometimes the amounts of traffic necessary for statistical significance of tests put app publishers off. Indeed, a required sample size can be large that means a test lasts longer than you’d like.
However, this obstruction is not that dramatic if you run your A/B tests with help of SplitMetrics. The thing is the platform can apply an alternative approach called Bayesian Multi-armed Bandit (MAB), which can solve the above-mentioned drawback without even bothering you.
A Bayesian Multi-armed Bandit test allows choosing an optimal variation of the two or more. Unlike a classic A/B test, which is based on statistical hypotheses testing, a Bayesian MAB test proceeds from Bayesian statistics. In this post, we’ll learn more about the principles behind it.