One of the most popular questions app publishers ask is how much traffic they need to run valid A/B tests. Unfortunately, there is no answer with a magic number that will fit every single experiment. An optimal traffic volume is individual and depends on such factors as a traffic source, app’s conversion rate, and targeting.
However, we can get to the bottom of sample size calculations for А/В tests. It’s really important to have a full understanding of it as sample size has a considerable effect on checking the significance of the observed difference in variations performance.
In this post, we’ll also review one of sample size measuring methods which is widely used and helps to make a statistically valid decision based on the results of your A/B test.