Sequential A/B Testing: Workflow and Advantages over Classic Experiments

Liza Knotko, 1 week ago
ASO, A/B Testing
Comparing classic and sequential A/B testing

When it comes to A/B tests, anyone has a natural desire to get trustworthy results without spending a heap of money on traffic. Alas, it’s not always possible with classic A/B testing which requires enormous sample sizes at times. 

Is there a better way? Sure, there is!

Sequential A/B testing might become a robust alternative. Such experiments don’t only optimize necessary traffic volumes but also reduce the likelihood of mistakes. Let’s take a closer look at this method and how it differs from the classic A/B testing flow.
Read more

ASO checklist

Improve your ASO with the World's Most Ultimatest App Store Optimization Guide!

10 Ways to Use Mobile A/B Testing in Pre-Launch: from Ideas Validation to Product Page Refinement

Liza Knotko, June 6th
ASO, Cases, A/B Testing
mobile A/B testing in pre-launch

Nowadays, winning users hearts is not an easy task it used to be at the dawn of major app stores. The industry matured and the competition is enormous. Judge for yourself, there are approximately  2.1 million Android apps in the Play Market and almost 1.8 iOS apps available in the App Store.

There’s no use developing a random app in the hope of overnight success. You have to play it smart and validate every single idea before bringing it to life. The best possible way of doing it is mobile A/B testing in pre-launch.
Read more

Case: Lab Cave Achieves 45.8% Conversion Boost by A/B Testing App Store Screenshots

Anastasia Sidoryk, December 21st
ASO, Cases, A/B Testing, Creatives
lab cave app store screenshots

It goes without saying, the two major app stores have different layouts, although the product page elements are quite the same. The differences were evident to the ASO experts at Lab Cave, a mobile growth company providing ASO and Ad Mediation services. After having tested their Play Store visuals, the company made a sensible decision to optimize their App Store screenshots.

Discovering what works better for different app stores was a strong reason, but the main one, however, was to identify what boosts the conversion rate from page visits to installs.
Read more

Guide to Mobile A/B Testing: Proven Strategies, Professional Tips, and App Store Optimization Best Practices

Liza Knotko, December 13th
ASO, A/B Testing
mobile A/B testing guide by SplitMetrics

Mobile A/B testing has been around for quite a while and for a good reason. It can be widely used for marketing purposes: from getting data on the behavior of the target audience to user acquisition on major app stores.

If you understand the value of data-driven decisions, mobile A/B testing might become your go-to solution as it lets you get beyond the guesswork. Splitmetrics teamed up with Apptimize and created a comprehensive guide to App Store and in-app A/B testing which will help you grow your mobile business.

Read more

All You Need to Know about A/A Testing – from Goal Setting to Results Interpretation

Liza Knotko, September 27th
ASO, A/B Testing

A/A testing is the tactic of using a testing tool to test two identical variations against each other. Whether it is worth to conduct A/A testing and, if so, for what purposes are the questions that invite conflicting opinions.

In this post, we explore why some users of testing tools like SplitMetrics practice A/A tests and dwell on the things they need to keep in mind while performing this sort of tests.
Read more

Category and Search A/B Tests: How to Improve App Store Search Ranking

Liza Knotko, August 29th
ASO, A/B Testing
category and search A/B test

Ever thought how category and search ranking affects your app’s conversion rate? Taking into consideration that about 65% of downloads are the result of a search in the App Store, the most successful mobile marketers never disregard the optimization of their apps for competitive surrounding.

If you aim to improve App Store search ranking, a consistent mobile A/B testing strategy is a must. Search and category split tests should become an integral part of such strategy and today we’ll discuss how to run this kind of A/B experiments to guarantee the best possible results.
Read more

Advancing Mobile A/B Testing with Bayesian Multi-Armed Bandit

Liza Knotko, June 28th
ASO, A/B Testing
A/B Testing with Multi-Armed Bandit

In the course of an A/B experiment, the correct calculation of a sample size is one of the key success ingredients. Yet, sometimes the amounts of traffic necessary for statistical significance of tests put app publishers off. Indeed, a required sample size can be large that means a test lasts longer than you’d like.

However, this obstruction is not that dramatic if you run your A/B tests with help of SplitMetrics. The thing is the platform can apply an alternative approach called Bayesian Multi-armed Bandit (MAB), which can solve the above-mentioned drawback without even bothering you.

A Bayesian Multi-armed Bandit test allows choosing an optimal variation of the two or more. Unlike a classic A/B test, which is based on statistical hypotheses testing, a Bayesian MAB test proceeds from Bayesian statistics. In this post, we’ll learn more about the principles behind it.
Read more

Creative Sets: Testing within Apple Search Ads and Limitations of This Method

Liza Knotko, June 6th
Search Ads, A/B Testing
Search Ads Creative Sets testing

The introduction of Creative Sets gives Apple Search Ads users a great opportunity to align various creatives to different keywords. On top of that, now app publishers can test Apple Search Ads assets. However, it doesn’t mean that Apple’s update made full-scale A/B tests possible within the App Store itself.

In this post, we will figure out how Creative Sets can be used for Apple Search Ads assets testing and what are the limitations of this experimental method.
Read more

SplitMetrics Latest Update: Advanced Video Analytics for App Preview A/B Tests

Liza Knotko, May 23rd
ASO, A/B Testing
SplitMetrics app preview A/B tests

Video content is conquering the world. According to Hubspot, 90% of users state that product videos facilitate the decision-making process. However, when it comes to app previews in the App Store, one shouldn’t rush to pack their product pages with three 30-second long videos.

There are a lot of cases when a clumsy app preview doomed app’s conversion giving the impression that the publisher doesn’t really care. Nevertheless, smart app previews work their video magic and boost downloads. According to our study, optimization of video previews triggers the average conversion increase of around 16%.

A/B testing is one of the best ways to reduce the risk factor of app previews. To take your video experiments to the next level making them even more impactful, the SplitMetrics team prepared brand new advanced analytics for app previews in the App Store.
Read more

Determining A/B Testing Sample Size: Method Based on Statistical Hypothesis Testing

Liza Knotko, April 28th
ASO, A/B Testing
Sample Size for mobile A/B testing

One of the most popular questions app publishers ask is how much traffic they need to run valid A/B tests. Unfortunately, there is no answer with a magic number that will fit every single experiment. An optimal traffic volume for mobile A/B testing is individual and depends on such factors as a traffic source, app’s conversion rate, and targeting.

Now let’s get to the main point: how to determine the sample size for A/B tests? It’s really important to have a full understanding of it as sample size has a considerable effect on checking the significance of the observed difference in variations performance.

In this post, we’ll also review one of the A/B test sample size measuring methods which is widely used and helps to make a statistically valid decision based on the results of your mobile A/B testing.
Read more

12345