A/B Testing — 6 Jun 2021
10 Ways to Use Mobile A/B Testing in Pre-Launch
Maryna Dorash 
Nowadays, winning users hearts is not an easy task it used to be at the dawn of major app stores. The industry matured and the competition is enormous. Judge for yourself, there are approximately 2.1 million Android apps in the Play Market and almost 1.8 iOS apps available in the App Store.
There’s no use developing a random app in the hope of overnight success. You have to play it smart and validate every single idea before bringing it to life. The best possible way of doing it is an app pre-launch marketing plan.
Just imagine, you came up with a brilliant app idea which captures your imagination and it’s so tempting to rush into developing it right away. The plain truth is it’s exactly what you shouldn’t do.
By the way, the same concerns ideas resulted from endless product meetings and brainstorms, especially, when contrary opinions and visions collide within your team. It’s extremely important to preserve sound judgment.
Only the data-driven approach gives a solid ground for further app development.
Remember that any pet idea can’t be considered good enough unless it is thoroughly analysed and tested. What should you do then?
It makes sense to start with answering 3 questions:
Sure, these questions are not the easiest but they are extremely important. The answers lie in the deep understanding of the needs of your potential users and their experience with similar apps.
You can perform this pre-production research the old way. For instance, it can be done manually via customer surveys and focus groups. This method is more or less reliable but very time-consuming. The study can also be ordered from a third-party analytical company. However, mind that this approach is very expensive and has lots of potential pitfalls.
There’s no use overcomplicating this research stage. That’s why the best option is to use mobile A/B testing which doesn’t only provide trustworthy results but also gives you full control over the study progress. With the help of specialized tools like SplitMetrics, you can effortlessly emulate Product and Search pages with potential apps.
Let’s examine use cases of mobile A/B testing in pre-launch.
Above all, you can check all app ideas on actual users to identify if your potential app stands a chance in the severe competition of the App Store and Google Play.
If nobody clicks the “Get” button on the emulated app product page within your pre-launch A/B test, it’s a sure sign to leave the idea behind. It goes without saying that there’s no use investing time and money in an app nobody needs.
Mobile A/B testing in pre-launch allows you to check all app concepts in order to identify the right positioning and spot resonating messages. It’s handy to have such insights for a non-existent app, isn’t it?
Indeed, the same idea or technology might be presented in many different forms. For example, a hit app MSQRD relied on pre-launch A/B testing with SplitMetrics and ran a series of experiments that helped to choose between two different concepts.
Did you know that this well-known app with live masks could have become a messenger at some point? Yet, the face app idea got 35% better conversion rate and the team went in this direction.
A/B testing in pre-launch helps companies to come up with positioning which hits a chord with potential users. For example, SplitMetrics helped WANNA NAILS validate the positioning and perform their iOS app pre-launch optimization. The title featuring the phrase “Manicure Try On” clearly explained app essence and beat other variations with 26.4% conversion rate improvement.
It’s worth mentioning that a series of pre-launch tests and overall hard work of WANNA team eventually paid off — several months later, the app rocked the App Store, the company raised $2M investment and got featured on Product Hunt.
At times, choosing the right category may play an important part in app positioning as well. Mobile A/B testing in pre-launch won’t let you resign to your fate and ensures your app debuts in the right category. Such split-tests are especially important for gaming apps.
For instance, they can help you choose between such genres as action and strategy. We’ve seen the case when a game publisher experimented with the message of their first screenshot via pre-launch A/B tests. One variation focused on ‘strategiс’ aspect of the game while the other one positioned the app as a shooter. The first message won with 12% better conversion making it clear which subcategory to choose.
Pre-launch A/B testing is an amazing tool for determining the target audience for your upcoming app. With the help of deep segmentation, it’s possible to identify which group is interested in your future product. Experimenting with targeting you can collect data on preferable age, sex, location for your further marketing activity. It also can be used once your app is available in the app stores.
Mobile A/B testing in pre-launch may also assist in the decision-making process when it comes to determining the most performant app advertising channels.
It’s possible to qualify different ad channels like news, Facebook, cross promo, and various ad networks. Thus, you can allocate your marketing budget more effectively once your app is live.
Once you found out an idea which resonates with your target audience, pre-launch A/B testing may shed light on your further moves regarding app development. You can identify a must-have feature your app should have from day one and prioritize the rest.
As it was mentioned above MSQRD ran complex pre-launch A/B testing campaign. At deciding to create a face mask app, the team kept on testing and found out that the audience was 55% more interested in live masks than in face swap feature.
This insight helped the dev team to devote all energy to potentially viral feature. Eventually, they made the right call as those data-driven decisions in pre-launch helped MSQRD become ‘The best App of 2016’.
It is well understood that UI design may become an important contributing factor when it comes to popularity among users. So why don’t you check which interface layout triggers more clicks on the ‘Get’ button.
All it takes is preparing a few sets of screenshots which represent various UI concepts and launch an A/B test with SplitMetrics.
Sometimes the first days in an app store may predefine the future of your app. That’s why it’s so important to be prepared and harness all opportunities. Isn’t it great to be able to debut in a store with pitch-perfect icon, screenshots, and video app preview? On top of that, you can optimize other elements such as app name, subtitle, etc.
For instance, intelligent pre-launch optimization with SplitMetrics helped Rovio get 2,5 million of additional installs in just a week after the launch of ‘Angry Birds 2′. This example proves once again that the elements of store product pages should never be neglected and are definitely worth testing.
When it comes to icons, here is a couple of pre-launch cases:
The pre-launch period is a great time to prepare your app for launch in different regions. However, mind that localization isn’t a mere translation of description and subtitles, it’s vital to localize beyond the text. Mobile A/B testing is a handy and efficient way of verifying various cultural hypotheses.
If you opt for specialized tools like SplitMetrics, mobile A/B testing in pre-launch can also assist in collecting emails of potential users and building a primary base of your app early adopters.
It’s possible to use these contacts for surveys and these people should definitely be the first to know about the release of your new app.
It comes as no surprise that mobile A/B testing in pre-launch calls for thorough preparation. There are a few aspects you should consider to make the most of your experiments.
First and foremost, you have to decide if your approach is:
In case of the app-centric approach, it makes sense to start with experiments which contain only one variation reflecting a core idea of your potential app via a set of screenshots, title, subtitle, icon and short description. Don’t forget about deep segmentation of your traffic (use ad groups with individual UTMs)
When your mobile A/B testing is finished, pay attention to the following aspects:
– Conversion – a core metric which lets you understand if your app idea is worth developing, use latest CR benchmarks across app categories as your reference point.
– Performance across audience segments – deep segmentation is what can help you identify your primary target audience (the more granular, the better). Later you can evaluate an ad group which was the most interested and evaluate its monetization prospects.
– Performance across ad channels – deep segmentation helps out here once again making it possible to identify the most efficient traffic sources.
– Bounce rate – this metric is a touchstone of your ad campaign’s settings. In case of serious bounce rate, check if your ad banners correlate with the message of the emulated app product page.
– ‘Explored’ section – it lets you see at which point of product page exploration people bounce. Thus you can see what elements and aspects of your app draw potential users away.
Once you opt for a certain app concept, target audience, and ad channels, it’s a good idea to get down to polishing your product page elements (icons, screenshots, names, etc). This time, it’s better to run experiments with two or more variations.
When it comes to the audience-centered approach, it’s vital to pay extra attention to tailoring your app store creatives and ad banners to tastes of your primary TA segment. For example, it’s unlikely that minimalistic monochrome UI of your potential app depicted on screenshots will win over teenagers. It’s better to experiment with bright colors and cheerful vibe then.
Whether you choose an app-centric or TA-centric approach, it’s really important to understand that pre-launch mobile A/B testing is a complex procedure and one test is hardly ever enough.
The best results can be achieved via a series of thought-over experiments.
Specialized A/B testing tools like SplitMetrics is a must for pre-launch experiments. It doesn’t only emulate product pages of the App Store and Google Play in a matter of minutes but also provides in-depth analytics which you can download as a report. The metrics SplitMetrics provides go beyond mere conversion rate of different variations.
The platform also lets you direct converted users to any link of your choice (web site, publisher page in an app store, the product page of another app, etc.). It’s also possible to collect emails of users who were ready to install your potential app. On top of that, SplitMetrics mobile A/B testing functions in accordance with GDPR.
When you decide which app store element to test, we’d recommend focussing on screenshots as they give you an amazing playground for reflecting your vision of a new app. App title and subtitle are other elements worth testing. An icon can be a good option too, for example, when you decide on central characters.
However, there’s no use investing time and money in creating an app video preview for a non-existing app, our study showed that the overwhelming majority of App Store pages with no videos perform better at the moment.
Pre-launch A/B testing with SplitMetrics may become a game changer for your company. If you leave the guesswork, the chances of producing the next big app will increase immensely. Yet, it’s important to understand that any type of mobile A/B testing activity calls for elaborate and consistent strategy and strong concepts worth testing.
