Paper by FiftyThree: how to get 33% more installs with screenshots A/B testing

You’ve definitely heard about a beautiful device, the stylus Pencil by FiftyThree. The company also has an awesome app – Paper. It allows creating notes, photo annotation and sketches on the iPad and iPhone. To increase the number of downloads and get insights on the on-page user behaviour, the FiftyThree team turned to SplitMetrics.

The case

Creating a great app is not enough anymore. Its app store page has to appeal to the users as it has to drive downloads and boost engagement. So when the team at FiftyThree decided to redesign their listing, there were many ideas on how to develop a new app store experience. The team had to choose just one of several approaches to best communicate the value of Paper and help customers convert quickly. Their next thought was to test the strongest visual representation of the app – screenshots – to discover which ones users like the most.

How it works

The experiment for an iPhone validated several assumptions: determine the most appealing background colour, need for screenshots localization and screenshots order. Any A/B testing activity should start with a hypothesis. The initial hypothesis was that target audience might react differently to screenshots captions in English and Chinese, and a coloured background vs monochrome. Another question is how different screenshot pictures and background colour might impact visitor conversion.

SplitMetrics is addictive.
Andrew Allen
Co-founder & Head of Design at FiftyThree

Screenshots order
When designing screenshots the FiftyThree team came up with several compositions. The screenshots include action shots displaying interaction with an app. Putting the most important screen first, the team wanted to make sure the second one is compelling enough. The choice was given between a small detail on a sketch and a bright graph.


There were 17% more installs after scrolling the second combination of screenshots. The SplitMetrics team assumed that it might be attributable to the fact that the first picture had a tiny detail in a screen, while the second one looked more appealing displaying a bright-coloured graph.

Screenshots background
Two of the many ideas considered were: different coloured or monochrome blue or brownish blurred background.


And the victory goes to the first combination with different coloured screenshots that obviously resonates and leads to more installs. The hypothesis is that the more screenshots differ the more engaging the page gets for the visitors. The combination higher screenshots scroll rate proves that they tend take a closer look at such screenshots.

Screenshots localization
Reaffirming the importance of localization for the Chinese-speaking market, the team decided to measure the impact of uploading localized screenshots.


Localized screenshots performed 33% better than the English ones. Having your app available to larger potential market means more revenue. And SplitMetrics provides a tool to determine if localizing the whole application is worth the effort, and if the app is more likely to be downloaded if it is targeted for specific languages.

Using the SplitMetrics A/B testing report, the FiftyThree team tracked and analyzed how visitors interacted and converted within newly-made app store pages. It provides data on how people interact with the listing: if they scroll it or not, if they read the description or not, how scrolling the screenshots correlates with conversion and more.

By running the A/B tests and analyzing the SplitMetrics on-page behaviour report, the marketing team at FiftyThree determined that the new screenshots design improves conversions by 33%.

Related articles

Angry Birds 2 as a case study for ASO and A/B testing
Case study: iPad screenshots for Skoda app