How to Interpret Test Results?

SplitMetrics tracks visitor behavior from the moment they arrive on your app store page via a campaign, through each view, click and scroll, to install or drop off. Our reports give you metrics to analyze user behavior and drastically improve conversion on your app pages. At a glance, you’ll be able to see and understand what’s working and what’s not, and quickly iterate design and content to drive more app installs.

Overview

Overview gives you basic information about the page visits. You’ll see how many people visited the page and how much time they spent on it. To learn more about each metric, click on a circled question mark right above the metric or go to the App Store Page Metrics page.

Unique Visitors

Each time a user lands on your app page via the campaign link, a visitor counter increments for that user. Unique Visitors metric helps you understand how many visitors went to your app page via a campaign link.

What is a good number of unique visitors for a campaign?

SplitMetrics Assistant will help you monitor experiments to make sure you have enough traffic for a statistically significant result. If the volume of traffic is too low, we’ll warn you of the high probability of random chance results to make sure you get reliable and actionable data.

Filtered Visitors

This metric shows the number of visitors that were filtered to exclude traffic from a particular source. For example, you may want to filter internal traffic coming from your company devices or users with a specific OS.

What if the number of unique visitors is too high or too low?

If the number of filtered visitors is too high or too low, check your campaign settings to make sure you’re limiting the correct data. A popular case is filtering iPhone and Android experiments; if the number of Filtered Visitors looks higher than expected, check if you’re sending visitors to the landing page designed for the platform you want to test.

Average session time

How long a visitor stays on the app page is a good indication that your campaign is attracting relevant traffic. It may also point to how much time your average user needs to make a decision whether or not to download the app after they land on the page via a campaign link.

Why is my average session time so low?

Average session time usually shows the level of interest and involvement. Low average time on page may indicate that users lost their interest when they arrived on the app page from the campaign link. Maybe the visual story on the screenshots wasn’t compelling enough; it could be that the ad creative and the app store content didn’t match up so the user left. On the other hand, low average session time combined with high conversion rate (unlikely, yet possible combination) shows the user didn’t need a call-to-action on the page and downloaded the app right away.

SplitMetrics Assistant

SplitMetrics Assistant is here to help you monitor traffic to rule out the probability of random chance results. It will analyze the number of visitors you’re sending to the app page and tell you if you have enough traffic to get statistically significant results.

Benchmarks

Benchmarks allow you to compare your app page’s performance to that of your competitors in the same category. We aggregate data for each category, shared by millions of users in a variety of our experiments. This provides you meaningful context to understand the conversion trends across categories and helps set better goals for optimization.

Variations Performance Overview

This dashboard shows how the pages are performing. You can see the winning alternative and understand which message and creative will improve your bottom line: conversion.

In the example experiment above, the winner is alternative A with 90% conversion, up 20.8% from Alternative B. Low Click-to-install rate means you have to pay more per click. In the case above, with a cost-per-click price of $.40 and the conversion rate of 69.2%, you will pay $0.60 per every user who installed your app. If you advertise with the second alternative that has a higher conversion rate of $90%, your cost per install will drop to $0.40, which can save you $2,000, should you pay for 10,000 clicks, and $20,000 if you’re planning to pay for 100,000 clicks. Improving your app page conversion ultimately optimizes your media spend and maximizes the output of your marketing efforts.

As you can see from the example experiment, higher engagement rate often correlates with higher conversion rate (90% in A vs. 84.6% in B). High engagement rate is a sign that your creatives are well-received by the page visitors. High average time on page is a good indication of highly engaging content. To understand how much more likely the winning alternative is going to outperform its counterpart, check Chance to Beat Control metric (9.6% in the example case).

Behavior Metrics

Behavior Metrics will give you insights on the number of visitors, how long they stayed on the page and how they interacted with it. If users spend little time on the page before they leave, there is something on the page that is driving them away – or you’re driving the wrong kind of traffic to the page. Either way, it’s generally bad news, because lower time on page correlates with lower conversion rate.

High engagement rate is a good marker of high-performing app page content. It is a good metric to assess if the page creatives spark visitors’ interest and makes them want to see more. If they like it, they click it, simple as that. High engagement rate correlates with higher conversion. Explored shows the number of people who engaged with your page but didn’t convert. Bounced visitors are those who left your page instantly without any interaction; it doesn’t necessarily mean that the page content is underperforming – it might be because people accidentally clicked on the link, or maybe their Internet connection suddenly broke off right after they loaded the page. It also could be that the content on the app page didn’t quite match up with the ad that they clicked on. Not everyone likes surprises.

Why do I get many Explored and few Converted?

Some visitors arrive on the page, look around, click, engage, and then leave. Things were on the right track. What happened? Consider your app page as a teaser for a blockbuster that is coming up. Maybe, they were put off by a high price point? Simply lost interest? Perhaps, you’re not giving them a strong enough motivation to download. You want to keep your visitors engaged on the page, yet show them that the real deal is when they download the app. Start with a stronger call-to-action in captions and roll from there.

Why do I get many Bounced Visitors?

Bounced visitors load the page and leave; maybe, they didn’t want to click on the link in the first place. It could be that the ad that they clicked on looked very different from what they saw on the app page, so they left confused. Another reason is that the message was not clear: If users can’t tell in a couple of seconds they spent on your page what the app is about, they will not take the time to stay and figure it out. Also, check your ad campaign demographics – if many visitors bounce, you may be sending the wrong people to your pages.

Install Button

There are many ways visitors can convert to users. Data that shows actions people take before they tap “Install” will uncover their conversion paths and events that triggered more downloads from the page. This information will help you focus on what matters most in the conversion funnel.

Direct installs show converted visitors who skipped views and scrolls and installed the app the moment they got to the page. It may be because these people were already familiar with your app or heard people talk about it.

Installs after Screenshot Scroll show converted visitors who engaged with at least one screenshot before they downloaded the app.

Installs per Read More show visitors who expanded your text description.

Screenshots

This dashboard shows key metrics to help you compare and contrast the performance of each tested screenshot set. By looking at the data on screenshot view rate, average view duration, and number of installs after a screenshot view, you can tell which of your creatives convert and which you may want to tweak and optimize.

Few people will read past the first paragraph of an app’s description. The higher the percent of those who actually take the time to expand the text section and study the contents, the more converting your first line of text is. Think of your app’s description as an ad copy for your product. Experiment with catchy, exciting, and brief introductory sentences highlighting your killer USPs. Consider running a test with a positive user review or media mentions on top, or including an attractive price proposition for paid apps (e.g., “limited time discount”, etc.)For Google Play, it’s also important to consider the keywords’ relevance for search.

Scroll Depth

Scroll depth shows how visitors interact with your app’s creatives and helps you visualize the path they take on the page. Such report can help you discover which content keeps user engaged and helps you identify areas that need improvement.

Audience

Audience dashboard helps you track demographic differences in conversion. For example, you can see what works and what not for a specific geographic region and identify localization issues. Audience report helps you track content issues ranging by demographics early on and optimize to tailor the creatives to new markets.