Mobile A/B Testing with SplitMetrics: Onboarding Guide

This guide to mobile A/B testing with SplitMetrics will cover the following aspects:

  1. SplitMetrics registration;
  2. Adding apps to your account;
  3. Creating an experiment;
  4. Deleting an app;
  5. Account settings.

Mobile A/B Testing with SplitMetrics: Registration

1. Click the “Sign Up For Free” button in the top right-hand corner on SplitMetrics’ home page.

Mobile A/B Testing with SplitMetrics

2. Register by entering your Name and Email in the pop-up window. You can also sign up with Google+ or Facebook.

3. Tap the “Start Your Free Trial Now” button.

SplitMetrics registration

4. Check your Email Inbox. You’ll find a password to log in to your account there.

5. Activate Your Account by tapping the blue button.

SplitMetrics account settings

Note: you’re unable to Add your App unless you verify your Email. You will see the following message (see below), until verification. If there is no SplitMetrics’ message in your Inbox, please, check Spam.

In case you’ve resent verification email and there still no message, let us know about it using the in-app help button found on the bottom right.

SplitMetrics chat button

Mobile A/B Testing with SplitMetrics:  Adding Apps to Account

  1. Click the “Add New App” button in the top right-hand corner of your Dashboard.adding app for Mobile A/B Testing
  2. (a) If your App is already live, Enter URL of your app in the App Store in which you are going to run A/B tests.

For instance, if you want to perform mobile a/b testing in the US store, here are the examples of relevant URLs:

  • App Store https://itunes.apple.com/us/app/xxxxx/id0000000000?mt=8
  • Google Play https://play.google.com/store/apps/details?id=xxx.xx.xx.xxxxxxx&hl=us

Note: 1 App means 1 localization. If you want to perform mobile a/b testing for different localizations, please, Add New App for each.

  1. (b) Tap the “Add application manually” button, if your App is on pre-launch (soft-launch) stage.    
adding app to SplitMetrics account

Note: all information for a live App is uploaded automatically. For App on pre-launch, you should upload Icon and fill in all gaps manually.

  1. Tap the “Add Application” button.
app settings

Well done! Your App is added. Let’s move to Experiment Creation.

Mobile A/B Testing with SplitMetrics: How to Create Experiments

  1. Log into your SplitMetrics Dashboard, in the left-hand navigation find ‘Experiments’ and click the “Create New Experiment” button.
SplitMetrics Mobile A/B Testing
  1. Select the appropriate platform device, the store page and the element you are testing.

Note: landing pages for phones and tablets are different and you should run separate tests for them. Make sure you’re sending iPhone users only to an iPhones experiment when you set up a campaign. Otherwise, we will filter out all ‘wrong devices’.

The same principles are applied with Android phones and Android tablets mobile A/B testing.

Step 1. Experiment Set Up

type of a/b experiment
  1. Pick the Element you’d like to test. There is a list of the elements that you can test with SplitMetrics.
element for testing
  1. Name your experiment. This name you will see only on your dashboard.
  2. (a) If your App is live, the system will automatically choose ‘Go to custom link’, which means users will be redirected from your Landing Page to the App Store/ Google Play in case of clicking ‘Get”.

With ‘Go to custom link’ ­ option, you will be able to add a tracking link on the 3rd step of the experiment if you want to track the number of installs in the real store (after the redirect from the experimental page to your real app store page).

Note: we have integration with AppsFlyer, so the tracking is activated when your experiment is published.

  1. (b) If your App is not live yet (pre-launch), we do advise to pick up “Collect user’s email”. When a user taps the “Get” button on the experimental page, the pop-up message will appear saying that the app isn’t live yet and asking to leave an email address. We provide the list of emails when the experiment is finished. Below, you can find the example of a pop-up message for collecting Emails. 
collecting email in pre-launch

However, even if your App is in the pre-launch phase, you can choose “Go to custom link” and on the 3rd step put absolutely any link you want users to be redirected to, e.g. a page with a list of your other apps, main store page, your website and so on.

using custom link in splitmetrics
  1. Then, click “Next Step” button.

Step 2. Experiment details

  1. In this step you should upload the information that will be the same for all variations. Information is mostly pulled from the store if your app is already live. For apps in pre-launched, you need to add information manually.
editing SplitMetrics experiment

Some elements, such as subtitles, video and featured banner are always uploaded manually. Apple API doesn’t allow us to upload these from the store.

  • For Reviews by rating, you can use either percentage (100% altogether without “%”) or take information from iTunes Connect.
app review settings
  • You can add 1­-3 videos manually. SplitMetrics requires a Video Preview Poster and you need to upload it separately from the video. ‘Video Preview Poster’ is the image that your visitors see while the Video Preview is loading (1-2 seconds).

Note: Recommended specifications

Please keep in mind the following limitations:

  •      Icons Size — 380×380;
  •      Screenshots Size — not less than 960×640 (all screenshots should be of the same size); We recommend using .png, 2048×2732 for portrait, 2732×2048 for landscape
  •      Videos Size: iPhone — 750×1334, iPad — 1200×900, Android — 1920×1080;
  •      File Format — mp4,
  •      File size ~20-30Mb, <30sec.
app description setting
  • The store language is defined for each particular experiment. If you have uploaded an app from the US store, but the language in the experiment is French, your app page elements will be pulled from the US store, but all buttons and subtitles will be in French.
finitinh experiment editing

Step 3. Variations Editing

  1. On this step, we add/delete/edit variations for further mobile A/B testing. We recommend to create 3-4 variations and test one variable at a time.
variations settings

Custom Install URL is for the link generated inside your tracking partner to track the number of actual installs after the redirect to your real App Store page. You generate a link for each variation inside Adjust, Tune, Kochava and etc.

Note: Splitmetrics is integrated with AppsFlyer, so you don’t need to insert the tracking links on the 3rd step of the experiment. The integration is activated inside the Running experiment.

variations for a/b testing
  1. You can publish experiment or save it and publish later. Mind that you are able to make changes only to Drafts. However, if you’ve already published the experiment, we can edit it on the backend.
  2. Once the experiment is published, you will see a pop-­up message providing the link that you have to copy and use in your Ad campaign. We’ll split the traffic equally and randomly between alternatives.
launch of mobile experiment

Congratulations! You’re a master of experiment creation at SplitMetrics.

If you need any help getting started with mobile A/B testing, use the in-app help button found on the bottom right and we’ll arrange a demo or walkthrough.

Mobile A/B Testing with SplitMetrics: How to Delete Apps

If you want to delete an App from your dashboard and add another one, you have to delete all experiments for this App as well.

There are 3 experiment statuses at SplitMetrics: Draft, Running and Finished:

  • Draft is an unpublished Experiment.
  • Running is a published but not finished Experiment.
  • Finished is an experiment which was finished.

Note: you’re allowed to delete only Draft and Finished experiments. Thus, if you want to delete Running experiment, you should finish it first.

When all experiments were deleted, on your dashboard, you should Tap on the App you plan to delete. Then, double check that there is no experiment and in the bottom right-hand corner Tap “Delete” button.

tracking app status

So, you successfully deleted the App.

Mobile A/B Testing with SplitMetrics: Account settings

Tap on Avatar in the top right-hand corner.

SplitMetrics profile settings

You can edit your personal information, add Avatar, or change password in Profile Settings.

SplitMetrics profile details

Billing Info shows you information about your quota and Payment history. You can add/change your Payment Method on your Billing Page.

splitMetrics payment method

In Team section, you can add/delete members to/from your team and change their status.

Note: you’re allowed to add only registered users that have already signed up for SplitMetrics as members of your Team. You can invite them by sending an email (see ‘Send invite’).

SplitMetrics team status

There are 3 statuses of Team member:

  • Owner gives you access to everything. You can see and change billing info and payment method, invite new users, and so on.
  • Editor users can create experiments and launch Facebook campaigns. They can’t change billing info and payment method or invite new members.
  • Read Only users can only read reports. They can’t change account settings or invite new members. They can view and search through all the experiments in your account.

Happy Mobile A/B Testing!