A/B testing is an extremely powerful way to increase conversions. If you own a website or a mobile application, you have no excuse not to use A/B testing. Indeed, it is extremely easy to implement. There are many specialized tools online that can help you to set up your A/B testing campaign. This blog article will give a general overview of this increasingly popular marketing technique, especially in app marketing. We’ll also give you our selection of 5 testing tools.
What is A/B testing and why should you use it?
A/B testing is a MUST in terms of app marketing. Whether you’re an indie developer or an app marketer, it is absolutely necessary for you to test, test and test again your app and any changes you’re willing to make. Good news for you: it is really easy to do. Plus, it guarantees an increase in conversions if well executed. And, isn’t it what you want?
This discipline is also knows as split testing, online controlled experiments or randomized controlled experiments. The goal is to identify which design or content changes within an app will actually bring more conversion or engagement from users.
As the name implies, A/B testing is a way to test your product. Two versions (A and B) are compared simultaneously. They are almost identical except for one variation. Their impact on consumers’ behavior regarding both versions is then precisely measured and compared in order to determine which version is the most effective one.
In the case of a mobile app marketing strategy, 50 % of the users will be confronted to the version A while the other 50% will be exposed to the version B. Of course, you can split your audience in various ways (60%-40%, 70%-30%, etc.). Regarding your success metric predetermined (number of downloads, amount of conversion, % of interaction, level of engagement…), you will be able to choose the “winning version” and use it afterwards for everyone.
A/B testing is mostly used for design and ergonomic. Screenshots are often subject to A/B testing in mobile app marketing. But also names, icons, descriptions, call to actions, sizes, colors, disposition, headlines, pricing, landing pages, ads, etc. Basically, anything can be tested! It only depends on what your marketing objectives are and where you are willing to make a change.
5 Useful Tools for Mobile App A/B Testing
There are more and more online platforms or services specialized in A/B testing. This profusion might overwhelm and discourage some of you to undertake a campaign. You can rapidly be confronted with too many different choices and get lost. That’s why AppTweak has decided to give you a hand! We picked for you our 5 favorite A/B testing online tools.
“Optimizely is the #1 website optimization platform in the world”
Optimizely is the most famous A/B testing tool. It was founded in 2010 by two former Google product managers: Dan Siroker and Pete Koomen. Working for Obama’s 2008 presidential as the Director of Analytics, Dan became an A/B testing expert. After the campaign, he decided to create an online easy-to-use optimization platform as he noticed that this marketing discipline was quite hard to implement back then. He therefore teamed up with Pete and worked hard on building this innovative platform that we know today as Optimizely.
Since recently, Optimizely offers a special service for mobile apps optimization. It includes a code-free visual editor allowing you to make any visual changes you wish without having to wait for App Store review (instant rollout). It’s all very fast and easy to install. Based on WYSIWYG (What You See Is What You Get) editing and real-time reporting, Optimizely is currently only available in beta version and for iOS apps.
Pricing: Various plans are available starting from $17/month to $359/month. A custom enterprise plan is also available. Free 30 days trial.
“Mobile A/B Testing Made Simple”
Apptimize is another popular A/B testing online platform. Unlike Optimizely, Apptimize is specialized in mobile applications and is available for both iOS and Android apps. Founded in 2013 by Nancy Hua and Jeremy Orlow, the company allows app developers to test various versions of their apps while having a single app listed in the App Store or Google Play Store.
It is also based on a WYSIWYG editor called Visual Apptimizer. With Apptimize, you can easily create A/B testing campaigns and make changes in real-time without needing to resubmit to the App Store.
Apptimize also features flagging which enables you to target some of your changes to certain users only. You can also hide any features you want from any users you want.
Unlike Optimizely, Apptimize requires you to do some coding in order to get it work. But if you can code, this tool will be very easy to use for you.
Pricing: Various plans are available starting from free (the free plan is already a very good start with 25,000 Monthly Active Users + basic A/B testing features) to $300/month. A custom enterprise plan is also available.
“The Most Powerful A/B Testing Platform for Mobile Apps”
Leanplum is another very simple A/B testing tool, especially if you’re an app marketer or product manager. Like Optimizely, no code is required to get started.
The company was founded by Momchil Kyurkchiev and Andrew First who both worked at Google optimizing video ads. They realized that mobile A/B testing was too complicated and that it was lacking of an intuitive tool. That’s how they come up with the idea of launching Leanplum.
Pure WYSIWYG A/B testing tool, Leanplum delivers quick results (real-time), offers flexible A/B testing, powerful analytics and advanced segmentation. No App Store approval is required.
Leanplum is available of every platform: iOS, Android, HTML5, Unity and Rest API; unlike Apptimize and Optimizely.
Pricing:Various plans are available starting from free (500 daily active users) to $399/month (10,000 daily active users). A custom enterprise plan is also available.
“Your mobile A/B testing solution”
Arise.io is a French company based in Paris that was created in early 2013.
Arise delivers an effective A/B testing tool but it is necessary to have coding knowledge in order to use it. It offers real-time results and does not require any App Store approval. Segmentation was added in early 2014. It can be used for iOS and Android apps.
Pricing: __Various plans are available starting from free (5000 tested users) to $299/month (10,000 daily active users). A custom enterprise plan is also available.
“Mobile A/B Testing for iOS, Android and Unity Apps”
Founded in 2013, Splitforce is a very complete A/B testing tool. You can get it started with 6 quick steps that can be completed within few minutes. You can test texts, numbers, colors, switches and other custom elements. You can also set any goal types you want (conversion, event, time or quantity goal).
Splitforce also enables personalization (sample size, targeting and localization). It is available on iOS, Android and Unity platform.
Pricing: Various plans are available starting from $20/month to $500/month. A custom enterprise plan is also available.
This is of course a non-exhaustive list of all the A/B testing tools available online. There are actually tons of them. However, these are the ones we picked out for you; trying to find various offers to match different profiles.
The best way to chose your A/B testing tool is to select the one that will suit you the most: one that is easy to understand and to use. Preferably one that does not require any coding if you can’t code and are rushed to get quick results.
Try to see which tool is the most suitable for your app and your audience. Don’t necessarily go for the most sophisticated tool; some basic features may be enough in your case.
A/B Testing Case Study – SoundCloud (Presented by Moritz Daan at the APS London)
The AppTweak team attended the App Promotion Summit in London on Thursday 10th of July. Olivier Verdin and Sebastien Dellis, the two co-founders of the company were thrilled to meet passionate people and assist to impressive presentations.
Among others, an interesting conference held by Moritz Daan retained their attention. As a Growth Channel Specialist at SoundCloud and responsible of the App Store Optimization of the app, Moritz showed his audience the importance of A/B testing in his work and how successful it can be.
The A/B testing campaign he directed was testing several elements of the SoundCloud app: the icon, the title, the description and the screenshots.
This optimization campaign actually brought an increase of +10% conversion.
So… app developers and marketers… are you now convinced by the importance of A/B testing? Nowadays, the competition is fierce in the apps world. You need to use all available opportunities to optimize your app. Even if you’re satisfied with the number of downloads… Don’t forget that App Store Optimization is a never-ending process and that trends are evolving very quickly.
What do you think about A/B testing? Did you already try this app marketing analytics technique? What is your favorite tool? Feel free to share your opinion in the comment section below, thanks!
By Laurie Galazzo