A/B Testing – the Future of Apps?

Hugh r swrveHaving committed to offering Facebook via hybrid web apps, something Mark Zuckerberg eventually called the biggest mistake he ever made, the company backtracked in 2012 on the realisation that they couldn’t offer a comparable experience using HTML5. 

But the downside of this was that they wouldn’t be able to deploy the A/B testing platform they had been building to see which app updates would be most popular.

Facebook made a somewhat unusual announcement last week, unveiling an updated in-house platform built to A/B test native app updates called Airlock. Not exciting news for consumers, but for the marketing world, this kind of evolution is enabling brands and app developers to test their apps in a way they never could before.

In order to understand the difficulties associated with native app testing, we got in touch with Swrve, a San Fran app relationship marketing company, which announced that it had built an off-the-shelf A/B testing product back in December.

“In the web world, we were used to landing page optimisation, HTML5 and web tech, which all give you the ability to change things on the fly,” explains Hugh Reynolds, CEO of Swrve. “But what increasingly came with that were people getting viruses, phished and spammed. It became a scarier place than it used to be.

“Consumers love native apps. They’re effectively packaged software, a walled garden built from the ground up, where an individual knows they can stay safe. Facebook went very quickly from ‘we’re all about HTML5’ to ‘we’re all about native’.”

The app submission and re-submission process is notoriously rigorous on iOS, and while that is less the case on Android, devs still have to force an update to get a new version of the app out. An ‘unfriendly’ thing to do to your users and not a swift way to improve your product.

“To make a native app your own you have to be able to find out what works and what doesn’t work,” Reynolds said. “What Facebook is now doing is separating the program – the base scaffolding – from some of the data that controls it. Instead of baking the data-driven pieces in, it maked them controllable from a server somewhere.”

As Facebook said in its blog post about Airlock, the ability to separate the bones of the app from the rest means they are now able to run up to 15 over-the-air experiments at once, across millions of users. While initially companies are using this capability to fine-tune shadows, spacing and fonts, this is evolving to help companies answer big questions on usability. Reynolds terms this an extension of relationship marketing using real-time machine learning. “Its a pretty new field and a pretty new approach,” he said.

“At its most sophisticated, developers can start using artificial intelligence and machine learning to personalise the app experience. Whether that’s tailored to where you are, what time it is or what kind of user you are, we can hone how the app works for you in real-time. Thats the ultimate goal and is a win/win for developers and users.

“Doing this stuff at scale is pretty scary but the learning works better with more users, using the power of big numbers, so Swrve is trying to build a platform like that that can accommodate that.”