Google Store Listing Experiments for mobile apps

Dave Bell, Co-founder and CEO of Gummicube, looks at how app developers can use A/B testing to establish what elements are driving conversions in the Google Play Store. 

With millions of apps for users to choose from in the Google Play Store, continuously updating your app listing is essential to staying relevant. A/B tests are a tried and trusted way of evaluating different page variants to determine what elements drive conversions over others. Google’s Store Listing Experiments give you the opportunity to test different variables and pinpoint the exact strategy that resonates with your audience.

Below you can find information on what Google Store Listings Experiments are and how you can use them in your mobile app marketing plan to understand your audience more and increase conversions.

Understanding Google Store Listing Experiments
Google Store Listing Experiments are Google’s proprietary A/B testing tools that you can leverage to evaluate different Play Store page variants. The primary objective of running A/B tests through Experiments is to optimize your app listing through different text and graphics combinations and understand what garners more downloads.

Google gives you two different types of experiments that you can run, each aimed at positioning your app in front of different audiences and testing distinct variables. Google offers default graphics experiments and localized experiments as its two main A/B test options. Depending on your A/B testing objectives you’ll pick one or the other.

Default Graphics Experiments
If you’re looking to run A/B tests in your app listing’s default language and only test your creative set, default graphics experiments may be for you. These experiments allow you to test all of the creative elements found on your Play Store listing. From your app icon and screenshots to your feature graphics and promo video, experiments give you the creative liberty to try out different combinations in search of a winning store listing.

Opting to run A/B tests through default graphics experiments could be a recommended option if you want to focus on your primary target market or test adjustments locally before pushing them to your wider global audience.

Localized Experiments
Suppose testing your creatives simply isn’t enough, or perhaps you’d like to test your store listing in different languages as well. Localized experiments will be your best bet for examining additional elements not available through default graphics experiments.

Through localized experiments, you can target up to five different languages your app is available in and modify not only your app creatives but the descriptions as well. This provides a great opportunity to test different marketing messages without needing to update your existing Play Store listing and target audiences in other territories.

Localized experiments are also your only alternative for testing text elements – meaning that any copy variations you would like to evaluate must be run through a localized test no matter the number of languages your app is available in.

Analyzing results
Once you publish your experiments, Google will begin to collect data and provide metrics for you to follow and evaluate your A/B tests. A good rule of thumb is to let your experiments run for at least seven days before making any changes. Giving the Play Store enough time to position your listing variants accordingly and allow users to engage with them will provide more accurate user behavioral insights as your test progresses.

Google will give you its own report based on the results of your experiments and will compare your A/B test to your current Play Store listing. Within your Play Console, you’ll be able to see how your test is going with Google notifying which of your test variants is performing best, if it is performing better than your current listing, or if more data is needed.

You may then receive recommendations based on your A/B test results. Google will give you the option to apply the winning variant, select among several winning ones, or leave your test to collect more data if more information is needed.

ASO best practices for Experiment optimization
To run an effective Google Store Listing Experiment the variables you’re looking to test should be optimized following ASO best practices. A good start would be to analyze your current Play Store listing before running an experiment for areas of improvement. While an updated screenshot set could positively influence your conversion rate, if your app icon, title, short description, and other metadata and creative elements aren’t optimized, you’re leaving valuable fields unpolished.

Below find how you can optimize the fields directly pertaining to your experiments to effectively run A/B tests on Google’s native platform.

App Icon – As the first creative found on your Play Store listing, it should immediately stand out. Whether running a test or not, your app icon is the first visual impression users have when laying eyes on your listing, so creating one that attracts attention and relates to your app features is the recommended strategy to start your app discovery journey on the right foot.

App Short Description – Having a strong one or two-liner for your short description is an effective way of providing just enough information to interest a user in learning more about your app. Testing different short description variants in your localized experiments are a great way of seeing which sentence combinations and keywords cause a user to continue reading or even download your app directly.

App Description – Another Play Store field you can test in localized experiments, your app description is where you provide in-depth app information through strategic keyword placement and sentence structuring. You can test a plethora of different strategies in your description – from highlighting certain features over others to trying out a new tone of voice you’re curious about implementing.

App Feature Graphic – Struggling to get users to view your promo video? Maybe your feature graphic is to blame. Your feature graphic acts as a thumbnail just before your promo video is played and is, in many cases, the first image someone sees. An optimized feature graphic highlighting popular characters or app value propositions can prompt a user to play your promo video and keep them engaged. 

App Promo Video – As one of the most important creatives in your listing, a promo video can be the defining factor in what causes users to download or alternatively be fluff taking up your first screenshot slot. An optimized video ideally conveys your most important app information in a way that’s highly engaging and quickly consumable. A short, sweet, and informative video is a great way of attracting users and increasing conversions.

App Screenshots – Another powerful conversion tool found on your Play Store listing, screenshots can provide users with valuable app information while also being visually engaging. Keeping your screenshots consistent with brand colors, highlighting important features, and mentioning promotions are all recommended ASO strategies you can use to make sure your screenshots not only stand out but provide value to users.

New capabilities – Custom Store Listings
In May of 2022, Google expanded its Custom Store Listing coverage by 10x – from 5 to 50. Developers can now create Custom Store Listings for a given territory or installation state, each containing custom language they want that particular market subset to see.

For Google Play Store Listing Experiments, this opens the door for significantly more testing opportunities.

For instance, a developer may have a Custom Store Listing set to serve all users located in Japan. Within that subset, they have established custom English US and Japanese graphic and text assets they want users in Japan to see. This can be separate from the Custom Store listing on English US and Japanese established for other territories in other Custom Store Listings. Running an Experiment through these custom pages allows developers to see how users in a given territory convert differently, as opposed to a global conversion metric for a given language.

In the past, this was limited to five territories. For apps that are to scale worldwide, or those simply interested in understanding behavioral differences in priority market segments, 10 times the amount of insights can potentially be gathered.

Keep in mind that traffic volume is a key factor when setting up tests this way. While this setup is ideal for a territory receiving a steady stream of traffic, territory and language combinations without enough traffic volume may not accrue enough data for an actionable test finding.

Before testing, use Google Play Developer Console to understand where your traffic is – and where it isn’t. Implementing a successful ASO strategy to increase traffic in a priority territory is a must before using Google’s expanded Custom Store Listing capabilities.

Summary
Google Store Listing Experiments can be effective in providing insight as to what Play Store page variants yield better results. Continuously evaluating different creative sets and description possibilities will help you get closer to understanding your target audience’s behavior so you can stay one step ahead of the competition. In conjunction with Google’s latest expanded Custom Store Listing capabilities, Store Listing Experiments allow for even further fine-tuning of Play Store Performance.

Your app won’t optimize itself!. Use Google’s native A/B testing platform today to stay ahead of the competition and apply the ASO recommendations detailed above to ensure long-term Play Store growth.

About the Author
Dave Bell is Co-founder and CEO of Gummicube. Gummicube is a global leader in App Store Optimization with more than 12 years of experience optimizing and marketing apps. We offer the leading enterprise ASO technology and agency services, providing support to clients around the world. Our company is trusted by thousands of enterprise brands and leading startups including Microsoft, LinkedIn, Bethesda, SWEAT, GrubHub, McAfee and many others.