How to A/B test screenshots, step by step
Follow these steps in order. Each one is something we do on client work.
- Write a hypothesis. "Leading with the outcome will beat leading with the feature list."
- Change one thing. Usually screenshot one or the caption order.
- Set up the test. Product page optimization on iOS; store listing experiments on Play.
- Run long enough. Wait for the console to report confidence, and cover full weeks.
- Apply the winner. Then write down what you learned.
- Plan the next test. Build on the result.
A/B test App Store screenshots: key rules and context
Pick a hypothesis. Good tests: caption style, first screenshot, light vs dark, frame vs frameless, with or without social proof. Bad tests: two random designs.
Run it
- iOS: product page optimization.
- Android: store listing experiments.
Read results. Don't stop early. Seasonal changes and featuring can skew results. Keep a log of hypotheses and outcomes.
Traffic. Low-traffic apps should improve metadata first (ASO) to get enough visitors.
Worked example (hypothetical)
This example is illustrative. The app is made up, and so are any numbers in it.
Say SpendWise tests screenshot one: "See where your money goes" (current) versus "Save more each month" (treatment). If the console shows the treatment winning with confidence, we'd apply it and next test the second screenshot.
What to test first
| Element | Why |
|---|---|
| Screenshot one | Seen in search results and decides most visits |
| Caption wording | Cheap to change |
| Order of screenshots | Benefit priority |
| Background colour | Stand-out in search |
Common mistakes
- Testing many changes at once. No clear lesson.
- Stopping early. Noise looks like a win.
- Testing in low-traffic markets only. Results take ages.
- Not recording results. Teams repeat old tests.
For more on this topic, our ASO guides hub collects the rest of this series.
Native testing tools compared
Use the store's own test tool before any third-party guesswork.
| Apple product page optimization | Google Play store listing experiments | |
|---|---|---|
| What you can test | Icon, screenshots, previews | Graphics and text (icon, screenshots, descriptions) |
| Variants | Up to 3 treatments vs original | Up to 2 variants vs current listing |
| Localization | Treatments can be localized | Run per language (localized experiments) |
| How winners are judged | Estimated conversion lift with confidence | Performance vs current listing over the test |
Screenshot specs we design to
Both stores publish exact requirements. These are the ones that matter most for a phone-first app.
| Store / device | Size or rule | Count |
|---|---|---|
| App Store, iPhone 6.9-inch | 1320 × 2868, 1290 × 2796 or 1260 × 2736 px (portrait) | 1–10 per device size |
| App Store, iPad 13-inch | 2064 × 2752 or 2048 × 2732 px (portrait) | 1–10 per device size |
| App Store, format | .jpeg,.jpg or.png | — |
| Google Play, phone | JPEG or 24-bit PNG, 320–3,840 px, long side no more than 2× the short side | Minimum 2 to publish, up to 8 per device type |
| Google Play, Chromebook / tablet | 1,080–7,680 px, 16:9 or 9:16 | Minimum 4 recommended |
How we measure the result
For A/B test App Store screenshots, success is measured on conversion, not looks. Store listing experiments in Play Console compare your variants against the current listing. A polished set that doesn't lift conversion is still worth replacing.
Checklist
Copy this list into your release notes or ticket and tick each item off before you ship.
- Hypothesis written
- One variable
- Test configured
- Full weeks covered
- Winner applied
- Learning logged
Sources
- Apple: Product page optimization (accessed 26 October 2026)
- Google Play Console Help: Run store listing experiments (accessed 26 October 2026)
- Apple App Store Connect Help: Overview of product page optimization (accessed 26 October 2026)
- Apple App Store Connect Help: Screenshot specifications (accessed 26 October 2026)
- Play Console Help: Add preview assets to showcase your app (accessed 26 October 2026)
Questions
How long should a screenshot test run?
Until confidence, usually at least 1–2 weeks.
Can I test more than two sets?
Apple allows up to 3 treatments.
Test on both stores at once?
Yes; results can differ.
Do I need a new build?
Not for screenshots.
Can you make both sets?
Yes.
How long should a screenshot test run?
Until the console reports a confident result, and for at least a full week to cover weekday and weekend behaviour.