Automating play store screenshots for fun and additional testing

Juliane Lehmann / Tue, Apr 12, 2016

Taking screenshots for the play store is boring as hell: several locales, three form factors, setting up content data for the screenshots, possibly varying by locale… For the last release of QuickFit, I decided to automate.

What you get in this post

a complete step-by-step description
in a real project
how to automate taking play store screenshots
and simultanously increase your test coverage (or get started with some UI tests at all!)

And because UI tests are needed and a happy byproduct of the process, it comes with

a very short introduction to Espresso
a very short introduction to UiAutomator
examples for testing RecyclerViews with Espresso
examples for setting up test data

I’ll walk you through my journey on automating the screenshots for the QuickFit app. It’s open source, [on github][github-quickfit] and on the [play store][playstore-quickfit]. [github-quickfit]: https://github.com/strooooke/quickfit [playstore-quickfit]: https://play.google.com/store/apps/details?id=com.lambdasoup.quickfit

But that’s a solved problem! Use screengrab!

Other people have this problem too, and solved this problem too. There’s screengrab, and it is great, and we’re going to use it. What it does for us:

provide a JUnit4 TestRule that sets the locale on the attached device
drives a test suite through a list of locales on a single attached device, transferring the screen shots taken by the test suite from the device to your development machine

What it doesn’t do for us:

take actual system screenshots
drive the multi-locale test suite through three different AVD images for the different form factors
write the tests.

Let’s work on that.

Get started

First, install screengrab with

sudo gem install screengrab

and make sure that your module build.gradle file declares the appropriate dependencies for scope androidTestCompile:

dependencies {
    ...
    androidTestCompile 'junit:junit:4.12'
    androidTestCompile 'tools.fastlane:screengrab:0.3.0'
    ...
}

Now, we need a very basic test.

So, switch your build variant to “Android Instrumentation Tests” and, in the androidTest source set, create a class like

package com.lambdasoup.quickfit.screenshots;

import android.support.test.rule.ActivityTestRule;
import android.support.test.runner.AndroidJUnit4;

import com.lambdasoup.quickfit.ui.WorkoutListActivity;

import org.junit.ClassRule;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.TestRule;
import org.junit.runner.RunWith;

import tools.fastlane.screengrab.Screengrab;
import tools.fastlane.screengrab.locale.LocaleTestRule;

@RunWith(AndroidJUnit4.class)
public class BasicTest {
    @ClassRule
    public static final TestRule classRule = new LocaleTestRule();

    @Rule
    public final ActivityTestRule<WorkoutListActivity> activityTestRule = new ActivityTestRule<>(WorkoutListActivity.class);

    @Test
    public void takeScreenshot() throws Exception {
        Screengrab.screenshot("mainactivity");
    }
}

Now, we need additional permissions in the debug apk to allow screengrab to do its magic. As we don’t want those in all debug builds, let’s solve this through product flavors. In your build.gradle, add a new flavor (and a regular one):

    productFlavors {
        screengrab {}
        regular {}
    }

and create a manifest that gets merged just for this flavor in app/src/screengrab/AndroidManifest.xml:

<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="com.lambdasoup.quickfit">

    <!-- screengrab: https://github.com/fastlane/fastlane/tree/master/screengrab#readme -->
    <!-- Allows unlocking your device and activating its screen so UI tests can succeed -->
    <uses-permission android:name="android.permission.DISABLE_KEYGUARD"/>
    <uses-permission android:name="android.permission.WAKE_LOCK"/>

    <!-- Allows for storing and retrieving screenshots -->
    <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
    <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />

    <!-- Allows changing locales -->
    <uses-permission android:name="android.permission.CHANGE_CONFIGURATION" />
</manifest>

Create a file named Screengrabfile in your project root, replacing package names and locales as appropriate:

app_package_name 'com.lambdasoup.quickfit'
use_tests_in_packages ['com.lambdasoup.quickfit.screenshots']
app_apk_path 'app/build/outputs/apk/app-screengrab-debug.apk'
tests_apk_path 'app/build/outputs/apk/app-screengrab-debug-androidTest-unaligned.apk'
locales ['en-US', 'de-DE']
clear_previous_screenshots true

Now, let’s compile and run! In your project root, run

./gradlew assembleScreengrabDebugAndroidTest assembleScreengrabDebug
screengrab

and enjoy your first screenshots in the freshly generated fastlane subdirectory. Add it to .gitignore.

Wait! screengrab crashed!

Did you run into this issue? It looks like this:

/var/lib/gems/2.1.0/gems/fastlane_core-0.39.0/lib/fastlane_core/command_executor.rb:79:in `execute': [!] undefined method `exitstatus' for nil:NilClass (NoMethodError)

Note your fastlane_core version number (0.39.0 in the example) and revert fastlane_core to version 0.38.0:

sudo gem install fastlane_core -v 0.38.0
sudo gem uninstall fastlane_core -v ${your fastlane_core version}

Wait! That’s not a proper screenshot!

Indeed, it is not. It is the contents of the window of the activity instantiated by the ActivityTestRule. This is currently a known quirk, and the solutions discussed in that thread are appropriate. So, create your own function that delegates screenshot taking to UI Automator, like this and add UI Automator as a dependency for scope androidTest in your build.gradle:

androidTestCompile 'com.android.support.test.uiautomator:uiautomator-v18:2.1.1'

Call your own implementation instead of the screengrab one, compile, run screengrab, and enjoy.

That’s not the screenshot I want! First, some things need to be clicked!

That’s what Espresso is there for! Of course, it pays to read the documentation, but the very very short version is: first, find a view, then either perform an action on it, or assert that it has some property. Espresso will monitor some sources of asynchronicity for you (and you can make it handle more via registering your own idling resource), and so automatically waits for example for loaders to have loaded and so on before checking that a view in fact exists.

Add Espresso dependencies to your build.gradle file:

    androidTestCompile 'com.android.support.test:runner:0.5'
    androidTestCompile 'com.android.support.test:rules:0.5'
    androidTestCompile 'com.android.support.test.espresso:espresso-core:2.2.2'

Animations are a problem, so on your test device, turn off the three categories of animations in the developer options. That does not, in fact, turn off all animations, but it helps.

Add another test to your BasicTest class:

import static android.support.test.espresso.Espresso.onView;
import static android.support.test.espresso.action.ViewActions.click;
import static android.support.test.espresso.matcher.ViewMatchers.withId;
...
    @Test
    public void takeScreenshotAfterClick() throws Exception {
        onView(withId(R.id.fab)).perform(click());
        SystemScreengrab.takeScreenshot("mainactivity_afterclick");
    }

Presto, you probably just added some test coverage too! Feeling inspired by that? Then go and add some view assertions, while you’re at it:

    @Test
    public void takeScreenshotAfterClick() throws Exception {
        onView(withId(R.id.fab)).perform(click());
        onView(withId(R.id.workout_list_empty)).check(
                ViewAssertions.matches(withEffectiveVisibility(ViewMatchers.Visibility.GONE)));
        SystemScreengrab.takeScreenshot("mainactivity_afterclick");
    }

I want canned data for my screenshots, and I don’t want to test entering it all

You want setup that is different for each test case? Put it into an @Before annotated method. You want setup that you can reuse for several test cases? Put it into a test rule, like this:

package com.lambdasoup.quickfit.screenshots;

import android.content.ContentValues;
import android.database.sqlite.SQLiteDatabase;
import android.support.test.InstrumentationRegistry;

import com.google.android.gms.fitness.FitnessActivities;
import com.lambdasoup.quickfit.persist.QuickFitContract.WorkoutEntry;
import com.lambdasoup.quickfit.persist.QuickFitDbHelper;

import org.junit.rules.TestRule;
import org.junit.runner.Description;
import org.junit.runners.model.Statement;

public class SimpleDbTestRule implements TestRule {
    private QuickFitDbHelper dbHelper;

    static ContentValues w1 = new ContentValues();

    static {
        w1.put(WorkoutEntry.COL_ID, 1l);
        w1.put(WorkoutEntry.COL_ACTIVITY_TYPE, FitnessActivities.DANCING);
        w1.put(WorkoutEntry.COL_DURATION_MINUTES, 90);
        w1.put(WorkoutEntry.COL_LABEL, "Fun with music");
    }

    public SimpleDbTestRule() {
        this.dbHelper = new QuickFitDbHelper(InstrumentationRegistry.getTargetContext());
    }

    @Override
    public Statement apply(Statement base, Description description) {
        return new Statement() {
            @Override
            public void evaluate() throws Throwable {
                try (SQLiteDatabase conn = dbHelper.getWritableDatabase()) {
                    conn.delete(WorkoutEntry.TABLE_NAME, null, null);
                    conn.insert(WorkoutEntry.TABLE_NAME, null, w1);
                }
                base.evaluate();
            }
        };
    }
}

Depending on whether you want this executed before each test, or before all tests in a single test case, register it with your test case either as a class rule

    @ClassRule
    public static final RuleChain classRules = RuleChain.outerRule(new LocaleTestRule())
            .around(new SimpleDbTestRule());

or as a rule:

    @Rule
    public final RuleChain rule = RuleChain.outerRule(new SimpleDbTestRule())
            .around(new ActivityTestRule<>(WorkoutListActivity.class));

Sometimes, the screenshots show my RecyclerView only half populated! / I need to interact with a RecyclerView!

Let’s use Espresso to wait until the loader behind it has finished loading. Were it an AdapterView, we could use onData, but it isn’t, and we can’t. But there’s support for recycler views in Espresso, so add another dependency to your build.gradle file (the excludes are necessary, because otherwise espresso-contrib pulls in outdated dependencies on support library modules, leading to conflicts on the classpath during test execution):

    androidTestCompile ('com.android.support.test.espresso:espresso-contrib:2.2'){
        exclude group: 'com.android.support', module: 'appcompat-v7'
        exclude group: 'com.android.support', module: 'support-v4'
        exclude group: 'com.android.support', module: 'recyclerview-v7'
        exclude group: 'com.android.support', module: 'design'
    }

Edit (2017-09-05): I’m not going to update this post with current versions, but by now, I disagree with the conflict resolution by exclude. The in my opinion currently correct way to fix support library versions is described here.

Now you can use recycler view actions like this:

    @Test
    public void recyclerViewAction() throws Exception {
        onView(withId(R.id.workout_list)).perform(RecyclerViewActions.scrollToPosition(1));
    }

There are no recyclerview assertions yet, and performing actions on a descendant view of an item isn’t supported either. So make your own view matcher, or use this one like this:

onView(withRecyclerView(R.id.workout_list).atPosition(0, withId(R.id.schedules))).perform(click());

finds a view matching a view matcher withId(R.id.schedules) that is a descendant of the recycler view item at position 0, then performs a click action on it. We could also check an assertion on it, instead.

Now your tests are better tests, but the problem of not fully displayed item views still persists. I suspect that the recyclerview animations are to blame, and resorted to sleeping for 200 millis before taking the screenshot after all. I’d be very happy about any suggestions on how to do better!

My test data must be localized!

We can use screengrab’s LocaleUtil.getTestLocale to access the locale currently under test, while running under the screengrab script. When executing the tests from Android Studio, null gets returned. So let’s write a wrapper:

public class LocaleUtil {

    /**
     * Not final, change in your test code if necessary
     */
    public static Locale FIXED_LOCALE = Locale.US;

    public static Locale getTestLocale() {
        Locale fromScreengrabScript = tools.fastlane.screengrab.locale.LocaleUtil.getTestLocale();
        return fromScreengrabScript != null ? fromScreengrabScript : FIXED_LOCALE;
    }
}

Now we can do locale dependent test setup like this:

public class SimpleDbTestRule implements TestRule {
    private QuickFitDbHelper dbHelper;

    static ContentValues w1 = new ContentValues();
    static Map<Locale, Map<ContentValues, String>> labels = new HashMap<>();

    static {
        w1.put(WorkoutEntry.COL_ID, 1l);
        w1.put(WorkoutEntry.COL_ACTIVITY_TYPE, FitnessActivities.DANCING);
        w1.put(WorkoutEntry.COL_DURATION_MINUTES, 90);

        Map<ContentValues, String> labelsEn = new HashMap<>();
        labelsEn.put(w1, "Fun with music");

        Map<ContentValues, String> labelsDe = new HashMap<>();
        labelsDe.put(w1, "Spaß mit Musik");

        labels.put(Locale.US, labelsEn);
        labels.put(Locale.GERMANY, labelsDe);
    }

    public SimpleDbTestRule() {
        this.dbHelper = new QuickFitDbHelper(InstrumentationRegistry.getTargetContext());
    }

    @Override
    public Statement apply(Statement base, Description description) {
        return new Statement() {
            @Override
            public void evaluate() throws Throwable {
                try (SQLiteDatabase conn = dbHelper.getWritableDatabase()) {
                    Locale testLocale = LocaleUtil.getTestLocale();
                    
                    conn.delete(WorkoutEntry.TABLE_NAME, null, null);
                    conn.insert(WorkoutEntry.TABLE_NAME, null, w1);

                    for (Map.Entry<ContentValues, String> label : labels.get(testLocale).entrySet()) {
                        ContentValues v = new ContentValues();
                        v.put(WorkoutEntry.COL_LABEL, label.getValue());
                        conn.update(WorkoutEntry.TABLE_NAME, v, WorkoutEntry.COL_ID + "=" + label.getKey().get(WorkoutEntry.COL_ID), null);
                    }
                }
                base.evaluate();
            }
        };
    }
}

and tests like this:

    @Test
    public void recyclerViewAction() throws Exception {
        Locale testLocale = LocaleUtil.getTestLocale();
        onView(withRecyclerView(R.id.workout_list)
                .atPosition(0, withId(R.id.label)))
                .check(matches(withText(SimpleDbTestRule.labels.get(testLocale).get(SimpleDbTestRule.w1))));
    }

I want to show a notification in one of my screenshots!

While Espresso allows interaction with your application code, UI Automator allows interaction with the device in general. The documentation is helpful as always and I really cannot improve upon it as a basic overview. One thing I’d like to stress is that uiautomatorviewer is a helpful tool in general, allowing you to inspect the view hierarchy on a device.

Here’s a very concrete example on

                             @RunWith(AndroidJUnit4.class) style="color:#069;font-weight:bold">public class BasicNotificationTest { @ClassRule public static final RuleChain classRules = RuleChain.outerRule(new FixedLocaleTestRule(Locale.US)) .around(new LocaleTestRule()) .around(new DatabasePreparationTestRule()); private static UiDevice deviceInstance; // just here because we want our activity as the background for the screenshots style="color:#09f;font-style:italic">    @Rule public final ActivityTestRule<WorkoutListActivity> workoutListActivityActivityTestRule = new ActivityTestRule<>(WorkoutListActivity.class); @BeforeClass public static void setUp() throws Exception { deviceInstance = UiDevice.getInstance(InstrumentationRegistry.getInstrumentation()); /* style="color:#09f;font-style:italic">         * make app display the notification you're after style="color:#09f;font-style:italic">         */ // open the notification shade style="color:#09f;font-style:italic">        deviceInstance.openNotification(); } @Test public void takeScreenshot() throws Exception { SystemScreengrab.takeScreenshot("notification"); } // Clean up after this test case, so that the other screenshots won't have the notification in it style="color:#09f;font-style:italic">    @AfterClass public static void tearDown() { Context targetContext = InstrumentationRegistry.getTargetContext(); ((NotificationManager) targetContext.getSystemService(Context.NOTIFICATION_SERVICE)).cancel(Constants.NOTIFICATION_ALARM); } style="color:#555">}

While we’re at it, why not test that the notification actually looks correct? Let’s try out uiautomatorviewer to find out how to identify the view we want, then assert that it at least exists and is clickable (we could also check that it has some specific text):

    @Test
    public void snoozeButtonPresent() throws Exception {
        UiObject snoozeButton = deviceInstance.findObject(new UiSelector()
                .className(android.widget.Button.class)
                .description("Snooze")
                .clickable(true));
        assertTrue("Missing Snooze button", snoozeButton.exists());
    }

Test cases are fine, but I’m still missing the automation!

Indeed. Let’s do some quick and dirty shell scripting. I’ll present the script (complete file) block by block.

We get started by killing all running emulators first. This relieves us from tracking emulator ports and thus making the script much more complicated. I’m away getting coffee anyway while this script runs.

#!/bin/bash

adb emu kill

Now, there are already AVD images lying around for all three form factors, from the bad old times when we had to take the play store screenshots manually. Find out their names (as seen in the AVD Manager, but with spaces replaced by underscores, or by running emulator -list-avds) and ram them into an array, with a second parallel array containing the device type constants used by screengrab:

avd_images=( "Nexus_5_API_23_x86" "Nexus_7_API_23" "Nexus_10_API_23" )
device_types=( "phone" "sevenInch" "tenInch" )

Let’s compile our apks once

./gradlew assembleScreengrabDebugAndroidTest assembleScreengrabDebug

and then for each device type first start an emulator (will run on port 5554 and be named emulator-5554, as we took care to kill all emulators):

for i in "${!avd_images[@]}"
do
  # start emulator
  /opt/android-sdk/tools/emulator -netdelay none -netspeed full -avd ${avd_images[$i]} &
  echo Emulator started, waiting for boot

Now we need to wait until our device has completely finished booting and is sitting at the lock screen. I haven’t found a solution that works under all circumstances (the following fails for example when the emulator image updates) and would be very happy about suggestions! Experiments with other device properties (sys.boot_completed, dev.bootcomplete) did not yield better results than using init.svc.bootanim.

  # wait until adb is connected to device, so that we can issue adb shell commands
  adb wait-for-device
  
  # wait until boot is completed (see [http://ncona.com/2014/01/detect-when-android-emulator-is-ready/] )
  output=''
  while [[ ${output:0:7} != 'stopped' ]]; do
    output=`adb shell getprop init.svc.bootanim`
    sleep 1
    echo ...waiting
  done
  sleep 1

Unlock the lockscreen, run screengrab, kill the emulator and continue with the next device type:

  # unlock lockscreen
  adb shell input keyevent 82
  screengrab --device_type ${device_types[$i]}
  adb -s emulator-5554 emu kill
done

Directions for future work

A collection of test rules that adjust system time, day/night mode, 24-hour/am/pm mode (did I miss something?) would be helpful, and could be realized with UI Automator.

Ultimately, I should use Jenkins to build the project and perform all these tasks, but this quick and dirty solution here got me up and running.

Comments and feedback are welcome on reddit.