A set of TestRules and ActivityScenarios to facilitate UI Testing under given configurations: FontSizes, Locales

Overview

AndroidUiTestingUtils

A set of TestRules, ActivityScenarios and utils to facilitate UI testing/snapshot testing under certain configurations, independent of the framework you are using. It currently supports the following:

  1. FontSizes
  2. Locales

In the near future, there are plans to also support the following, among others:

  1. Orientation
  2. Display
  3. Dark mode
  4. Reduce snapshot testing flakiness

Integration

To integrate AndroidUiTestingUtils into your project:

Add jitpack to your root build.gradle file:

allprojects {
    repositories {
        maven { url 'https://jitpack.io' }
    }
}

Add a dependency to build.gradle

dependencies {
    androidTestImplementation 'com.github.sergio-sastre:AndroidUiTestingUtils:1.0.0-SNAPSHOT'
}

What to keep in mind

  1. For standard UI testing, use TestRules.
  2. For snapshot/screenshot testing any UI component that does not test the full Activity, prefer ActivityScenarioConfigurator to TestRules. That is because:
    • Some TestRules, like FontSizeTestRule, execute the corresponding adb shell commands to change the configuration, and that change might not happen immediately, slowing down your test execution.
    • ActivityScenarioConfigurator launches a default Activity with the given configuration. Therefore, such configuration changes apply to all UI elements that are inflated with the Activity context! Check the examples in Road-to-Effective-Snapshot-Testing to see how to snapshot test any view using the Activity context.

Known issue: As of AndroidUiTestingUtils:1.0.0-SNAPSHOT, pseudolocales (i.e. en_XA and ar_XB) do not work with LocaleTestRule. This will be fixed in the next version

Usage

  1. To use ActivityScenarioConfigurator you need to add the following activities to your debug/manifest
<application
   ...
   <activity android:name="sergio.sastre.uitesting.utils.activityscenario.ActivityScenarioConfigurator$SnapshotConfiguredActivity"/>
   ...
</application>
  1. To use LocaleTestRule, add this permission in your debug.manifest:
<uses-permission android:name="android.permission.CHANGE_CONFIGURATION"
        tools:ignore="ProtectedPermissions"/>

Code examples

Screenshot test examples with Shot from pedrovgs

  1. ActivityScenario
@Test
fun activityScenarioWithLocaleAndFontSize(){
    val activityScenario = 
           ActivityScenarioConfigurator.Builder()
                .setFontSize(fontScale)
                .setLocale(locale)
                .launchConfiguredActivity()
                .waitForActivity()

    val view = 
           waitForView {
                val layout = activity.inflate(layoutId)
                ...
           }

    compareScreenshot(
           view = view,
           name = testName
    )
              
}
  1. Jetpack Compose + ActivityScenario
    In order to test Jetpack Compose views with ActivityScenarioConfigurator, use createEmptyComposeRule(), for instance:
@get:Rule
val composeTestRule = createEmptyComposeRule()

@Test
fun composeWithFontSizeTest() {
    val activityScenario = 
           ActivityScenarioConfigurator.Builder()
                .setFontSize(fontScale)
                .setLocale(locale)
                .launchConfiguredActivity()
                .onActivity {
                     it.setContent {
                          myComposeView()
                     }
                }

    compareScreenshot(composeTestRule, name = testName)
}

TestRule example

@get:Rule
val localeTestRule = LocaleTestRule(locale)

@get:Rule
val fontSizeTestRule = FontScaleRules.fontScaleTestRule(fontScale)

@Test
fun yourTest() {
   // here your UI or snapshot Test
   ...
}
Comments
  • LocaleTestRule not working for the first test only if < API 27

    LocaleTestRule not working for the first test only if < API 27

    Describe the bug LocaleTestRule not working for the first test only if < API 27

    Environment The setup in which the bug is reproducible:

    • Device or Emulator: Emulator
    • API level: < API 27
    • AndroidUiTestingUtils version: 1.0.0
    • Affected Component: Activity (ONLY) -> LocaleTestRule
    • Buggy Configuration: Locale

    Expected behavior It displays the following error:

    Caused by: java.lang.SecurityException: Permission Denial: updateConfiguration() from pid=28072, uid=10065 requires android.permission.CHANGE_CONFIGURATION
    

    although the permission is added in debug/manifest

    Remarks:

    1. Does not happen on API 27+.
    2. Only the first test is buggy. The locale change becomes effective afterwards on every test.
    opened by sergio-sastre 2
  • LocaleTestRule for in-app locale preference, does not reset correctly

    LocaleTestRule for in-app locale preference, does not reset correctly

    Describe the bug LocaleTestRule does not reset correctly. It happens, for example, if the app has no in-app language set. After the test execution, the System Language is set as in-app locale pref instead. This may affect the rest of the tests, since the in-app local has preference over the System Locale.

    Environment The setup in which the bug is reproducible:

    • Device or Emulator: An<
    • API level: Any
    • AndroidUiTestingUtils version: 1.2.1
    • Affected Component: LocaleTestRule
    • Buggy Configuration: In-app Locale pref

    Expected behavior The in-app locale is properly reset

    opened by sergio-sastre 1
  • Documentation error: LocaleTestRule works together with ActivityScenarioConfigurator.ForActivity(), contrary to what is stated in the KDOC

    Documentation error: LocaleTestRule works together with ActivityScenarioConfigurator.ForActivity(), contrary to what is stated in the KDOC

    Describe the bug Documentation error

    It does not work with ActivityScenarioForActivityRule, but it works with ActivityScenarioConfigurator.ForActivity(), as stated in the README.md

    opened by sergio-sastre 1
  • Complex locales are not supported

    Complex locales are not supported

    Describe the bug Complex locales, like "sr-Latn-RS" / "sr-Cyrl-RS" or "zh-Latn-TW-pinyin" cannot be set.

    Environment The setup in which the bug is reproducible:

    • Device or Emulator: Any
    • API level: Any
    • AndroidUiTestingUtils version: 1.1.2
    • Affected Components: FragmentScenarioConfigurator.setlocale(), ActivityScenarioConfigurator.setLocale(), SystemLocaleTestRule, LocaleTestRule, ActivityScenarioForActivityRule(systemLocale = "my_locale"), fragmentScenarioConfiguratorRule(locale = "my_locale")

    Expected behavior The locale should also be set correctly for those languages

    Additional context All the methods/components to set the Locale use the same methods as fastlane/Screengrab for converting strings into Locales. It means, they do not use Locale.forLanguageTag(), which supports such complex locales, but its own Localeutil method. It is worth noticing that Locale.forLanguageTag() only supports IETF locale strings e.g. "en-US" but not ISO locale strings e.g. "en_US".

    Moreover, the pseudolocales "en_XA" and "ar_XB" only exist as ISO locale Strings.

    That is important to add support for complex locales, while still supporting ISO locales (at least as good as till AndroidUiTestingUtils 1.2.1) as well as pseudolocales

    opened by sergio-sastre 1
  • SystemLocaleTestRule failing for the first test ONLY on first run.

    SystemLocaleTestRule failing for the first test ONLY on first run.

    Describe the bug SystemLocaleTestRule does not always work the very first time the Tests run on a device/emulator, silently failing for the very first test ONLY with a SecurityException. Therefore, for that test ONLY is the System Locale not changed.

    Environment The setup in which the bug is reproducible:

    • Device or Emulator: Device or emulator
    • API level: any API
    • AndroidUiTestingUtils version: 1.2.1
    • Affected Component: SystemLocaleTestRule and ActivityScenarioForActivityRule(systemLocale ="my_locale"), which uses SystemLocaleTestRule internally.

    Expected behavior Must change the SystemLocale for every single test.

    Additional context This is happening because the SystemLocalTestRule is trying to change the system locale before the CHANGE_CONFIGURATION permission is granted.

    opened by sergio-sastre 1
  • DisplaySizeTestRule timing out very often on all API levels

    DisplaySizeTestRule timing out very often on all API levels

    DisplaySizeTestRule timing out very often. Using DisplaySizeTestRule#withTimeout(timeout) does not help.

    Device or Emulator: Emulator API level: Any API level AndroidUiTestingUtils version: 1.1.1 Affected Component: Activity (ONLY) -> DisplaySizeTestRule Buggy Configuration: DisplaySizeTestRule Expected behavior: It does not time out

    Remarks: It seems to be an issue with the context. Using instrumentation context seems to fix the issue. Moreover, it seems that sometimes it hangs if the emulator does not have enough RAM (OOM exception).

    opened by sergio-sastre 1
  • FontSizeTestRule timing out from very time to time on API 25+

    FontSizeTestRule timing out from very time to time on API 25+

    FontSizeTestRule timing out from very time to time. Using FontSizeTestRule#withTimeout(timeout) does not help.

    Device or Emulator: Emulator API level: API 24+ AndroidUiTestingUtils version: 1.1.1 Affected Component: Activity (ONLY) -> FontSizeTestRule Buggy Configuration: FontSize Expected behavior: It does not time out

    Remarks: It seems to be an issue with the context. Using instrumentation context seems to fix the issue.

    opened by sergio-sastre 1
  • setOrientation in Activity not always rotating back on API 31 once the test finished

    setOrientation in Activity not always rotating back on API 31 once the test finished

    Describe the bug setOrientation in Activity: when using it, it does not rotate back to the initial orientation after the test finished. That means, the next tests might start with the wrong orientation.

    Environment The setup in which the bug is reproducible:

    • Device or Emulator: Emulator
    • API level: API 31 at least. Could not reproduce it on API 23,26,27,28 though.
    • AndroidUiTestingUtils version: 1.0.0
    • Affected Component: Activity only
    • Buggy Configuration: Orientation

    Expected behavior After the test where we use setOrientation finishes, the layout orientation returns to the one before the test started.

    Reproducible sample Run Road to effective snapshot testing on API 31.

    opened by sergio-sastre 1
  • Release/1.2.1

    Release/1.2.1

    Add support for:

    • Locale in 2 different forms:
      1. App Locale (i.e. per-app language preference)
      2. System Locale
    • Fragments via FragmentScenarioConfigurator
    • Custom themes for Views and Fragments via setTheme()
    • ActivityScenarioConfigurator & FragmentScenarioConfigurator rules for less verbose tests
    opened by sergio-sastre 0
  • Release/1.2.0

    Release/1.2.0

    Support for:

    • Locale in 2 different forms:
      1. App Locale (i.e. per-app language preference)
      2. System Locale
    • Fragments via FragmentScenarioConfigurator
    • Custom themes for Views and Fragments via setTheme()
    • ActivityScenarioConfigurator & FragmentScenarioConfigurator rules for less verbose tests
    opened by sergio-sastre 0
  • release 1.1.0-beta

    release 1.1.0-beta

    • Change Display Size configuration via TestRule & ActivityScenario
    • Enable to increase/decrease timeout for DisplaySizeTestRule and FontSizeTestRule to take effect. Both TestRules change the config via adb. Therefore, this can be helpful on emulator/devices that take too long for the corresponding adb commands to take effect.
    opened by sergio-sastre 0
  • LocaleTestRule for in-app language does not work only on API 33

    LocaleTestRule for in-app language does not work only on API 33

    Describe the bug Only on API 33, LocaleTestRule does not take effect on API 33

    Environment The setup in which the bug is reproducible:

    • Device or Emulator: any
    • API level: API 33 Only
    • AndroidUiTestingUtils version: 1.2.2
    • Affected Component: LocaleTestRule
    • Buggy Configuration: Locale (in-app language, not the system Locale)

    Expected behavior The LocaleTestRule should change the in-app locale and reset it once the test finishes.

    opened by sergio-sastre 0
  • ActivityScenarioForActivityRule flakiness when reseting values like orientation

    ActivityScenarioForActivityRule flakiness when reseting values like orientation

    Describe the bug When setting values like orientation via ActivityScenarioActivityRule, sometimes they are not reset properly. This means, for example, that the next test might start with use the wrong orientation, and its snapshot would be in the wrong orientation, e.g.Landscape instead of Portrait.

    It seems this happens because the activityscenario.close() method is not called once the test finishes.

    Environment The setup in which the bug is reproducible:

    • Device or Emulator: Any
    • API level: Any, but specially on API 33
    • AndroidUiTestingUtils version: 1.2.2
    • Affected Component:ActivityScenarioTestRule
    • Buggy Configuration: Observed with orientation, but others might be affected too.

    Expected behavior All values should be reset when the test finishes

    opened by sergio-sastre 0
  • java.lang.IllegalStateException: No compose hierarchies found in the app.

    java.lang.IllegalStateException: No compose hierarchies found in the app.

    Describe the bug

    java.lang.IllegalStateException: No compose hierarchies found in the app. 
    Possible reasons include: 
    (1) the Activity that calls setContent did not launch; 
    (2) setContent was not called; 
    (3) setContent was called before the ComposeTestRule ran. If setContent is called by the Activity, make sure the Activity is launched after the ComposeTestRule runs
    

    it happens very often on API 33, have also seen it on API 26 but much rarely.

    Environment The setup in which the bug is reproducible:

    • Device or Emulator: Emulator
    • API level: Mainly API 33, rarely API 26.
    • AndroidUiTestingUtils version: 1.2.2 (could not run on API 33 before)
    • Affected Component: Composable

    Expected behavior A clear and concise description of what you expected to happen.

    Reproducible sample If applicable, create a small repo where the problem is reproducible as well as a link to it. You can also fork from Road to effective snapshot testing. That repo has already a working pedrovgs/Shot configuration in place.

    If applicable, add screenshots to help explain your problem.

    Additional context

    1. Might have to do with the fix on 1.2.2 to run tests on API 33, adding androidTestImplementation("androidx.test:core:1.5.0-beta01")
    2. Might be a compatibility problem between createEmptyComposeRule() and ActivityScenario? Check solution here Another option would be not to use createEmptyComposeRule() at all and use a bitmap for comparing the screenshot instead
    opened by sergio-sastre 0
  • ActivityScenarioConfigurator.forActivity().setUiMode() must be called from UI thread

    ActivityScenarioConfigurator.forActivity().setUiMode() must be called from UI thread

    Describe the bug AndroidUitestingUtils crash

    Environment The setup in which the bug is reproducible:

    • Device or Emulator: Emulator
    • API level: Not sure
    • AndroidUiTestingUtils version: 1.2.2
    • Affected Component: Activity
    • Buggy Configuration: UiMode

    Additional context It might happen due to AppCompatDelegates being called before launching the Activity. Needs investigation

    opened by sergio-sastre 0
  • UiMode not reset after calling ActivityScenarioConfigurator.forActivity().setUiMode() or ActivityScenarioForActivityRule with UiMode

    UiMode not reset after calling ActivityScenarioConfigurator.forActivity().setUiMode() or ActivityScenarioForActivityRule with UiMode

    Describe the bug UiMode not reset after calling ActivityScenarioConfigurator.forActivity().setUiMode(). So any other subsequent test (does not need to be Activity, could be Fragment, View, Composable...) keeps that UiMode if not overriden.

    Environment The setup in which the bug is reproducible:

    • API level: Any
    • AndroidUiTestingUtils version: All
    • Affected Component: ActivityScenarioConfigurator.forActivity() and ActivityScenarioForActivityRule
    • Buggy Configuration: UiMode

    Expected behavior The UiMode should be reset once the test finishes.

    opened by sergio-sastre 0
Releases(1.2.2)
Owner
Sergio Sastre Flórez
I make things happen
Sergio Sastre Flórez
Selenium locators for Java/Kotlin that resemble the Testing Library (testing-library.com).

Selenium Testing Library Testing Library selectors available as Selenium locators for Kotlin/Java. Why? When I use Selenium, I don't want to depend on

Luís Soares 5 Dec 15, 2022
A set of AssertJ helpers geared toward testing Android.

AssertJ Android A set of AssertJ assertions geared toward testing Android. Deprecated The support libraries and play services are developing at a rate

Square 1.6k Jan 3, 2023
Write Tests as Examples on the Method-under-test

Write Tests as Examples on the Method-under-test

Paul Methfessel 1 Apr 12, 2022
Android UI Testing

User scenario testing for Android Robotium is an Android test automation framework that has full support for native and hybrid applications. Robotium

null 2.8k Dec 14, 2022
A programmer-oriented testing framework for Java.

JUnit 4 JUnit is a simple framework to write repeatable tests. It is an instance of the xUnit architecture for unit testing frameworks. For more infor

JUnit 8.4k Jan 9, 2023
Android Unit Testing Framework

Robolectric is the industry-standard unit testing framework for Android. With Robolectric, your tests run in a simulated Android environment inside a

Robolectric 5.6k Jan 3, 2023
Android UI Testing

User scenario testing for Android Robotium is an Android test automation framework that has full support for native and hybrid applications. Robotium

null 2.7k Apr 8, 2021
A programmer-oriented testing framework for Java.

JUnit 4 JUnit is a simple framework to write repeatable tests. It is an instance of the xUnit architecture for unit testing frameworks. For more infor

JUnit 8.4k Dec 28, 2022
A sample repo describing best practices for snapshot testing on Android

Road to effective snapshot testing A sample repo describing best practices for snapshot testing on Android. This includes for now: Parameterized Tests

Sergio Sastre Flórez 86 Jan 1, 2023
Turbine is a small testing library for kotlinx.coroutines Flow.

A small testing library for kotlinx.coroutines Flow

Cash App 1.8k Jan 5, 2023
Fixtures for Kotlin providing generated values for unit testing

A tool to generate well-defined, but essentially random, input following the idea of constrained non-determinism.

Appmattus Limited 191 Dec 21, 2022
Morsa: Jetpack Compose UI Testing Framework

Morsa: Jetpack Compose UI Testing Framework Test library to ease UI testing with Jetpack Compose Purpose This library aims to add some useful wrappers

HyperDevs 10 Dec 3, 2022
Lovely Systems Database Testing

Lovely DB Testing This repository provides opinionated testing helpers for database testing used at Lovely Systems. License This plugin is made availa

Lovely Systems GmbH 1 Feb 23, 2022
Project for testing intern candidate simple level

RickAndMorty-TestTask Тестовый проект 'Гайд по мультфильму Рик и Морти' для практикантов начального и продвинутого уровня. Структура проекта Структура

null 0 Nov 18, 2021
Very simple Morse API text translator. Mainly to do some testing with jitpack, API development etc.

Very simple Morse text translator API. Far from finish or even being reliable. Use for single sentence. Mainly to test github dependency for Android

Piotr Dymala 0 Dec 30, 2021
Snapshot Testing framework for Kotlin.

KotlinSnapshot Snapshot Testing framework for Kotlin. What is this? Snapshot testing is an assertion strategy based on the comparision of the instance

Pedro Gómez 157 Nov 13, 2022
Testify — Android Screenshot Testing

Testify — Android Screenshot Testing Add screenshots to your Android tests Expand your test coverage by including the View-layer. Testify allows you t

ndtp 43 Dec 24, 2022
Selenium WebDriver and Appium based Web, Mobile (Android, iOS) and Windows desktop Automation Framework with BDD & Non-BDD implementation support

Selenium WebDriver and Appium based Web, Mobile (Android, iOS) and Windows desktop Automation Framework with BDD & Non-BDD implementation support

null 10 Dec 5, 2022
A tool for scan and replace file content. Instead of global search and replace, it can specify replace scope

ScanReplace.kt A tool for scan and replace file content Instead of global search and replace, it can specify replace scope Usage java -jar ScanReplace

CWKSC 4 Jan 27, 2022