Android Testing: how to perform instrumented tests

Dependencies

These are the dependencies we will add the build.gradle file of the app module for the examples provided below.

androidTestImplementation 'androidx.test:runner:1.1.1'
androidTestImplementation 'androidx.test:rules:1.1.1'
androidTestImplementation 'androidx.test.ext:junit:1.1.0'
androidTestImplementation 'androidx.test.espresso:espresso-core:3.1.1'
androidTestImplementation 'com.nhaarman.mockitokotlin2:mockito-kotlin:2.0.0'
androidTestImplementation 'org.mockito:mockito-android:2.23.4'

In order to continue using Mockito-Kotlin in this kind of tests on
the Android VM, it is necessary to add
the Mockito-Android dependency.

The other dependencies reference the runner, which in this case will be AndroidJUnit4, and Espresso, the framework used for instrumented
tests.

To ensure compatibility of the runner we will use, make sure you have this configuration included in the build.gradle file, referencing the "androidx" package (not the support):

android {
   //...
   defaultConfig {
       //...
       testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
   }
   //...
}

Note: Since the release of Android Jetpack, the various support packages have been retrofitted and included under the androidx package.

The example screen

We will work with a very simple screen, representing the famous MainActivity.

The layout consists of the following elements:

EditText (id: myEditText)
TextView (id: myTextView)
Button (id: myButton)

We will use MVP to structure it, consisting of:

MainView: interface implemented by MainActivity.
MainPresenter: interface implemented by the MainPresenterImp
class.

In addition, we will create a use case:

GetTextUseCase: use case that returns some text based on another text.

We define the following functional requirements:

The initial state must be as follows:
- The EditText must be empty.
- The TextView must show the text "HelloWorld".
- The button must be active
The following should occur when the button is clicked:
- The use case must be called to get some text from the value entered
  in the EditText.
- The value returned by the use case will be returned to the TextView,
  replacing its previous value.
- The content of EditText must be cleared.
- The button must be deactivated, preventing any future click.

For simplicity of this example, we will initialise the presenter during onCreate, although it would be best to inject it. In any case, as
we give it public visibility, we can replace it for our tests.

Simplifying the classes are left as follows:

interface MainView {
   fun getEditTextValue(): String?
   fun cleanEditText()
   fun getTextViewValue(): String?
   fun setTextViewValue(value: String)
   fun disableButton()
   fun isButtonEnabled(): Boolean
}

class MainActivity : AppCompatActivity(), MainView {

   lateinit var mainPresenter: MainPresenter

   override fun onCreate(savedInstanceState: Bundle?) {
       super.onCreate(savedInstanceState)
       setContentView(R.layout.activity_main)
       mainPresenter = MainPresenterImpl(this, GetTextUseCaseImpl())
       myButton.setOnClickListener { mainPresenter.onButtonClick() }
   }

   override fun getEditTextValue(): String? = myEditText.text?.toString()

   override fun cleanEditText(){ myEditText.text = null }

   override fun getTextViewValue(): String? = myTextView.text?.toString()

   override fun setTextViewValue(value: String) { myTextView.text = value }

   override fun disableButton() { myButton.isEnabled = false }

   override fun isButtonEnabled(): Boolean = myButton.isEnabled
}

interface MainPresenter {
   fun onButtonClick()
}

class MainPresenterImpl(private val mainView: MainView, private val getTextUseCase: GetTextUseCase) : MainPresenter {
   override fun onButtonClick() {
       val output = getTextUseCase.getText(mainView.getEditTextValue())
       mainView.setTextViewValue(output)
       mainView.cleanEditText()
       mainView.disableButton()
   }
}

interface GetTextUseCase {
   fun getText(input: String? = "no text"): String
}

class GetTextUseCaseImpl : GetTextUseCase {
   override fun getText(input: String?): String = "This is the UC result for '$input'"
}

Creating the Test class

The skeleton of a Test class for instrumented tests is very similar
to the one we saw for unit tests in our second post.

Again, we have the annotations @Before,
@Test and @After, and we continue to be able to create and inject mocks as
required.

However, in this case we need the specific Runner for this type of tests using a class annotation @RunWith and a field annotation @Rule that allows us to define the Activity we are going to work with:

@RunWith(AndroidJUnit4::class)
class MainActivityUiTest {

   @Rule
   @JvmField
   var mActivityTestRule : ActivityTestRule<MainActivity> = ActivityTestRule(MainActivity::class.java, true, false)

   @Before
   fun setUp(){
       val intent = Intent()
       //Customize intent if needed (maybesome extras?)
       mActivityTestRule.launchActivity(intent)
   }

   @Test
   fun someTest(){
       //...
   }

}

There are alternative ways
to structure an instrumented test and, in fact, the
one created by default on creating a new project is somewhat different.
However, the one I present is the way described in official UI test
documentation.

The ActivityTestRule we
have defined has3 parameters:

The first indicates the Activity class to be executed.
The second indicates if the activity should be configured in "touch
mode" on start.
The third indicates if the activity must be re-launched automatically before
each test.

You may want to set this last parameter as true. For the example, I
thought it best to show you how to manually start the activity in the setUp(), so that we can use a custom
intent.

This allows us to add "extras" and simulate multiple
scenarios depending on the activity input data, when its behaviour depends on
them.

And what if we want to

test a Fragment?

In this case, you should
start the Activity that contains it in the same way. Assuming the Fragment
is loaded at the start of the Activity, you are ready.

If on the contrary, you need to load a different Fragment; getting
to it will depend on your navigation
architecture. You can manually run this navigation or perhaps condition the
Fragment loaded initially depending on the Bundle received in the intent, which
we have already seen is something we can edit for the tests.

In any case, bear in mind you can work with the Activity as much as
you need to prepare the scenario before executing the test, although this
implies a prior navigation step.

Controlling data

When working with UI
tests, it is advisable to isolate the presentation layer from the rest of
layers, which are not really the ones being put to
test**.**

We can see these as
integration tests between the view and its presenter and any other controller
class, such as for example the ViewModel
if you use the architecture components of Android JetPack.

In any case and within this example, it would be enough to mock the use case bridging between the Presenter
and the domain layer, and mock how the data is returned.

As already mentioned, the Presenter that the Activity works with, despite not being injected, is accessible, hence we have the possibility of substituting it in the setUp, just after launching the Activity.

@RunWith(AndroidJUnit4::class)
class MainActivityUiTest {

   @Rule
   @JvmField
   var mActivityTestRule : ActivityTestRule<MainActivity> = ActivityTestRule(MainActivity::class.java, true, false)

   //Collaborators
   lateinit var getTextUseCase: GetTextUseCase

   @Before
   fun setUp(){
       val intent = Intent()
       //Customize intent if needed (maybe some extras?)
       mActivityTestRule.launchActivity(intent)

       val activity = mActivityTestRule.activity
       System.setProperty("org.mockito.android.target", activity.getDir("target", Context.MODE_PRIVATE).path) //needed workaround for Mockito

       getTextUseCase = mock()
       whenever(getTextUseCase.getText(any())).thenReturn("This is the UC mock result")
       val mainPresenter: MainPresenter = MainPresenterImpl(activity, getTextUseCase)
       activity.mainPresenter = mainPresenter
   }

   @Test
   fun someTest(){
       //...
   }

}

We can get the Activity
instance that has been launched from the ActivityTestRule
and, later, substitute the Presenter
for it to use a UseCase mock.

Now, the test scope is
exclusively limited to the behaviour of the view and its presenter, and we
could therefore change the data type returned or raise exceptions and verify
that the screen displays the error to the user as expected.

Note: as at the date of writing this post, operating with the Mockito
version for Android VM (directly or through Mockito-Kotlin) causes an error
indicating we must set the "org.mockito.android.target"
system property. The issue has been registered in the Mockito GitHub and seems
it will be corrected in upcoming versions without having to add it manually,
but in any case, we solve the problem with this line.

How to interact with the view

The major difference between how we work a function for a unit test
and how we do it for UI test comes now.

onView(...).perform/check(...)

In UI tests we first need
to identify the view on which we will operate (first
block) and, later, interact with it (second
block), either executing an action on it or simply checking its state.

Although we may find more complex scenarios, we will almost always
work with these two blocks.

For example, let us assume we want to click on the button in order to later verify other view states. We could achieve this as follows:

@Test
fun someTest(){
   onView(withId(R.id.myButton))   //first block
       .perform((click()))         //second block
}

The first block returns an object ViewInteraction and expects Matcher as its parameter. We use withid (search by id) in the example but
there many others such as withText
that allow you to search a view by its text, either for the String resource ID
or the String itself.

Matchers can be combined in pairs and cascaded, so as to search the view the matches all of them:

onView(
   Matchers.allOf(
       ViewMatchers.withText(R.string.textResId),
       Matchers.allOf(
           ViewMatchers.isDescendantOfA(ViewMatchers.withId(R.id.ascendantId)),
           ViewMatchers.isDisplayed()
       )
   )
)

When we want to validate a condition on this view, just change the second block now using the "check(...)" function, which expects a ViewAssertion as input. It is very easy to create one through the Matchers as follows.

onView(ViewMatchers.withId(R.id.viewId))
   .check(ViewAssertions.matches(ViewMatchers.isDisplayed()))

First test

Back to work!

We will create a test to verify that when the Button is clicked, the text returned by the UseCase is displayed in the TextView.

In addition, we know that the text must be mocked in the setUp: "This is the UC mock result".

@Test
fun whenButtonIsClickedTheUseCaseTextIsShown(){
   onView(withId(R.id.myButton)).perform((click()))
   onView(withId(R.id.myTextView)).check(ViewAssertions.matches(withText("This is the UC mock result")))
}

As you can see, the "withText"
Matcher, just like any other, can be used in any of the two blocks, either
to identify a view or to carry out a
check on it.

On execution, it will ask us to select the device (real or emulated)
and we will see live how the operations configured in each test are carried
out. In addition, you can see how the Activity
is re-launched for each test without preserving any state from the previous
execution.

As it could not be otherwise, the test passed correctly. We will now create a few more:

@Test
fun whenButtonIsClickedTheEditTextIsCleaned(){
   onView(withId(R.id.myButton)).perform((click()))
   onView(withId(R.id.myEditText)).check(ViewAssertions.matches(withText("")))
}

@Test
fun whenButtonIsClickedItIsDisabled(){
   onView(withId(R.id.myButton)).perform((click()))
   onView(withId(R.id.myButton)).check(ViewAssertions.matches(not(isEnabled())))
}

The first checks that the EditText
is empty and the second that the button is not enabled. The notation is quite
descriptive and I think it does not need much explanation.

We will now mix the power of Espresso and Mockito in order to also verify the graphical behaviour, resulting from correctly calling the UseCase (and not, for example, the presenter painting a hard coded value).

@Test
fun whenButtonIsClickedUseCaseIsCalledWithTextFromEditText(){
   onView(withId(R.id.myEditText)).perform(replaceText("Test text"))
   onView(withId(R.id.myButton)).perform((click()))

   val captor: KArgumentCaptor<String> = argumentCaptor()
   verify(getTextUseCase).getText(captor.capture())
   assertEquals(captor.firstValue, "Test text")
}

We can also verify that the initial states set in the requirements
are met on starting the Activity, or how we treat a possible exception thrown
from the UseCase, but as it does not
require any element that we have not already seen in this or any of the previous
posts, allow me to skip over it.

On a graphics level, our example is so simple that there is not much
to test, although of course a more complex view may require some additional
tools.

Related views

Sometimes we need to verify that a view we are going to operate on
is related to another view, such as for example a Toolbar or a Dialog.

This is especially useful if we do not know the ID or simply prefer
to maintain a "pure" black box mentality and perform all searches by
value and not by identifier. Two examples.

First, imagine you want to
verify the "Detail" text of a toolbar being displayed, in order to
verify for example that are seeing a new screen.

For this, we will add a Matcher in the following example indicating that the view we want to work with must be a descendant from another view with the identifier R.id.toolbar:

@Test
fun thisIsATest() {
   //perform some operation over some view...
   onView(
       Matchers.allOf(
           ViewMatchers.withText("Detail"),
           ViewMatchers.isDescendantOfA(ViewMatchers.withId(R.id.toolbar))
       )
   ).check(ViewAssertions.matches(isDisplayed()))

Two**, suppose you want to check that a text "This is a Dialog" is being displayed in a view whose root is a dialog**. We could have something like:

@Test
fun thisIsATest() {
   onView(ViewMatchers.withText("This is a Dialog"))
       .inRoot(RootMatchers.isDialog())
       .check(ViewAssertions.matches(isDisplayed()))
}

In this latter case, we change the scope of the verification to the
root of the view with the text "This
is a Dialog" (inRoot
function), also indicating it must be a dialog.

If the check confirms it is effectively being shown, it means this
dialog with this text is present on screen in one of its elements.

There is a broad variety of cases, but as it is an introductory post
I encourage you to read the documentation and try other options!

Navigation tests

This type of tests are
perfectly valid for verifying that in the event of
various conditions**, the app navigation
performs correctly, both when going forward and when going back**.

After all, we have a complete version of the app install in the
device and we can interact with it as much as we like.

Following the black box principle, a fairly well accepted approach
to verifying you navigated to the right view is not to check the current
activity or fragment but to verify some of its visual elements (for example the
title of the toolbar), and we can perform
this kind of checks with the tools that we already know.

End to End (E2E) tests

In the example we have been using, we made use of the setUp function in order to substitute
the Presenter and mock the UseCase. This created a controlled
scenario in which only the View and
the Presenter were being tested,
without involving the rest of classes and layers that take part in the actual
app.

Well, if we so wish, we cannot create any kind of mock on any
element and launch the app with an intent identical to the one that would be
used in the real scenario, without making any other modifications.

In that case, we would already be working with the actual app, even
with the network calls consumed by the corresponding web services, putting the
whole system to the test.

In short, we can imagine it as if a member of the team manually
installed the app, started it and you start to interact with it in order to
check that everything is OK, only that in this case the process is done
automatically. As you can imagine, the
time we save throughout the project is immense.

We can even take advantage of these tests to take screenshots and
record videos that allow us to later, at a glance, check there are no undesired
mismatches, especially when these tests are executed on multiple devices.

If you are not sure, it will be interesting to know that Android Studio has a very useful for
recording Espresso tests (Run > Record Espresso Test) detecting the interactions that we perform
on a test device and automatically generating a test function with all of them.

Any checks we want to perform must be manually added to the correct
point in the function, but we will have a good skeleton for "simulated
user" testing, in which we may want to perform a very complex navigation
through the app.

Simplifying Kotlin nomenclature

Although it is true that the nomenclature of the various Espresso
functions is in itself very easy to understand, it is also true that in more
complex view we may have functions with too much "boilerplating", when in reality the key elements are very few.

This is accentuated for navigation tests or the other examples we
mentioned in the previous section, where you may want to simulate dozens of
interactions in a single function.

Thanks to Kotlin and, specifically, a mixture of its infix and
extension functions**, in the last project
I worked in, we defined functions that considerably simplified the reading of
this type of tests**.

I think they can be very useful, so let me show you some examples so
that you will be able by yourselves to create all the ones you need.

Utility functions:

infix fun Int.perform(action: ViewAction) {
   onView(ViewMatchers.withId(this)).perform(action)
}

infix fun Int.checkThat(matcher: Matcher<in View>) {
   onView(ViewMatchers.withId(this)).check(ViewAssertions.matches(matcher))
}

infix fun Int.checkThatTextIs(text: String) {
   onView(ViewMatchers.withId(this)).check(ViewAssertions.matches(withText(text)))
}

infix fun Int.replaceTextWith(text: String?) {
   onView(ViewMatchers.withId(this)).perform(ViewActions.replaceText(text))
}

Refactoring the tests (we also import .R.id.*), we have something as follows:

@Test
fun whenButtonIsClickedTheUseCaseTextIsShown() {
   myButton perform click()
   myTextView checkThatTextIs "This is the UC mock result"
}

@Test
fun whenButtonIsClickedTheEditTextIsCleaned() {
   myButton perform click()
   myEditText checkThatTextIs ""
}

@Test
fun whenButtonIsClickedItIsDisabled() {
   myButton perform click()
   myButton checkThat not(isEnabled())
}

@Test
fun whenButtonIsClickedUseCaseIsCalledWithTextFromEditText() {
   myEditText replaceTextWith "Test text"
   myButton perform click()

   val captor: KArgumentCaptor<String> = argumentCaptor()
   verify(getTextUseCase).getText(captor.capture())
   assertEquals(captor.firstValue, "Test text")
}

You can create as many functions of this type as you want, depending
on the type of most common operations in your tests. In the end, you will have
something as simplified and easy to read as we have seen.

State and thread management

We will try to answer the following question**, how do instrumented tests actually work?**

The truth is that when we are executing one of these tests, what
really happens is that the test app is being installed with an additional app,
which is in charge of executing your app and operating on it to run the tests
(we shall call it the controller app).

This is a very important consideration, given it indicates we have
an app (test app) with its own Main Tread, and the controller app, with a
different Main Thread.

There are two things to
consider when we perform any operation on a view through
Espresso:

The controller app waits for the
Main Thread of the test app to be idle and does not continue until this
is the case.
After detecting this state, the controller app queues the task in
the Main Thread of the app being tested and, as expected, continues its
executions.

With regards to the first point, there are certain elements that can
lead the test app to never reach that state, such as the animations (for
example a ProgressBar rotating continuously waiting for something to happen).

This may cause the test to
stop at a certain point and end failing. From what
I have seen, this type of blocks vary even depending on the Android version we
are testing with.

Hence, the Espresso documentation advises we deactivate any device animations we use in order to run the
tests.

There are some automated proposals to deactivate animations before
executing a test (Gradle configurations or certain Rules) that are not quite
producing the expected results in all devices, and the truth is that the only
"infallible" solution is to follow the advice provided in the
official documentation and manually deactivate all animations.

With regards to the second point, as these are two processes running
asynchronously**, it is normal to find
Espresso performing some checks on a view without it having given enough to
time complete the execution of a task in the test app**.

Following this example, clicking on the button calls the UseCase, which runs a certain process in
background. The response is received when this process completes and the TextView displays the result. Once this
flow is finished, the button is disabled.

For the test, we perform a "perform
click" on the button and immediately after check it is not enabled. It
is very likely the test will fail because the button has not yet been disabled
when checking this condition.

In order to try to control
this asynchronous execution, Espresso provides the IdleResources, which you can configure a
wait period to continue with subsequent actions and checks.

There are also other patterns based on retries with timeout or even
adding custom processes in the architecture.

In any case, as this is an introductory post, I think it is enough
to understand this and be aware of these possible failures or apparent
incoherencies. If you find yourself in this scenario, you know where to start
looking.

Ready, set, go!

By now we can say you are fully initiated in this Android app
oriented testing thing, with knowledge ranging from the initial design of the
app architecture to the development of unit, integration, UI and E2E tests.

If you have really been able to put all these concepts and tools to
the test, which as we have seen are not that many, I assure you that you are more than ready to face the vast
majority of tests you will need in any of your developments.

It is now up to you to put this knowledge into practice, expand on
them and become a real pro! Good luck!

Jorge Gironda

I'm an Android Developer by profession for about 5 years, although I was already messing with the development of Android applications on my own since its first versions, when I was still in college. I'm passionate about everything to do with technology and particularly robotics. As hobbies, I don't miss any Real Madrid game and I love the series.

View more of Jorge.