Android Testing: test quickly and easily.

If you are involved in the development of mobile apps, it will
surely not surprise you if I tell you that for years I have met dozens of good
programmers that, however, have spent little (or nothing) on testing their
apps. I will not lie to you; I was one of them for a long time!

Perhaps because they are "lightning projects" and times
are so tight that they do not allow more than "painting screens" as
soon as possible, or perhaps because the possibility of doing manual tests is
so in the palm of our hands (literally in this case) that have led us to
believe testing is a kind of unnecessary luxury...

In any case, the truth is that finding apps with a good test base is not as common as it should be in a
professional development environment**.**

The objective of this post is to make a quick introduction to the implementation of Android app oriented tests,
so that any colleagues wishing to leave that group and take a step further have
a small initial guide.

As this testing thing can be as complex as you like, we will make it
as simple as possible, from developer to developer, without going into many
formalisms and definitions, and broken down into three posts.

In this first post we will make an introduction and start by
reviewing some essential concepts.

In the case of Android apps,
the interface tests are performed through what is known as "instrumented
tests", which require an emulated or actual device.

Due to their characteristics, these tests are also valid for more
complete integration tests (from the interface to the data access layer) and,
in particular, for End-to-End (E2E) tests, given that we run tests on a full
version of the app, even consuming actual services, and run tests involving the
complete system in a very simple manner.

But is it worth it?

We have all inherited projects at some point and, in developing some
new functionality not fully aware of the app's behaviour and its intricacies,
we have put our hands into it with the fear of "breaking something".

For me, that is the basic reason for testing. It is not just for the
peace of mind of your successors, but your own as the project evolves and you
start to forget details of your previous implementations.

Test beneficiaries are largely documented and common to any software
development, so we will not go into details, but these stand out for me in the
world of mobile apps:

• You are forced to
  understand the functionality in detail: even when
  you think you know it, by the mere fact of writing tests you will always end up
  discovering some scenarios had not been fully defined.
• Changes to the environment
  stop being "scary": we will see that E2E
  tests are key.
• You save hours and hours
  of work: although it initially seems costly, in the
  mid-term you are capable of testing tens, hundreds or even thousands of
  scenarios, both in a controlled environment and consuming real services...

In a few minutes. In
addition, you will not depend on the help of peers from other layers of the
system to simulate these scenarios.

• You can use them to
  automatically extract up-to-date graphical information, such as screenshots or videos browsing the app.
• You can run comprehensive
  stability and performance tests.

What should you know before you start?

We shall now look at a series of basic concepts and very important considerations in order to easily
cope with the development of quality tests that add real value to your app.

The Mocks

Simply put, a mock is nothing
more than an "empty" implementation (we will explain this) of an
interface whose inputs and outputs we can control at will, regardless of the
actual implementation we have given to our app.

Mocks allow us to count
the number of interactions with a method/function, verify the type of input or return any kind of data, among many
other things.

Although we can create our own mocks "by hand", frameworks
such as Mockito allow us to easily create powerful mocks from our interfaces,
abstract classes or even non-final classes.

In the case of the latter with certain limitations (it may only
operate with the methods than can be overwritten). Static methods cannot be
controlled either.

There are additional libraries, such as PowerMock, which allows us to get our hands on static methods, but in general it is advisable to avoid this and define an architecture that does not require it, as it normally leads us to certain conflicts between libraries.

Although we will go into more detail later, in order for you to
better understand the subsequent sections, I would like to highlight there is a
way to initialise a mock with Kotlin and mockito-kotlin (a library with utility
functions for Kotlin over Mockito), which is as follows:

val interfaceName: InterfaceName = mock()

Yes, it is that easy.

The architecture

One of the main
difficulties when it comes to including these tests in an app is not the tests themselves, but the architecture. This structure must be sufficiently decoupled to
allow us to control the entire test scenario. Let us see an example:

Suppose we have three layers consisting of five classes
collaborating with each other:

• Presentation layer
  • UserDetailPresenter: presenter of a view that shows the detailed data of a user.
• Domain layer
  • GetUserUseCase: usage case that only allows getting user data based on its ID.
• Data layer

  • UserRepository: class that allows operating with the data of a user, regardless of
    their origin. We can get and edit such data.

  • UserApi: class that allows retrieving or modifying a user's data through
    Web Services.

  • UserDao: class used to retrieve or
    modify a user's data from a local Database.

For this example, the desired behaviour is as follows:

1. The UserDetailPresenter
   requests the user's data by calling the use case GetUserUseCase.
2. The use case, through the UserRepository class, gets the user
   data and returns them to the presenter.
3. The UserRepository
   class attempts to obtain locally stored data using the UserDao class and,
   if these are not available, gets them from the Web Service through the UserApi class. In the latter case,
   before returning the data, it stores them using the UserDao class to expedite future queries.

Well, let us say we want to check the logic in the UserRepository class is correctly
implemented. We should highlight that:

• We are only testing the UserRepository
  class; not the entire flow.
• The other classes must not
  intervene in the test.

Now, imagine that our UserRepository
class looks like this, bearing in mind we are going to simplify as much as
possible, without considering optimisations, errors or threads:

class UserRepository {

   private val userDao: UserDao = UserDao()
   private val userApi: UserApi = UserApi()

   fun getUser(id: Int): User{
       userDao.getUser(id)?.let { user ->
           return user
       } ?: run{
           val user = userApi.getUser(id)
           userDao.storeUser(user)
           return user
       }
   }

   fun updateUser(user: User){
       userApi.updateUser(user)
       userDao.storeUser(user)
   }

}

Assuming it is well implemented, the desired functionality is met, we
have no way to control the entire test context. It is true we could create a
test invoking the getUser(id) method
and verify we are returned a user, but as surely you are already imagining,
this has the following deficiencies:

• We cannot tell if the user retrieval attempt was from the DB or the
  WS.
• In this second case, we cannot tell if the user has been saved for
  future queries.
• We do not know if the returned user is actually the user obtained
  from one of these sources.
• In the event of error, we have no way of telling if the error is
  from the UserRepository class, the UserDao class, the UserApi class or simply from any other additional layer (you will
  certainly depend on more classes to call the WS or DB).

In order to meet the objectives**,
we must isolate the method and control both input and output from collaborating
classes** (UserDaoandUserApi).

Not only this, we need to control how the UserRepository class interacts with them, checking parameters as
well as the number and order of invocations. If this is met, we can test that
the getUser(id) method works as per
requirements and we will (almost) have a unit test.

This is where mocks come into
play, and Mockito is a great tool, as we have already seen. However,
although creating mocks is very easy, because of how the UserRepository class has been defined, it is not possible for us to
"embed" them, or as is colloquially known: inject them.

The problems are adding up: on one hand, we are not working with
interfaces and, on the other, the instances to the collaborating classes are
being initialised in the UserRepository
class itself, meaning we cannot replace them.

This is why a decoupled architecture, beyond its many other
advantages, is essential to implement tests in a simple and complete manner.

Let us see how to fix it:

• We must define interfaces
  between the various layers, so that our UserRepository
  implementation invokes the functions defined in such interfaces regardless of
  what classes implement them.
• We must have a way to define
  what implementation we are going to use in this execution of the UserRepositoryentity.

A simple solution would be the following:

1. The UserDetailPresenter, GetUserUseCase, UserRepository, UserDao and UserApi, become interfaces.
2. These interfaces are implemented in their
   corresponding classes, which we can name for example by adding the suffix "Impl". It would look
   like the following:

Interface:

interface UserApi {
   fun getUser(id: Int): User
   fun updateUser(user: User)
}

Implementation:

class UserApiImpl: UserApi {
   override fun getUser(id: Int) = User(id)
   override fun updateUser(user: User) {} //empty for the example
}

1. The specific implementations used by the UserRepositoryImpl class should not be initialised in the class itself, but must be injected.

• There are well-known libraries for this purpose, such as Dagger, and I recommend using it as it has become a sort of standard.
• In any case, the concept of injection is an independent library design pattern, and it is sufficient for us to ensure that the class in question receives already initialised interfaces from outside, for example through its constructor.

class UserRepositoryImpl(private val userDao: UserDao, private val userApi: UserApi): UserRepository {
   override fun getUser(): User{
       userDao.getUser()?.let { user ->
           return user
       } ?: run{
           val user = userApi.getUser()
           userDao.storeUser(user)
           return user
       }
   }
   override fun updateUser(user: User){
       userApi.updateUser(user)
       userDao.storeUser(user)
   }
}

Now, in our test we can initialise the UserRepositoryImpl class to which we inject our mocks, as follows:

@Test
fun exampleTest(){
   val userDao: UserDao = mock()
   val userApi: UserApi = mock()
   val userRepository: UserRepository = UserRepositoryImpl(userDao, userApi)
  
   //You are ready to go!!
}

The purpose of this section does not include going into detail on
how to implement the test, but I can tell you that now:

• We can mock (force, so to speak) what we want the UserDao interface to return when we call
  the getUser(id) method.
• We can mock what we want the UserApi
  interface to return when calling the getUser(id)
  method.
• We can verify how often and in which order have the getUser(id) or storeUser(user) functions have been called from the various
  implementations.
• We can verify that the User
  object returned is the same we mocked in the UserDao and/or UserApi
  interfaces, unadulterated.

With all this, we now have the ability to verify that the
implemented behaviour is as expected; applying this philosophy to all app layers,
our architecture is ready (pending some touches we will see later) to get to
work.

Threads

Another important consideration to prepare our architecture is that we must have control over the threads to
be created and how the various tasks will be queued.

This is because in the testing context, we will have a single
executor thread and any operations running in dynamically created threads will
do so asynchronously, as would be expected.

However, the test will continue its execution regardless of that asynchronous
code block and the result will, most probably, not represent reality.

Do not worry if you are linked to RxJava or any other library, given
it is common to define the ExecutorService (or any other thread management
interface) to manage the app threads and queue the various tasks in them.

Other libraries provide both asynchronous and synchronous methods,
giving you the choice to carry out your own management.

In any case, one we have control over this Executor, what we must do
in the tests is create a specific implementation and inject it wherever
necessary.

The particularity of this implementation is that it will execute the
tasks in the same thread that invokes it, ensuring that the entire test is run
synchronously and that the responses to be verified are correct.

This is an example of an ExecutorService implementation that, knowing it is used only with the submit method, executes the task in the same thread, as we want:

class TestExecutor: ExecutorService {
   //overriding the other functions with empty body…(never called)

   override fun submit(task: Runnable?): Future<*> {
       task?.run() //just run the runnable block in the same thread, synchronously
       return Mockito.mock(Future::class.java)
   }

Let us understand each other!

Before we continue, allow me to clarify the nomenclature we will use
throughout the rest of this post.

As we have seen, what initially was a class, has now become two: an interface and an implementation. This has happened in all layers.

In order to not have to constantly differentiate them, we will talk
about them as entities (from a design point of view), always assuming that all
entities talk to each other through the interfaces and never directly with
their implementations.

In addition, for simplicity and given that we will always handle the
same example, we shall simplify the references to the various entities as
follows:

• Any reference to "Presenter"
  shall refer to any interaction with the entity UserDetailPresenter.
• Any reference to the "UseCase"
  shall refer to any interaction with the entity GetUserUseCase.
• Any reference to the "Repository"
  shall refer to any interaction with the entity UserRepository.
• Any reference to the "DAO"
  will refer to any interaction with the entity UserDao.
• Any reference to the "API"
  will refer to any interaction with the entity UserApi.

Finally, I am aware that the Spanish term "mockear" is not
correct and, why not say it, sounds very bad, but it is broadly used on a daily
basis in this field; so let me indulge and use it from time to time.

One test per scenario

If you are wondering how to test the functionality described above
in one single test, the answer is you cannot, or at least you should not.

As you can imagine, completely verifying this simple behaviour
requires in reality several different tests under different scenarios, and it
would be advisable to separate them. As an example:

• You can check that when the DAO returns a User, the Repository returns
  the same User, without requesting information from the API.
• You can check that when the DAO does not return a User, the
  information is requested from the API, returning the same User.
• You can check that, in the previous case, before requesting information
  from the API, an attempt was made to get it through the DAO.
• You can see that, in the example above, the DAO storeUser(user)
  method is also called, receiving the same user returned by the API as its
  parameter.
• You can verify the behaviour is correct when errors occur in DAO
  and/or API calls.

  • A possible test would be to verify that the error is propagated as
    it is generated in any of these entities, without controlling it or wrapping it
    in the Repository, if this is the expected behaviour. This would be at least
    two different tests, one per entity.

  • In this example we have not looked at the errors, but it is possible
    to control and record an error when requesting data from the DAO (DB) in a log,
    but would still continue to try to obtain information from the API (WS), in
    order to also verify that in the event of an error in DAO, the information is
    requested from the API and that the Logger class has been invoked with
    appropriate information.

As you can see, you can reach a level of detail as broad as required,
and you will surely come up with several other tests to add to this simple
function.

However, it is also important to rationalise when developing tests,
and we must think about the time available. A good balance must be found.

From my point of view, we must never ignore tests and the main
functionalities must be covered, but we can continue without behaviours that
are less relevant to the functionality, such as if the Logger class has been
called with the correct parameters. In any case, this will always depend on the
project context and the developer's opinion.

Once this test series is ready, we can be reassured that if we and
our colleagues need to modify the function, the previously defined behaviour
will continue to remain intact... And if not, the tests will let you know!

Unit Test

Testing a single method is
actually divided into a set of isolated tests that
we can now call Unit Tests, since each one
tests a very narrow part of the functionality for a well-defined scenario.

One concept to bear in mind when dividing tests is that it must be
sufficiently isolated so that, in the event of any other interaction not relevant to the test does not behave as
expected, the results of this test are not hidden.

I.e. if a prior requirement
has not been met, it must be detected in another specific test for this
requirement, so that we can quickly
identify the real source of the error and does not lead us to believe that
the problem occurs in the current test scenario.

If the order in which certain business logic is carried out is
correctly defined and relevant, tests can also be sequenced following this same
order.

This may be useful for very complex processes, so that when multiple tests fail, you know you need
to correct the first thing that failed and, with a little luck, this will
correct all the latter ones.

However, by default these
tests do not have a specific execution sequence and if sufficiently well
isolated, that will be more than enough.

Nomenclature

Nomenclature is a delicate matter in almost any field of any
language, and as is often said, there are colours for every taste. In any case,
I would like to discuss some guidelines that I have personally found useful
when working with them.

The test name must be very
descriptive, even if it hurts the eyes!

I learned this from a speaker in a TDD course and is something I
agree on. With a professional infrastructure, test execution should not
normally be limited to your machine, but it should be executed on some
Continuous Integration server as well.

Therefore, when a test fails in this context, what little you know
at first is the name of the class and the test function that failed.

This being the case, a name such as "getUserTest()" does not describe what part of the flow
has failed. This means you have to look into the code in detail in order to
detect the error.

Therefore, and following the example used on several occasions in
this post, a name of a test function tested exclusively when a User is
retrieved from the API, it is also locally stored through DAO, could be called "IfUserIsRecoveredFromApiThenUserIsStoredThroughDao".

I know that is not what we are accustomed to, but this is not
production code, but code expressly designed to detect errors as quickly and
efficiently as possible.

This is why, if this test fails, we know without having to read the
code that the specific error is that the User is not being stored locally when
retrieved from the WS and can immediately anticipate the consequences.

Updating tests

There is a possibility that, due to project requirements, the user
retrieval flow may have changed, and it is no longer necessary to store
locally, always calling on the WS.

In this case, the error was to not update the tests, which must
evolve with the production code. It is advisable to regularly execute the tests
locally during development in order to not delay these updates when necessary.

Although it may sometimes be tedious, the truth is that it serves to
refresh what conditions must still be met after this change in requirements.

Implementation context

An important point to take into account is that unit tests and all
other tests performed with the generic
JUnit runner run on the Java Virtual Machine, without full access to the
Android framework, although it does to a reduced version.

This means that in your tests
with JUnit you can reference some Android classes, but cannot make use of them
and calling them may lead to errors. This makes it essential to dedicate some
time wrapping regular Android API classes that you may need to call from your
business logic, such as the Log or Base64 class.

The advantage of using wrappers, as a custom Logger class that in
turn uses the Android Log class, is that you can add additional control to
deactivate calls when these are in a test environment, or you can return
specific values for your tests.

This is a simple example:

class Logger {
   companion object {
       @VisibleForTesting(otherwise = VisibleForTesting.PRIVATE)
       var enabled = true

       fun d(tag: String, message: String){
           if (enabled) Log.d(tag, message)
       }
   }
}

The @VisibleForTesting
annotation allows us to guarantee that the variable that disables logging can
only be modified in a test.

There are alternatives for this, of which I would say two are the
most popular:

• Gradle configuration
  parameter: this allows the Android API, instead of
  releasing an exception on call, to return a default value (0 or null). This can
  solve problems in some tests, but we continue to not have control of what
  happens in these calls.

android {
   //...
   testOptions {
       unitTests.returnDefaultValues = true
   }
}

• Roboelectric: a framework that, through its runner, can make use of the Android API from JVM with a behaviour as close as possible to running on a device, although again, there are some limitations and problems. There is plenty of documentation on the web if you want to know more.

Types of tests

We have already discussed some of the main types of tests available,
but the number of tests by objective, number of elements included, the
execution context and other nuances continue to increase.

As this is just an introduction, allow me to limit myself to the
most common types in the Android app world and, although they can be dealt with
in different ways, tell you how we have done it in the work teams I have been
involved in.

Unit Tests

We have already spoken about these and, these are tests that try a very narrow business logic
within a specific entity. These tests can involve more than one function,
but never more than one class.

As we have seen, if more than one class is involved in a given
functionality (which is usual), we must mock such classes or at least have
sufficient control on how they behave during the test.

When defining them, a good way is to think what would have been done if working with TDD. Again, I urge you to look for more formal definitions on the web, but in summary, you could say that TDD is a development methodology that consists in defining the architecture layers, interfaces between them and, before developing implementations, developing the tests.

In fact, the tests are developed before the business logic of the
various layers. I.e. the tests are
developed without having thought of how the functionality will be developed on
an internal level. This allows defining some tests focused exclusively on
covering a functionality, not covering an already known logic.

Theory says that later, the developer must produce the minimum
implementation that allows it to pass all tests, so as to simplify the
development to merely satisfying these conditions defined in advance, without
"ornaments". This allows defining better tests and simpler code.

TDD has its advantages and disadvantages, it is more useful when
multiple developers are involved and ultimately I think it is something more
advanced that what can be covered in this post. In any case, there is
considerable documentation on the web about it, if you are up to it.

Well, considering this scenario, even if we add the tests after
development, it is still good practice to isolate ourselves as much as possible
of how the implementation was made, treating the entity like a "black
box" and focus on what casuistic or specific scenarios exist and how we
expect the output to be.

Integration Tests

An integration test
involves performing a test to more than one entity
(say classes), generally of different
layers. The objective is not to try each one of them but how they work
together.

As an example, let us say we have proven the Repository propagates
any exception without wrapping it. We have also proven that DAO generates an
exception on requesting a user if the ID is unknown (without defining a
specific type of exception).

However, suppose that the Repository entity is expected by design in
this scenario to propagate to the Presenter a specific "UserException".

In this case, individual unit tests marker by layer are correct, but
on unifying the test between the two layers there is a global requirement not
considered in the DAO entity, which in this case is a specific exception to be
thrown.

In general**, if an integration
test fails, it is because the unit tests by layer were not completely defined,** but
they precisely help us to realise this**.**

As you can imagine, the
integration tests can include as many layers are required, to the point of
testing all app layers as a whole. As this would involve at least in one
Android app, testing also the UI, I personally find it easier to perform this
kind of tests through instrumented tests.

In order to continue to be
considered an integration test and not an E2E test, as we will see in the next section**, it is important to isolate the test from other elements of the
platform,** i.e. not depend on network connections or calls to real services**.**

The best thing to do when
wanting to perform full integration tests within an app is to mock through some
mechanism the result from a network call, so that
the entire behaviour is exclusively delimited to how the app was implemented
and not the state of other layers or services. There are libraries for such
purpose or you can intercept calls directly and force the responses.

End to End (E2E) test

When a test includes the entire flow (all layers) of a system, it is
called End to End test.

This E2E test is also used
for end to end tests within the app context, which would correspond to the
example of "complete integration tests" we have seen above, without
including external dependencies.

This is a personal consideration and certainly could be discussed,
but in the mobile app development content (or any "Front" app), I tend to consider that E2E tests must
include the other system layers, including the API and the "Back".

In summary, an E2E test would be equal to executing the app in a
device (emulator or real) pointing to the real endpoint of a given environment,
and testing a functionality, from when the user interacts with the UI until the
feedback is displayed on screen.

This does not mean that the tests should be carried out manually,
because instrumented tests allow us to do so.

We can automate a sequence consisting of launching a screen, filling
in a field with the user ID, click a button and check that after a maximum of 2
seconds, for example, the remaining user data is displayed on screen.

This test involved all system layers, including those in charge of
retrieving and serving user information from a remote server.

This type of tests are ideal to detect problems to the system as a
whole as would be experienced by a user, without going into details of which
part failed but warning us that we must correct something in the problematic
flow.

Interface (UI) test

These tests are limited to
checking the UI behaves as expected in a given scenario, from a perspective
much more focused on user experience than handing data and requests.

For example, we may have tested through the layers, using the
aforementioned tests, in which the user is obtained as should be from the right
source. However, there may be a requirement that this information be displayed
in a dialog with a blue title showing the user's name.

This allows verifying the UI tests: what is being displayed on
screen as a result of an interaction with the system, either through
initialisation, a click or any other trigger. In this example, we would verify
that the dialog exists, that it has a title, that it is blue and that it represents
the name of the retrieved user.

Again, we must control the
times and depth of tests, prioritising the more relevant tests. In this
example, proving that the elements displayed to the user are correct would have
a higher priority in terms of data and less priority in terms of style
(colour).

In the teams I have worked with, we have rarely dedicated time to
test something style-related, limiting ourselves to checking the information
and, at best, the position (for example if displayed within a dialog, a side
menu, toolbar, button, etc.). In any case, this again depends on the project
context, how closed the designs are and the times it handles.

This type of tests
involves isolating the presentation layer from the rest of the system, so that it is not an E2E test, but rather an integration test
between the view and its presenter.

For this, we could inject a UseCase
mock in the Presenter and control
when a user is returned (and what data), when an error or when we simulate a delay in the response, for example.

Even at the risk of being repetitive, I remind you that these tests are implemented on Android through
the aforementioned "instrumented tests", in an emulator or an actual device (or more).

In addition, we can use certain libraries or services to generate
screenshots and/or videos of these tests, and probably more interestingly run
them on several devices with different sizes and Android versions.

This gives us the ability to, without our intervention, check that
our functionality and graphical experience is met in all tested versions and
screen sizes and, furthermore, have these captures available to verify with a
quick glance that there are no unwanted variations in any of these devices.

We are ready!

Although it is true we have not gone into much detail in any of the
items, I think that for someone who has decided to take the step of getting
started in the development of tests for Android apps, this guide is enough to
become aware of what elements are involved in the tests, what considerations
must be taken into account with regards to the app design, what are the top
tools available and how can we ultimately develop tests without fear.

It is time to get to work... Let us go for the Unit Tests!