3.2: Testing Apps With Unit Tests

Contents:

Testing your code can help you catch bugs early on in development — when they are the least expensive to address — and improve the robustness of your code as your app gets larger and more complex. With tests in your code, you can exercise small portions of your app in isolation, and in an automatable and repeatable manner.

Android Studio and the Android Testing Support Library support several different kinds of tests and testing frameworks. In this practical you'll explore Android Studio's built-in functionality for testing, and learn how to write and run local unit tests.

Local unit tests are tests that are compiled and run entirely on your local machine with the Java Virtual Machine (JVM). Use local unit tests to test the parts of your app (such as the internal logic) that do not need access to the Android framework or an Android device or emulator, or those for which you can create fake ("mock" or stub) objects that pretend to behave like the framework equivalents. Unit tests are written with JUnit, a common unit testing framework for Java.

What you should already KNOW

From the previous practicals you should be familiar with:

  • How to create projects in Android Studio.
  • The major components of your Android Studio project (manifest, resources, Java files, gradle files).
  • How to build and run apps.

What you will LEARN

  • How organizing and running tests works in Android Studio
  • What a unit test is, and how to write unit tests for your code.
  • How to create and run local unit tests in Android Studio.

What you will DO

  • Run the initial tests in the SimpleCalc app.
  • Add more tests to the SimpleCalc app.
  • Run those unit tests the see the results.

App Overview

This practical uses the same SimpleCalc app from the last practical. You can modify that app in place, or copy your project into a new app.

Task 1. Explore and run SimpleCalc in Android Studio

You both write and run your tests (both unit tests and instrumented tests ) inside Android Studio, alongside the code for your app. Every new Android project includes basic sample classes for testing that you can extend or replace for your own uses.

In this task we'll return to the SimpleCalc app, which includes a basic unit testing class.

1.1 Explore source sets and SimpleCalc

Source sets are a collection of related code in your project that are for different build targets or other "flavors" of your app. When Android Studio creates your project, it creates three source sets:

  • The main source set, for your app's code and resources.
  • The test source set, for your app's local unit tests.
  • The androidTest source set, for Android instrumented tests.

In this task you'll explore how source sets are displayed in Android Studio, examine the gradle configuration for testing, and run the unit tests for the SimpleCalc app. You'll use the androidTest source set in more detail in a later practical.

  1. Open the SimpleCalc project in Android Studio if you have not already done so. If you don't have SimpleCalc you can find it at this download link.

  2. Open the Project view, and expand the app and java folders.

    The java folder in the Android view lists all the source sets in the app by package name (com.android.example.simplecalc), with test and androidTest shown in parentheses after the package name. In the SimpleCalc app, only the main and test source sets are used. Source sets

  3. Expand the com.android.example.simplecalc (test) folder.

    This folder is where you put your app's local unit tests. Android Studio creates a sample test class for you in this folder for new projects, but for SimpleCalc the test class is called CalculatorTest.

  4. Open CalculatorTest.java.

  5. Examine the code and note the following:

    • The only imports are from the org.junit, org.hamcrest, and android.test packages. There are no dependencies on the Android framework classes here.
    • The @RunWith(JUnit4.class) annotation indicates the runner that will be used to run the tests in this class. A test runner is a library or set of tools that enables testing to occur and the results to be printed to a log. For tests with more complicated setup or infrastructure requirements (such as Espresso) you'll use different test runners. For this example we're using the basic JUnit4 test runner.
    • The @SmallTest annotation indicates that all the tests in this class are unit tests that have no dependencies, and run in milliseconds. The @SmallTest, @MediumTest, and @LargeTest annotations are conventions that make it easier to bundle groups of tests into suites of similar functionality.
    • The setUp() method is used to set up the environment before testing, and includes the @Before annotation. In this case the setup creates a new instance of the Calculator class and assigns it to the mCalculator member variable.
    • The addTwoNumbers() method is an actual test, and is annotated with @Test. Only methods in a test class that have an @Test annotation are considered tests to the test runner. Note that by convention test methods do not include the word "test."
    • The first line of addTwoNumbers() calls the add() method from the Calculator class. You can only test methods that are public or package-protected. In this case the Calculator is a public class with public methods, so all is well.
    • The second line is the assertion for the test. Assertions are expressions that must evaluate and result in true for the test to pass. In this case the assertion is that the result you got from the add method (1 + 1) matches the given number 2. You'll learn more about how to create assertions later in this practical.

1.2 Run tests in Android Studio

In this task you'll run the unit tests in the test folder and view the output for both successful and failed tests.

  1. In the project view, right-click the CalculatorTest class and select Run 'CalculatorTest'.

    The project builds, if necessary, and the testing view appears at the bottom of the screen. At the top of the screen, the dropdown (for available execution configurations) also changes to CalculatorTest. Run configuration for CalculatorTest

    All the tests in the CalculatorTest class run, and if those tests are successful, the progress bar at the top of the view turns green. (In this case, there is currently only the one test.) A status message in the footer also reports "Tests Passed."
    All tests successful

  2. In the CalculatorTest class, change the assertion in addTwoNumbers() to:
    assertThat(resultAdd, is(equalTo(3d)));

  3. In the run configurations dropdown at the top of the screen, select CalculatorTest (if it is not already selected) and click Run Run Icon.

    The test runs again as before, but this time the assertion fails (3 is not equal to 1 + 1.) The progress bar in the run view turns red, and the testing log indicates where the test (assertion) failed and why.

  4. Change the assertion in addTwoNumbers() back to the correct test and run your tests again to ensure they pass.
  5. In the run configurations dropdown, select app to run your app normally.

Task 2. Add more unit tests to CalculatorTest

With unit testing, you take a small bit of code in your app such as a method or a class, and isolate it from the rest of your app, so that the tests you write makes sure that one small bit of the code works in the way you'd expect. Typically unit tests call a method with a variety of different inputs, and verifies that the particular method does what you expect and returns what you expect it to return.

In this task you'll learn more about how to construct unit tests. You'll write additional unit tests for the methods in the Calculator utility methods in the SimpleCalc app, and run those tests to make sure they produce the output you expect.

Note: Unit testing, test-driven development and the JUnit 4 API are all large and complex topics and outside the scope of this course. See the Resources for links to more information.

2.1 Add more tests for the add() method

Although it is impossible to test every possible value that the add() method may ever see, it's a good idea to test for input that might be unusual. For example, consider what happens if the add() method gets arguments:

  • With negative operands.
  • With floating-point numbers.
  • With exceptionally large numbers.
  • With operands of different types (a float and a double, for example)
  • With an operand that is zero.
  • With an operand that is infinity.

In this task we'll add more unit tests for the add() method to test different kinds of inputs.

  1. Add a new method to CalculatorTest called addTwoNumbersNegative(). Use this skeleton:

    @Test
public void addTwoNumbersNegative() {
}

    This test method has a similar structure to addTwoNumbers: it is a public method, with no parameters, that returns void. It is annotated with the @Test annotation, which indicates it is a single unit test.

    Why not just add more assertions to addTwoNumbers? Grouping more than one assertion into a single method can make your tests harder to debug if only one assertion fails, and obscures the tests that do succeed. The general rule for unit tests is to provide a test method for every individual assertion.

  2. Run all tests in CalculatorTests, as before.

    In the test window both addTwoNumbers and addTwoNumbersNegative are listed as available (and passing) tests in the left panel. The addTwoNumbersNegative test still passes even though it doesn't contain any code -- a test that does nothing is still considered a successful test.

  3. Add a line to invoke the add() method in the Calculator class with a negative operand.

    double resultAdd = mCalculator.add(-1d, 2d);

    The "d" notation after each operand indicates that these are numbers of type double. Since the add() method is defined with double parameters, floats or ints will also work. Indicating the type explicitly enables you to test other types separately, if you need to.

  4. Add an assertion with assertThat().

    assertThat(resultAdd, is(equalTo(1d)));

    The assertThat() method is a JUnit4 assertion that claims the expression in the first argument is equal to the one in the second argument. Older versions of JUnit used more specific assertion methods (assertEquals(), assertNull(), assertTrue()), but assertThat() is a more flexible, more debuggable and often easier to read format.

    The assertThat() method is used with matchers. Matchers are the chained method calls in the second operand of this assertion (is(equalto()). The available matchers you can use to build an assertion are defined by the hamcrest framework (Hamcrest is an anagram for matchers.) Hamcrest provides many basic matchers for most basic assertions. You can also define your own custom matchers for more complex assertions.

    In this case the assertion is that the result of the add() operation (-1 + 2) is equal to 1.

  5. Add a new unit test to CalculatorTest for floating-point numbers:

    @Test
public void addTwoNumbersFloats() {
   double resultAdd = mCalculator.add(1.111f, 1.111d);
   assertThat(resultAdd, is(equalTo(2.222d)));
}

    Again, a very similar test to the previous test method, but with one argument to add() that is explicitly type float rather than double. The add() method is defined with parameters of type double, so you can call it with a float type, and that number is promoted to a double.

  6. Run all tests in CalculatorTests, as before.

  7. Click Run Run Icon to run all the tests again.

    This time the test failed, and the progress bar is red. This is the important part of the error message:

    java.lang.AssertionError:
Expected: is <2.222>
     but: was <2.2219999418258665>

    Arithmetic with floating-point numbers is inexact, and the promotion resulted in a side effect of additional precision. The assertion in the test is technically false: the expected value is not equal to the actual value.

    The question here is: when you have a precision problem with promoting float arguments is that a problem with your code, or a problem with your test? In this particular case both input arguments to the add() method from the Calculator app will always be type double, so this is an arbitrary and unrealistic test. However, if your app was written such that the input to the add() method could be either double or float and you only care about some precision, you need to provide some wiggle room to the test so that "close enough" counts as a success.

  8. Change the assertThat() method to use the closeTo() matcher:

    assertThat(resultAdd, is(closeTo(2.222, 0.01)));

    For this test, rather that testing for exact equality you can test for equality within a specific delta. In this case the closeTo() matcher method takes two arguments: the expected value and the amount of delta. Here that delta is just two decimal points of precision.

2.2 Add unit tests for the other calculation methods

Use what you learned in the previous task to fill out the unit tests for the Calculator class.

  1. Add a unit test called subTwoNumbers() that tests the sub() method.
  2. Add a unit test called subWorksWithNegativeResults() that tests the sub() method where the given calculation results in a negative number.
  3. Add a unit test called mulTwoNumbers() that tests the mul() method.
  4. Add a unit test called mulTwoNumbersZero() that tests the mul method with at least one argument as zero.
  5. Add a unit test called divTwoNumbers() that tests the div() method with two non-zero arguments.

Challenge: Add a unit test called divByZero() that tests the div() method with a second argument of 0. Hint: Try this in the app first to see what the result is.

Solution Code: 

@Test
public void addTwoNumbers() {
   double resultAdd = mCalculator.add(1d, 1d);
   assertThat(resultAdd, is(equalTo(2d)));
}

@Test
public void addTwoNumbersNegative() {
   double resultAdd = mCalculator.add(-1d, 2d);
   assertThat(resultAdd, is(equalTo(1d)));
}
@Test
public void addTwoNumbersFloats() {
   double resultAdd = mCalculator.add(1.111f, 1.111d);
assertThat(resultAdd, is(closeTo(2.222, 0.01)));
}
@Test
public void subTwoNumbers() {
   double resultSub = mCalculator.sub(1d, 1d);
   assertThat(resultSub, is(equalTo(0d)));
}
@Test
public void subWorksWithNegativeResult() {
   double resultSub = mCalculator.sub(1d, 17d);
   assertThat(resultSub, is(equalTo(-16d)));
}
@Test
public void mulTwoNumbers() {
   double resultMul = mCalculator.mul(32d, 2d);
   assertThat(resultMul, is(equalTo(64d)));
}
@Test
public void divTwoNumbers() {
   double resultDiv = mCalculator.div(32d,2d);
   assertThat(resultDiv, is(equalTo(16d)));
}
@Test
public void divTwoNumbersZero() {
   double resultDiv = mCalculator.div(32d,0);
   assertThat(resultDiv, is(equalTo(Double.POSITIVE_INFINITY)));
}

Solution code

Android Studio project: SimpleCalcTest

Coding challenges

Note: All coding challenges are optional and are not prerequisites for later lessons.

Challenge 1: Dividing by zero is always worth testing for, because it a special case in arithmetic. If you try to divide by zero in the current version of the SimpleCalc app, it behaves the way Java defined: Dividing a number by returns the "Infinity" constant (Double.POSITIVE_INFINITY). Dividing 0 by 0 returns the not a number constant (Double.NaN). Although these values are correct for Java, they're not necessarily useful values for the user in the app itself. How might you change the app to more gracefully handle divide by zero? To accomplish this challenge, start with the test first -- consider what the right behavior is, and then write the tests as if that behavior already existed. Then change or add to the code so that it makes the tests come up green.

Challenge 2: Sometimes it's difficult to isolate a unit of code from all of its external dependencies. Rather than artificially organize your code in complicated ways just so it can be more easily tested, you can use a mock framework to create fake ("mock") objects that pretend to be dependencies. Research the Mockito framework, and learn how to set it up in Android Studio. Write a test class for the calcButton() method in SimpleCalc, and use Mockito to to simulate the Android context in which your tests will run.

Summary

  • Android Studio has built-in features for running local unit tests.
    • Local unit tests use the JVM of your local machine and do not use the Android framework.
    • Unit tests are written with JUnit, a common unit testing framework for Java.
    • The "test" folder (Android Studio's Project View) is where JUnit tests are located.
    • Local unit tests only need the packages: org.junit, org.hamcrest and android.test
    • The @RunWith(JUnit4.class) annotation tells the test runner to run tests in this class.
    • @SmallTest, @MediumTest, and @LargeTest annotations are conventions that make it easier to bundle similar groups of tests
    • The @SmallTest annotation indicates all the tests in a class are unit tests that have no dependencies and run in milliseconds.
  • Instrumented tests are tests that run on an Android device or emulator.
    • Instrumented tests have access to the Android framework.
  • A test runner is a library or set of tools that enables testing to occur and the results to be printed to the log.

The related concept documentation is in Android Developer Fundamentals: Concepts.

Learn More

results matching ""

    No results matching ""