We’re continuing with our series of blogs about everything related to testing. In this blog, we’re focusing on real examples.
While the examples in this post are written using JUnit 5 and AssertJ, the lessons are applicable to any other unit testing framework.
JUnit is the most popular testing framework for Java. AssertJ is a Java library that helps developers write more expressive tests.
Basic test structure
The first example of a test we will look at is a simple calculator for adding 2 numbers.
class CalculatorShould { @Test // 1 void sum() { Calculator calculator = new Calculator(); // 2 int result = calculator.sum(1, 2); // 3 assertThat(result).isEqualTo(3); // 4 } }
I prefer using the ClassShould
naming convention when writing tests to avoid repeating should
or test
in every method name. You can read more about it here.
What does the test above do?
Let’s break the test line by line:
@Test
annotation lets JUnit framework know which methods are meant to be run as tests. It is perfectly normal to haveprivate
methods in the test class which are not tests.- This is the arrange phase of our test, where we prepare the testing environment. All we need for this test is to have a
Calculator
instance. - This is the act phase where we trigger the behaviour we want to test.
- This is the assert phase in which we inspect what happened and if everything resolved as expected.
assertThat(result)
method is part of the AssertJ library and has multiple overloads.
Each overload returns a specialized Assert
object. The returned object has methods that make sense for the object we passed to the assertThat
method. In our case, that object is AbstractIntegerAssert
with methods for testing Integers. isEqualTo(3)
will check if result == 3
. If it is, the test will pass and fail otherwise.
We won’t focus on any implementations in this blog post.
Another way of thinking about Arrange, Act, Assert is Given, When, Then.
After we write our sum
implementation, we can ask ourselves some questions:
- How can I improve on this test?
- Are there more test cases I should cover?
- What happens if I add a positive and a negative number? Two negative numbers? One positive and one negative?
- What if I overflow the integer value?
Let’s add these cases and improve on the existing test name a little bit.
We will not allow overflows in our implementation. If sum
overflows, we will throw an ArithmeticException
instead.
class CalculatorShould { private Calculator calculator = new Calculator(); @Test void sumPositiveNumbers() { int sum = calculator.sum(1, 2); assertThat(sum).isEqualTo(3); } @Test void sumNegativeNumbers() { int sum = calculator.sum(-1, -1); assertThat(sum).isEqualTo(-2); } @Test void sumPositiveAndNegativeNumbers() { int sum = calculator.sum(1, -2); assertThat(sum).isEqualTo(-1); } @Test void failWithArithmeticExceptionWhenOverflown() { assertThatThrownBy(() -> calculator.sum(Integer.MAX_VALUE, 1)) .isInstanceOf(ArithmeticException.class); } }
JUnit will create a new instance of CalculatorShould
before running each @Test
method. That means that each CalculatorShould
will have a different calculator
so we don’t have to instance it in every test.
shouldFailWithArithmeticExceptionWhenOverflown
test uses a different kind of an assert
. It checks that a piece of code failed. assertThatThrownBy
method will run the lambda we provided and make sure that it failed. As we already know, all assertThat
methods return a specialized Assert
allowing us to check which type of exception occurred.
This is an example of how we can test that our code fails when we expect it to. If at any point we refactor Calculator
and it does not throw ArithmeticException
on an overflow, our test will fail.
ObjectMother design pattern
The next example is a validator class for ensuring a Person instance is valid.
class PersonValidatorShould { private PersonValidator validator = new PersonValidator(); @Test void failWhenNameIsNull() { Person person = new Person(null, 20, new Address(...), ...); assertThatThrownBy(() -> validator.validate(person)) .isInstanceOf(InvalidPersonException.class); } @Test void failWhenAgeIsNegative() { Person person = new Person("John", -5, new Address(...), ...); assertThatThrownBy(() -> validator.validate(person)) .isInstanceOf(InvalidPersonException.class); } }
ObjectMother design pattern is often used in tests that create complex objects to hide the instantiation details from the test. Multiple tests might even create the same object but test different things on it.
Test #1 is very similar to test #2. We can refactor PersonValidatorShould
by extracting the validation as a private method then pass illegal Person
instances to it expecting them all to fail in the same fashion.
class PersonValidatorShould { private PersonValidator validator = new PersonValidator(); @Test void failWhenNameIsNull() { shouldFailValidation(PersonObjectMother.createPersonWithoutName()); } @Test void failWhenAgeIsNegative() { shouldFailValidation(PersonObjectMother.createPersonWithNegativeAge()); } private void shouldFailValidation(Person invalidPerson) { assertThatThrownBy(() -> validator.validate(invalidPerson)) .isInstanceOf(InvalidPersonException.class); } }
Testing randomness
How are we supposed to test randomness in our code?
Let’s suppose we have a PersonGenerator
that has generateRandom
to generate random Person
instances.
We start by writing the following:
class PersonGeneratorShould { private PersonGenerator generator = new PersonGenerator(); @Test void generateValidPerson() { Person person = generator.generateRandom(); assertThat(person). } }
And then we should ask ourselves:
- What am I trying to prove here? What does this functionality need to do?
- Should I just verify that the generated person is a non-null instance?
- Do I need to prove it is random?
- Does the generated instance have to follow some business rules?
We can simplify our test using Dependency Injection.
public interface RandomGenerator { String generateRandomString(); int generateRandomInteger(); }
The PersonGenerator
now has another constructor that also accepts an instance of that interface as well. By default, it uses the JavaRandomGenerator
implementation that generates random values using java.Random
.
However, in the test, we can write another, more predictable implementation.
@Test void generateValidPerson() { RandomGenerator randomGenerator = new PredictableGenerator("John Doe", 20); PersonGenerator generator = new PersonGenerator(randomGenerator); Person person = generator.generateRandom(); assertThat(person).isEqualTo(new Person("John Doe", 20)); }
This test proves that the PersonGenerator
generates random instances as specified by the RandomGenerator
without getting into any details of the RandomGenerator
.
Testing the JavaRandomGenerator
does not really add any value since it is a simple wrapper around java.Random
. By testing it, you would essentially be testing java.Random
from the Java standard library. Writing obvious tests will only lead to additional maintenance with little if any benefits.
To avoid writing implementations for testing purposes, such as PredictableGenerator
, you should use a mocking library such as Mockito.
When we wrote PredictableGenerator
, we actually stubbed the RandomGenerator
class manually. You could have also stubbed it using Mockito:
@Test void generateValidPerson() { RandomGenerator randomGenerator = mock(RandomGenerator.class); when(randomGenerator.generateRandomString()).thenReturn("John Doe"); when(randomGenerator.generateRandomInteger()).thenReturn(20); PersonGenerator generator = new PersonGenerator(randomGenerator); Person person = generator.generateRandom(); assertThat(person).isEqualTo(new Person("John Doe", 20)); }
This way of writing tests is more expressive and leads to fewer implementations for specific tests.
Mockito is a Java library for writing mocks and stubs. It is very useful when testing code that depends on external libraries you cannot easily instantiate. It allows you to write behaviour for these classes without implementing them directly.
Mockito also allows another syntax for creating and injecting mocks to reduce boilerplate when we have more than one test similar to what we are used to:
@ExtendWith(MockitoExtension.class) // 1 class PersonGeneratorShould { @Mock // 2 RandomGenerator randomGenerator; @InjectMocks // 3 private PersonGenerator generator; @Test void generateValidPerson() { when(randomGenerator.generateRandomString()).thenReturn("John Doe"); when(randomGenerator.generateRandomInteger()).thenReturn(20); Person person = generator.generateRandom(); assertThat(person).isEqualTo(new Person("John Doe", 20)); } }
1. JUnit 5 can use “extensions” to extend its capabilities. This annotation allows it to recognize mocks through annotations and inject them properly.
2. @Mock
annotation creates a mocked instance of the field. This is the same as writing mock(RandomGenerator.class)
in our test method body.
3. @InjectMocks
annotation will create a new instance of PersonGenerator
and inject mocks in the generator
instance.
For more details on JUnit 5 extensions see here.
For more details on Mockito injection see here.
There is one pitfall to using @InjectMocks
. It may remove the need to declare an instance of the object manually, but we lose the compile-time safety of the constructor. If at any point in time someone adds another dependency to the constructor, we would not get the compile-time error here. This could lead to failing tests that are not easy to detect. I prefer to use @BeforeEach
to setup the instance manually:
@ExtendWith(MockitoExtension.class) class PersonGeneratorShould { @Mock RandomGenerator randomGenerator; private PersonGenerator generator; @BeforeEach void setUp() { generator = new PersonGenerator(randomGenerator); } @Test void generateValidPerson() { when(randomGenerator.generateRandomString()).thenReturn("John Doe"); when(randomGenerator.generateRandomInteger()).thenReturn(20); Person person = generator.generateRandom(); assertThat(person).isEqualTo(new Person("John Doe", 20)); } }
Testing time-sensitive processes
A piece of code is often dependent on timestamps and we tend to use methods such as System.currentTimeMillis()
to get the current epoch timestamp.
While this looks fine, it is hard to test and prove if our code works correctly when the class makes decisions for us internally. An example of such a decision would be determining what the current day is.
class IndexerShould { private Indexer indexer = new Indexer(); @Test void generateIndexNameForTomorrow() { String indexName = indexer.tomorrow("my-index"); // this test would work today, but what about tomorrow? assertThat(indexName) .isEqualTo("my-index.2022-02-02"); } }
We should use Dependency Injection again to be able to ‘control’ what the day is when generating the index name.
Java has a Clock
class to handle use-cases such as this. We can pass an instance of a Clock
to our Indexer
to control the time. The default constructor could use Clock.systemUTC()
for backwards compatibility. We can now replace System.currentTimeMillis()
calls with clock.millis()
.
By injecting a Clock
we can enforce a predictable time in our classes and write better tests.
Testing file-producing methods
- How should we test classes which write their output to files?
- Where should we store these files for them to work on any OS?
- How can we make sure that the file does not already exist?
When dealing with files, it can be difficult to write tests if we try to tackle these concerns ourselves as we will see in the following example. The test that follows is an old test of dubious quality. It should test if a DogToCsvWriter
serializes and writes dogs to a CSV file:
class DogToCsvWriterShould { private DogToCsvWriter writer = new DogToCsvWriter("/tmp/dogs.csv"); @Test void convertToCsv() { writer.appendAsCsv(new Dog(Breed.CORGI, Color.BROWN, "Monty")); writer.appendAsCsv(new Dog(Breed.MALTESE, Color.WHITE, "Zoe")); String csv = Files.readString("/tmp/dogs.csv"); assertThat(csv).isEqualTo("Monty,corgi,brownnZoe,maltese,white"); } }
The serialization process should be decoupled from the writing process, but let’s focus on fixing the test.
The first problem with the test above is that it won’t work on Windows since Windows users won’t be able to resolve the path /tmp/dogs.csv
. Another issue is that it won’t work if the file already exists since it is not deleted when the test above executes. It might work ok in a CI/CD pipeline, but not locally if run multiple times.
JUnit 5 has an annotation you can use to get a reference to a temporary directory that gets created and deleted by the framework for you. While the mechanism of creating and deleting temporary files varies from framework to framework, the ideas remain the same.
class DogToCsvWriterShould { @Test void convertToCsv(@TempDir Path tempDir) { Path dogsCsv = tempDir.resolve("dogs.csv"); DogToCsvWriter writer = new DogToCsvWriter(dogsCsv); writer.appendAsCsv(new Dog(Breed.CORGI, Color.BROWN, "Monty")); writer.appendAsCsv(new Dog(Breed.MALTESE, Color.WHITE, "Zoe")); String csv = Files.readString(dogsCsv); assertThat(csv).isEqualTo("Monty,corgi,brownnZoe,maltese,white"); } }
With this small change, we are now sure that the test above will work on Windows, macOS and Linux without having to worry about absolute paths. It will also delete the created files after the test so we can now run it multiple times and get predictable results each time.
Command vs query testing
What is the difference between a command and a query?
- Command: we instruct an object to perform an action that produces an effect without returning a value (void methods)
- Query: we ask an object to perform an action and return a result or an exception
So far, we have tested mainly queries where we called a method that returned a value or has thrown an exception in the act phase. How can we test void
methods and see if they interact correctly with other classes? Frameworks provide a different set of methods for writing these kinds of tests.
The assertions we wrote thus far for queries were beginning with assertThat
. When writing command tests, we use a different set of methods because we are no longer inspecting the direct results of methods as we did with queries. We want to ‘verify’ interactions our method had with other parts of our system.
@ExtendWith(MockitoExtension.class) class FeedMentionServiceShould { @Mock private FeedRepository repository; @Mock private FeedMentionEventEmitter emitter; private FeedMentionService service; @BeforeEach void setUp() { service = new FeedMentionService(repository, emitter); } @Test void insertMentionToFeed() { long feedId = 1L; Mention mention = ...; when(repository.upsertMention(feedId, mention)) .thenReturn(UpsertResult.success(feedId, mention)); FeedInsertionEvent event = new FeedInsertionEvent(feedId, mention); mentionService.insertMentionToFeed(event); verify(emitter).mentionInsertedToFeed(feedId, mention); verifyNoMoreInteractions(emitter); } }
In this test, we first mocked our repository to respond with a UpsertResult.success
when asked to upsert mention in our feed. We are not concerned with testing the repository here. The repository methods should be tested in the FeedRepositoryShould
. By mocking this behaviour, we didn’t actually call the repository method. We simply told it how to respond next time it is called.
We then told our mentionService
to insert this mention in our feed. We know that it should emit the result only if it successfully inserted the mention in the feed. By using the verify
method we can make sure that the method mentionInsertedToFeed
was called with our mention and feed and wasn’t called again using verifyNoMoreInteractions
.
Final thoughts
Writing quality tests comes from experience, and the best way to learn is by doing. The tips written in this blog come from practice. It is hard to see some of the pitfalls if you never encountered them and hopefully, these suggestions should make your code design more robust. Having reliable tests will increase your confidence to change things without breaking a sweat each time you have to deploy your code.