For Loops in Test Cases or Multiple Assertions?

The content here is under the Attribution 4.0 International (CC BY 4.0) license

Testing has been a hot topic among developers for a long time now, with the Agile Manifesto already over 20 years old. Before that, we had eXtreme Programming (XP), which paved the way for modern testing practices. XP popularized the idea of having testable code as a discipline and shifted the need to write tests earlier in the process—sometimes even before writing the actual code.

To be clear, developers were testing their applications long before XP. But writing tests as an integrated part of the development process became widespread thanks to these methodologies. Along with this shift came new questions about best practices, especially for those following TDD (Test-Driven Development). TDD emphasizes the cycle: write a failing test, make it pass with the simplest possible solution, and then refactor.

The refactoring step is essential, and while many developers have practiced TDD for years, some common questions remain due to the lack of formal education in this area:

1. Should I use logic inside my tests?
2. Should I use multiple assertions in a single test?

This post will focus on both assertions and logic inside test cases, like using for loops.

To For Loop or Not?

At first glance, having logic inside a test case—whether it’s a for loop or an if statement—might seem like a red flag. As the xUnit Testing Patterns book notes [1], tests should ideally have no logic in them.

For instance, consider the classic if/else scenario:

• If variable X is true, assert that X is true.
• Otherwise, assert that Y is a string.

In this case, the test has two different reasons it could fail. This not only complicates the debugging process, but also makes it harder to understand the test’s intent. Dynamic code in test cases begs the question: “Who tests the test?” So, is it okay to use for loops in test cases? Let’s explore some data.

Survey Says

A Twitter poll I ran didn’t get much attention, but the results were interesting. The majority said that using for loops in tests is unnecessary, though the margin was small—just 2.5% separated the “No” votes from the “Yes” votes. So, while it’s not a landslide, most developers tend to avoid them.

Additionally, if you dive into some popular GitHub repositories used in the TDD anti patterns series, you’ll notice that the use of for loops in tests is pretty rare. The prevailing pattern follows the AAA structure (Arrange, Act, Assert), and this generally discourages loops in test cases.

What About Multiple Assertions?

When it comes to multiple assertions in a single test, the opinions are more divided. According to the Twitter poll, the majority believe there’s no hard rule—developers feel free to use “as many as I want”. The reasoning behind this is that if you’re testing multiple aspects of the same behavior, there’s no harm in including several assertions in one test.

Elliotte Rusty Harold, however, suggests sticking to one assertion per test method as a best practice. This makes it easier to pinpoint failures and reduces complexity.

In the end, assertions are context-dependent. Listening to your tests and the feedback they provide is crucial. Steve Freeman and Nat Pryce, authors of Growing Object-Oriented Software Guided by Tests, emphasize that tests should help you move forward efficiently. They should be easy to maintain, run quickly, and provide clear feedback when something breaks.

In 20218, Martin Fowler released the second edition of refactoring and in there he favors the use of a single assertion per test case [2].

Final Thoughts

While there’s no one-size-fits-all answer, the key takeaway is to “listen to your tests.” Avoid introducing unnecessary complexity, whether it’s through logic in test cases or overloading tests with multiple assertions. Tests should serve as a safety net, not a source of confusion. By following these principles, you’ll have tests that not only guide your code but also make your development process smoother and more reliable.

References

1. [1]G. Meszaros, XUnit Test Patterns: Refactoring Test Code. USA: Prentice Hall PTR, 2006.
2. [2]M. Fowler, Refactoring: improving the design of existing code. Addison-Wesley Professional, 2018.