TDD anti-patterns - episode 5 - The stranger, The operating system evangelist, Success against all odds and The free ride - with code examples in javascript and python

The content here is under the Attribution 4.0 International (CC BY 4.0) license

This is a follow up on a series of posts around TDD anti-patterns. The first of this series covered the liar, excessive setup, the giant and slow poke, those four are part of 22 more anti-patterns formalized in the James Carr post and discussed in a Stack Overflow thread.

Do you prefer this content in video format? The video is available on demand in livestorm.

In this blog post we are going to focus on four more of them, named: The stranger, The operating system evangelist, Success against all odds and The free ride. Each focuses on a specific aspect of code that makes testing harder. Sometimes this results from not practicing TDD, and sometimes from insufficient experience in software design. Either way, this is an attempt to explore these flaws and avoid them in our own codebase.

Takeaways

1. Tying up test code with the type of operating system brings portability issues
2. Avoid sliding assertions that requires a test case for itself
3. Focus on having each test case with a single responsibility and not that many assertions

Cross-Episode Connections: The Stranger relates to The Inspector (Episode 2)—both violate encapsulation. The Operating System Evangelist mirrors The Local Hero (Episode 2)—both create environment-specific brittle tests. Success Against All Odds extends The Giant (Episode 1)—excessive assertions hide the true intent. The Free Ride combines The Giant and The Generous Leftovers.

The stranger

In this blog post from java revisited, the law of demeter explanation gives us a hint on why The stranger is an anti-pattern, which we can also relate to the clean code book “talk to friends, not to strangers”. In their example, the method chain is the one that exposes the stranger more. The example is being used in production code. Carlos Caballero in his blog post “Demeter’s Law: Don’t talk to strangers!” also uses production code to depict what is and when the demeter law is not followed. He gives a snippet that ideally would need to be tested, and here we are going to expand that and implement the test code.

To start with, here goes the code that depicts the violation in the production code:

person
    .getHouse() // return an House's object
    .getAddress() // return an Address's object
    .getZipCode() // return a ZipCode Object

Such code could potentially lead to The stranger in the test code, for example, to test if the person given has a valid zip code, we could potentially write something like the following:

describe('Person', () => {
    it('should have valid zip code', () => {
        const person = FakeObject.createAPerson({ zipCode: '56565656' });
        person
            .getHouse()
            .getAddress()
            .getZipCode()
        expect('56565656').toEqual(person.house.address.zipCode);
    });
});

Note that if we want to access the zip code, we need to go all the way down to the ZipCode object, potentially, this could show that what we want to test is the Address itself and not the person.

describe('Address', () => {
    it('should have valid zip code', () => {
        const address = new Address(
            '56565656',
            '1456',
            'Street X',
            'My city',
            'Great state',
            'The best country'
        );
        expect('56565656').toEqual(address.getZipCode());
    });
});

The test itself has something here that could be improved, to avoid this almost one to one test class and production code, for example, the interaction between the person object, address and zip code could be “hidden” in a implementation and test the output of it, instead of going all the way down in the chain.

Before moving on to the next one, remember that The stranger could also be one of the anti-patterns that is related to test smells. There are some indications that you might be facing the stranger:

1. It depends on context
2. It is related to the xUnit pattern in the section “Test smells” (Meszaros, 2007)
3. It can be related to mocks

The stranger - root causes

1. Violating Law of Demeter—reaching through objects to test distant behavior
2. Testing object chains instead of breaking dependencies and testing each object independently
3. Insufficient use of object composition and dependency injection

The operating system evangelist

The operating system evangelist is related to how coupled the testing code is to the operating system, the way of coupling can be on different aspects of the code, for example, using a specific path that exists only on windows.

To depict such a case, the snippet that follows was extracted from the open source project Lutris. Lutris aims to run games that are for windows on linux. The premise of the project already gives some constraints that are expected in the code base. The result, is the following test case, that launches a linux process:

class LutrisWrapperTestCase(unittest.TestCase):
    def test_excluded_initial_process(self):
        "Test that an excluded process that starts a monitored process works"
        env = os.environ.copy()
        env['PYTHONPATH'] = ':'.join(sys.path)
        # run the lutris-wrapper with a bash subshell. bash is "excluded"
        wrapper_proc = subprocess.Popen(
            [
                sys.executable, lutris_wrapper_bin, 'title', '0', '1', 'bash', 'bash',
                '-c',
                "echo Hello World; exec 1>&-; while sleep infinity; do true; done"
            ],
            stdin=subprocess.DEVNULL, stdout=subprocess.PIPE, env=env,
        )

This test relies heavily on bash and would fail if run on Windows. This is not necessarily problematic—it represents a trade-off between the project’s focus and the cost of adding cross-platform abstraction.

In the book “Cosmic Python” (Chapter 3), the author discusses coupling and abstraction using file paths as an example. This concept also relates to the strategy design pattern.

The operating system evangelist also appeared in go lang in an issue that was trying to mitigate the new line on linux and windows. Actually, this issue is part of the definition of this anti-pattern: “A good example would be a test case that uses the newline sequence for windows in an assertion, only to break when run on Linux.” In that thread, a user complains about the issues that she has to run the same tests on windows. Most errors result from differences in line feed characters across operating systems.

Another anti-pattern that is related to the Operating System Evangelist is The Local Hero. The Local Hero is known for having everything in place locally to run an application, but as soon as you try to run it in another machine it will fail.

FLASH BACK - In the episode 2, besides the local hero we discussed the anti patterns the mockery, the inspector and the generous leftovers, the examples used are in javascript, kotlin and php

We already had a discussion about the local hero in the episode 2 of this series, but to reinforce on how they are connected, here goes an example from Jenkins source code:

@Test
public void testWithoutSOptionAndWithoutJENKINS_URL() throws Exception {
    Assume.assumeThat(System.getenv("JENKINS_URL"), is(nullValue()));
    // TODO instead remove it from the process env?
    assertNotEquals(0, launch("java",
            "-Duser.home=" + home,
            "-jar", jar.getAbsolutePath(),
            "who-am-i")
    );
}

This snippet is particularly interesting because whoever wrote it already noticed that there was some smell going on, with the comment: TODO instead remove it from the process env?

Last but not least, katas are usually the ones to catch those kinds of patterns early on, and push for an abstraction during the refactor phase. The WordWrap is an example of kata that aims to break into new lines if the content is greater than the expected. For an explanation on the differences in feed lines and operating systems, check Baeldung.com post.

The operating system evangelist - root causes

1. Hard-coded OS-specific paths, line separators, or environment variables in tests
2. Insufficient abstraction—tests tightly coupled to operating system differences
3. Lack of containerization or CI environment standardization

Success against all odds

Throughout this series, we have seen different anti-patterns that arise from a lack of practicing the test-first approach. Such behavior leads to different things that make testing difficult, for example, the excessive setup and the giant, which are related to the god object.

Success against all odds occurs when developers skip the fail-first step, immediately writing a test that passes without first verifying it fails. This approach defeats the purpose of test-driven development and can mask design issues.

To illustrate this, here is a repository implementation from Spring Boot that paginates and queries results based on a given string.

@Repository
class ProductsRepositoryWithPostgres(
    private val entityManager: EntityManager
) : Repository {

    override fun listFilteredProducts(query: String?, input: PagingQueryInput?) {
        val pageRequest: PageRequest = input.asPageRequest()
        val page: Page<Product> = if (query.isNullOrBlank()) {
            entityManager.findAll(pageRequest)
        } else {
            entityManager.findAllByName(query, pageRequest)
        }
        return page
    }

}

Here is the test code for the repository:

Note that the setup performs extensive data preparation, which could itself be a code smell.

Note: For the sake of the example, the teardown has been removed in order to keep it simple. The tear down removes all the data inserted in the database used during the test.

private fun setupBeforeAll() {
    productIds = (1..100).map { db().productWithDependencies().apply().get<ProductId>() }
    productIdsContainingWood.addAll(
        (1..3).map { insertProductWithName("WoodyWoodOrange " + faker.funnyName().name()) }
    )
    productIdsContainingWood.addAll(
        (1..3).map {
            insertProductWithName(
                faker.funnyName().name() + " WoodyWoodOrange " + faker.funnyName().name()
            )
        }
    )

With the setup in place, let’s examine the first test. This test verifies sorting by the CREATED_AT_ASC parameter.

@Test
fun `list products sorted by creation at date ascending`() {
    val pageQueryInput = PagingQueryInput(
        size = 30, page = 0, sort = listOf(Sort.CREATED_AT_ASC) // 1
    )
    val result = repository.listFilteredProducts("", pageQueryInput) // 2

    assertThat(result.currentPage).isEqualTo(0) // 3
    assertThat(result.totalPages).isEqualTo(4) // 4
    assertThat(result.totalElements).isEqualTo(112) // 5

    assertThat(result.content.size).isEqualTo(30) // 6
    assertThat(result.content).allSatisfy { productIds.subList(0, 29).contains(it.id) } // 7
}

Let’s examine what this code does:

1. The pagination request with sort order
2. Execution of the repository method
3. Verify the page is the first one
4. Verify there are 4 total pages
5. Verify there are 112 total elements
6. Verify the returned list size matches pagination
7. Verify the returned list matches the expected data

The next test case depicts a variant on what we might want to test, which is the reverse order. Instead of ascending, we now will test descending. Note that the majority of the asserts are the same as the previous test case.

@Test
fun `list products sorted by creation at date ascending`() {
    val pageQueryInput = PagingQueryInput(
        size = 30, page = 0, sort = listOf(Sort.CREATED_AT_ASC) // 1
    )
    val result = repository.listFilteredProducts("", pageQueryInput) // 2

    assertThat(result.currentPage).isEqualTo(0) // 3
    assertThat(result.totalPages).isEqualTo(4) // 4
    assertThat(result.totalElements).isEqualTo(112) // 5

    assertThat(result.content.size).isEqualTo(30) // 6
    assertThat(result.content).allSatisfy { productIds.subList(0, 29).contains(it.id) } // 7
}

Let’s avoid repeating the previous bullet list and focus on the items that are important.

The primary issue is the excessive number of assertions per test case. Items 3-6 verify pagination and collection size, but the test name indicates the focus should be on sorting, not pagination. A single assertion verifying sort order would suffice.

Item 7 is particularly important because such assertions often contribute to Success against all odds. In this example, the assertion checks a subset of the list that will always be true.

In the xunit patterns book, a way to avoid such false positive behavior is to have the code as simple as possible, with no logic in it. This is called Robust Test (Meszaros, 2007).

Refactoring the success against all odds

The proposed alternative splits responsibilities: focus on sorting first, then test pagination separately.

The first example here would be ordering the list in ascending order, it is worth mentioning that with this approach, we could potentially remove the big setup that was shown previously in the hook setupBeforeAll. For this approach, we instead, set up the data that is required for the test inside it.

@Test
fun `list products sorted by ascending creation date`() {
    db().productWithDependencies("created_at" to "2022-04-03T00:00:00.00Z").apply() // 1 
    db().productWithDependencies("created_at" to "2022-04-02T00:00:00.00Z").apply() // 2
    db().productWithDependencies("created_at" to "2022-04-01T00:00:00.00Z").apply() // 3

    val pageQueryInput = PagingQueryInput(sort = listOf(SortOrder.CREATED_AT_ASC))

    val result = repository.listFilteredProducts("", pageQueryInput)

    assertThat(result.content[0].createdAt).isEqualTo("2022-04-01T00:00:00.00Z")
    assertThat(result.content[1].createdAt).isEqualTo("2022-04-02T00:00:00.00Z")
    assertThat(result.content[2].createdAt).isEqualTo("2022-04-03T00:00:00.00Z")
}

Once that is in place, we then move to the descending order test case, which is the same, but the assertion and setup have changed:

@Test
fun `list products sorted by creation at date descending`() {
    db().productWithDependencies("created_at" to "2022-04-01T00:00:00.00Z").apply()
    db().productWithDependencies("created_at" to "2022-04-02T00:00:00.00Z").apply()
    db().productWithDependencies("created_at" to "2022-04-03T00:00:00.00Z").apply()

    val pageQueryInput = PagingQueryInput(sort = listOf(SortOrder.CREATED_AT_DESC))

    val result = repository.listFilteredProducts("", pageQueryInput)

    assertThat(result.content[0].createdAt).isEqualTo("2022-04-03T00:00:00.00Z")
    assertThat(result.content[1].createdAt).isEqualTo("2022-04-02T00:00:00.00Z")
    assertThat(result.content[2].createdAt).isEqualTo("2022-04-01T00:00:00.00Z")
}

Finally, we focus on testing pagination separately.

@Test
fun `should have one page when the list is ten`() {
    insertTenProducts()
    val page = PagingQueryInput(size = 10)

    val result = repository.listFilteredProducts(
        null,
        null,
        Page
    )

    assertThat(result.totalPages).isEqualTo(1)
}

The approach to decompose the tests into smaller “units” would help the communication between the team members that will be dealing with this code later on as well, and also supports the already-mentioned robust tests.

Success against all odds - root causes

1. Skipping the red phase—writing code without seeing the test fail first
2. Assertions loosely coupled to actual behavior—assertions designed after implementation
3. Excessive assertions that can’t possibly fail—logic in tests without verification

The free ride

The free ride is one of the least popular anti-patterns that was found in the survey, maybe this is because the name is not that welcoming when the matter is to recall the meaning.

The free ride appears in test cases that usually require a new test case to test the desired behavior, but instead, another assertion is put in place and sometimes even logic inside the test case is used for that end.

Let’s have a look at the following example that was extracted from the puppeteer project:

it('Page.Events.RequestFailed', async () => {
  const { page, server, isChrome } = getTestState();

  await page.setRequestInterception(true);
  page.on('request', (request) => {
    if (request.url().endsWith('css')) request.abort();
    else request.continue();
  });
  const failedRequests = [];
  page.on('requestfailed', (request) => failedRequests.push(request));
  await page.goto(server.PREFIX + '/one-style.html');
  expect(failedRequests.length).toBe(1);
  expect(failedRequests[0].url()).toContain('one-style.css');
  expect(failedRequests[0].response()).toBe(null);
  expect(failedRequests[0].resourceType()).toBe('stylesheet');

  if (isChrome)
    expect(failedRequests[0].failure().errorText).toBe('net::ERR_FAILED');
  else
    expect(failedRequests[0].failure().errorText).toBe('NS_ERROR_FAILURE');
  expect(failedRequests[0].frame()).toBeTruthy();
});

As already spoiled, the free ride is in the if/else statement. There are two test cases in this single test, but probably the idea was to reuse the same setup code and slide in an assertion in the same test case.

Another approach would be to split the test case in order to focus on a single scenario at time. Puppeteer itself already mitigated this issue using a function to handle such scenario, using that to split the test cases, we would have the first test case focuses on the chrome browser:

itChromeOnly('Page.Events.RequestFailed', async () => {
  const { page, server } = getTestState();

  await page.setRequestInterception(true);
  page.on('request', (request) => {
    if (request.url().endsWith('css')) request.abort();
    else request.continue();
  });
  const failedRequests = [];
  page.on('requestfailed', (request) => failedRequests.push(request));
  await page.goto(server.PREFIX + '/one-style.html');
  expect(failedRequests.length).toBe(1);
  expect(failedRequests[0].url()).toContain('one-style.css');
  expect(failedRequests[0].response()).toBe(null);
  expect(failedRequests[0].resourceType()).toBe('stylesheet');
  expect(failedRequests[0].failure().errorText).toBe('net::ERR_FAILED');
  expect(failedRequests[0].frame()).toBeTruthy();
});

And then, the second case for firefox.

itFirefoxOnly('Page.Events.RequestFailed', async () => {
  const { page, server } = getTestState();

  await page.setRequestInterception(true);
  page.on('request', (request) => {
    if (request.url().endsWith('css')) request.abort();
    else request.continue();
  });
  const failedRequests = [];
  page.on('requestfailed', (request) => failedRequests.push(request));
  await page.goto(server.PREFIX + '/one-style.html');
  expect(failedRequests.length).toBe(1);
  expect(failedRequests[0].url()).toContain('one-style.css');
  expect(failedRequests[0].response()).toBe(null);
  expect(failedRequests[0].resourceType()).toBe('stylesheet');
  expect(failedRequests[0].failure().errorText).toBe('NS_ERROR_FAILURE');
  expect(failedRequests[0].frame()).toBeTruthy();
});

Logic inside the test case is already an indication that the free ride is playing a role. The puppeteer example can be improved even further. Now that we split the logic into two test cases, there is more duplicated code (that could be an argument in favor of adopting the free ride). If that is the case, the testing framework can help us here.

To avoid code duplication in this scenario, we could use the hook beforeEach and move the required setup there.

Moving a bit from puppeteer, there are other ways in which the free ride can appears, the following code from the jenkins project:

public class ToolLocationTest {
    @Rule
    public JenkinsRule j = new JenkinsRule();

    @Test
    public void toolCompatibility() {
        Maven.MavenInstallation[] maven = j.jenkins.getDescriptorByType(Maven.DescriptorImpl.class).getInstallations();
        assertEquals(1, maven.length);
        assertEquals("bar", maven[0].getHome());
        assertEquals("Maven 1", maven[0].getName());

        Ant.AntInstallation[] ant = j.jenkins.getDescriptorByType(Ant.DescriptorImpl.class).getInstallations();
        assertEquals(1, ant.length);
        assertEquals("foo", ant[0].getHome());
        assertEquals("Ant 1", ant[0].getName());

        JDK[] jdk = j.jenkins.getDescriptorByType(JDK.DescriptorImpl.class).getInstallations();
        assertEquals(Arrays.asList(jdk), j.jenkins.getJDKs());
        assertEquals(2, jdk.length); // JenkinsRule adds a 'default' JDK
        assertEquals("default", jdk[1].getName()); // make sure it's really that we're seeing
        assertEquals("FOOBAR", jdk[0].getHome());
        assertEquals("FOOBAR", jdk[0].getJavaHome());
        assertEquals("1.6", jdk[0].getName());
    }
}

Another approach to avoid the free ride in this case, would be once again to split the test cases:

public class ToolLocationTest {
    @Test
    @LocalData
    public void shouldBeCompatibleWithMaven() {
        Maven.MavenInstallation[] maven = j.jenkins.getDescriptorByType(Maven.DescriptorImpl.class).getInstallations();
        assertEquals(1, maven.length);
        assertEquals("bar", maven[0].getHome());
        assertEquals("Maven 1", maven[0].getName());
    }

    @Test
    @LocalData
    public void shouldBeCompatibleWithAnt() {
        Ant.AntInstallation[] ant = j.jenkins.getDescriptorByType(Ant.DescriptorImpl.class).getInstallations();
        assertEquals(1, ant.length);
        assertEquals("foo", ant[0].getHome());
        assertEquals("Ant 1", ant[0].getName());
    }

    @Test
    @LocalData
    public void shouldBeCompatibleWithJdk() {
        JDK[] jdk = j.jenkins.getDescriptorByType(JDK.DescriptorImpl.class).getInstallations();
        assertEquals(Arrays.asList(jdk), j.jenkins.getJDKs());
        assertEquals(2, jdk.length); // JenkinsRule adds a 'default' JDK
        assertEquals("default", jdk[1].getName()); // make sure it's really that we're seeing
        assertEquals("FOOBAR", jdk[0].getHome());
        assertEquals("FOOBAR", jdk[0].getJavaHome());
        assertEquals("1.6", jdk[0].getName());
    }
}

The split would even help the mitigation if something fails in the test case.

The free ride - root causes

1. Reusing setup to avoid duplication instead of creating proper test fixtures
2. Lack of focus—attempting to test multiple scenarios in a single test
3. Insufficient refactoring skills to extract common setup patterns

Wrapping up

We are almost reaching the end of the testing anti-patterns journey, and as such, you might have the feeling that testing is not just something that helps increase the confidence in changing the code or something to be used as a way of regression. Testing can also be used as a way to break down functionality and improve the feedback loop.

It might be a feeling (also known as a smell) or might be something already shared with the software community as The Stranger, but if you see something that seems that needs improvement, it probably needs.

It is also important to keep (when possible) an abstraction between the “difficult parts” in the code such as the type of the operating system, or the file path to save data, we can refer to the cosmic python to dive more on the theme Coupling and Abstraction. Of course we need to test them as well, but we could benefit from different types of testing for that end.

Last but not least, we saw that assertions are also a subject of debate, we saw that sometimes we use assertions that are not the goal to test a given piece of code and it can happen that we just slide in an assertion instead of creating a new test case.

All in all, the testing anti-patterns are context-bounded, which means that having some of those in a code base could be known by the team and adopted as a trade-off.

Regardless of the reasons you might face them in your own code base, we share here four more anti-patterns that could be avoided in the hope of increasing the feedback loop and decreasing the pain perceived by developers when starting with the test-first approach.

References

1. Meszaros, G. (2007). xUnit Test Patterns - Refactoring test code. Addison Wesley.

Appendix

Edit April 21, 2021 - Codurance talk

Presenting the tdd anti-patterns at Codurance talk.

Changelog

• Feb 15, 2026 - Grammar fixes and minor rephrasing for clarity