Caution: Loops in automated tests

Be wary of loops in automated (JUnit) tests. I have written automated tests that didn’t actually verify anything because the body of the loop was never executed.

For example:

@Test
public void testVerifyStringLength() {
  for (String s : getStrings()) {
    Assert.assertEquals(3, s.length());
  }
}

The method title implies this test verifies all the strings in a Collection are exactly 3 characters long. However, if getStrings() returns an empty Collection, then this method doesn’t verify anything.

To protect against this, I often add another assertion to verify the Collection is the proper length. For example:

@Test
public void testVerifyStringLength() {
  final Collection<String> strings = getStrings();
  for (String s : strings) {
    Assert.assertEquals(3, s.length());
  }
  Assert.assertEquals(10, strings.size());
}

Here is another pattern:

@Test
public void testVerifyStringLength() {
  boolean testedSomething = false;
  for (String s : getStrings()) {
    testedSomething = true;
    Assert.assertEquals(3, s.length());
  }
  Assert.assertTrue(testedSomething);
}

Conclusion

Nearly any time I verify data inside a loop, I add another check that the code inside the loop was executed.