What Mistakes Can You Make in Designing Your Automated Tests

We subtitled this session "Mistaeks I Hav Made" in Nat Pryce's honour. Facilitated by Paul O'Keeffe.

The Laundry List

Here's the list of gotchas we discussed:

• Too much mocking.
• Unit tests too tight.
• Not refactoring test code as much as production code!
  • Copying and pasting test code.
• Commenting out broken tests.
• Testing at the wrong level.
• Too many tests - take too long to run.
• No cleanup after tests.
• Not starting with clean test data setup.
• Tests that don't respect the domain.
• Only having big integration tests.
• Not having team ownership of tests.
• No test independence.
• Tests that work by coincidence.
• Time/timing dependent tests.
• Not starting with a broken test.
• Testing things that don't matter (for example, minor details of HTML).
• Not testing exception cases.
• Tests that spit out noise.
• Tests that are just plain wrong.
• Tests that don't check anything.
• Not using data driven tests.