Feedback is best when it is given timely and regularly. Our tests are a form of feedback. Below are some methods I have leaned over the years to speed up tests so that they can give faster feedback.
- Analyze and measure the bottleneck just as we do when analyzing a production system to identify bottlenecks, apply the same unbiased analysis on your test suite - try avoid jumping to conclusions such as it is the DB, it is the WS call - usually these are the culprits but sometimes more analysis reveals interesting problems. Measure the performance before and after your change so you can be sure you made an improvement.
- Is the test needed? determine if the test is still required, is it providing enough value to justify existing. Make sure you understand why it was written, how it ended up the way it is and what it is really testing - use the git log, talk to people and have an open mind when asking if it is still needed - assume the authors had the best of intentions when writing the test. Try to understand the problem before concluding that deleting it is the best course of action.
- Can it be re-written? determine whether the test is going about the execution of the system in the right way, investigate to see if other simpler methods of execution and test data setup exist.
- Can the test be pushed down to a lower layer testing a system through the user interface is always going to be orders of magnitude slower than testing it through the interactions of classes and modules. Look at the test scenario and determine if it can be written as a unit or lower level test.
- Mock out external dependencies if the test is talking to a database, web service, file system or interacting with another class or module, investigate to see if that interaction can be replaced with a Mock or Stub.
- Group multiple assertions it is typically considered best practice to have a single assert per test - in most situations this is fine however when the test itself is expensive or the scenario is long running, re-executing a scenario to assert on a different value can be expensive, consider grouping multiple asserts into one test and use Aggregated Failures.
- Re-use test data entities question if each scenario needs their own Customer or Employee, if these entities are expensive to create, perhaps a set of tests can share one? Typically it is best practice to avoid coupling test data to ensure the result of a test is deterministic, sharing test data carries state across tests and can result in non-deterministic behavior as it is could now be dependent on the order scenarios are executed. When determining if test data entities can be shared it is best to use entities that don't store state or data that is immutable.
- Bulk setup test data would aggregating multiple data setup calls into a single call to "Setup test data for Scenario #1" be more efficient? When testing across systems we usually make multiple calls or inserts to a database to setup data - determine if aggregating those operations into a single operation will speed things up.
I am using the more general description of "test" here as I believe these methods can be applied to tests at any level of testing - whether it be unit, integration, functional or exploratory. Of course, certain methods don't make sense at some levels - such as mocking when doing integration testing so some common sense is required.
Also remember that the above methods are optimization's. Avoid prematurely optimizing your test suite. The primary focus should be to have a test suite that is valuable, simple, explicit and cheap to maintain. Applying some of these techniques too early and prematurely optimizing may result in a less optimal test.
