REVIEW: 3 Test Design Principles... by Melissa Benua

REVIEW: 3 Test Design Principles... by Melissa Benua

This concise review of priorities and test strategy for continuous integration testing is highly recommended. Quick summary:

1. Reliability

"If a test is only 99% reliable (one false report in every 100 test executions), and you run it 200 times a day, then your team will be investigating false-positive failures at least twice daily. "...

2. Importance

..."In a continuous integration system, the most precious resource you have to spend is engineers' time. Engineers have grown to expect results quickly, and they are unwilling to wait for something they perceive as wasting time. So ensure that you're getting relevant results back as quickly as possible."...

3. Specificity

..."But if you only have a budget of five minutes for an entire integration test pass, for example, and every integration test case takes 10 milliseconds, then you have time to run only 30,000 test cases. And if you are doing a UI-based test that is closer to 1 second per test, then you have time for only 300 test cases."...