REVIEW: 3 Test Design Principles... by Melissa Benua


REVIEW: 3 Test Design Principles... by Melissa Benua

This concise review of priorities and test strategy for continuous integration testing is highly recommended. Quick summary:

1. Reliability

"If a test is only 99% reliable (one false report in every 100 test executions), and you run it 200 times a day, then your team will be investigating false-positive failures at least twice daily. "...

2. Importance

..."In a continuous integration system, the most precious resource you have to spend is engineers' time. Engineers have grown to expect results quickly, and they are unwilling to wait for something they perceive as wasting time. So ensure that you're getting relevant results back as quickly as possible."...

3. Specificity

..."But if you only have a budget of five minutes for an entire integration test pass, for example, and every integration test case takes 10 milliseconds, then you have time to run only 30,000 test cases. And if you are doing a UI-based test that is closer to 1 second per test, then you have time for only 300 test cases."...

Comments

Post a Comment

Popular posts from this blog

Don't Let This Be YOU: Mazda bricks Radios with Software Defect

Image
 ARS Technica article on Mazda radio defect by  JONATHAN M. GITLIN  -  2/9/2022 SummARY: A critical real-time defect in Mazda radios was triggered by a Seattle radio station.  If you were unfortunate enough to have this  2014 to 2017  Mazda radio and live in the Seattle area listening to  94.9 FM , the radio was no longer usable.  Files with no extension were (like .jpg) were the use case that sent the radio into constant reboot condition rendering it useless.  Read the full article at link above.  Yes, I was slow to find this gem.  Please share if you find more like it!
Read more

F35 System Testing

Image
Many calendar years of development and testing have gone into one of the most complex vehicle systems ever -  the F-35. This article details how performance and  End User / Operational Testing of F-35  is being performed.
Read more