Software Test Optimization

Happy Leap Day!  What does your date-time testing test case library all contain?  For finance and reservation systems date-time testing is a high priority.  Your interface for date selection is obviously important and impacts which negative tests can actually be entered (but need to still be attempted if this software is in your scope).  Feel free to send me more test cases: Start/End Date selection: 1. past/future dates 2. invalid dates for numeric date-time entry 3. copy/paste ability 4. no entry error Business Logic considerations: 1. Holidays/excluded dates (e.g. flights) 2. Premium pricing Possible System/Configuration considerations: 1. Time zones 2. Incorrect system date-time 3. Daylight savings time boundaries for entries 4. entry format selection (m/d/y vs d/m/y)

Many of you have probably heard "do the next right thing" in some context, whether a children's movie song or a self-improvement program, but it definitely applies to software test improvement.  Every tester should be empowered to find and do the next right thing.  At every step in the day, we should be analyzing our options for paths forward and seeking better solutions.  But what about the process?  Well, this isn't a license to do everything your own way but it is a license to propose process improvements!

LILY HAY NEWMA N SECURIT Y 02.26.2020 04:59 PM How a Hacker's Mom Broke Into a Prison—and the Warden's Computer Security analyst John Strand had a contract to test a correctional facility’s defenses. He sent the best person for the job: his mother. Great story on security testing, that makes the general testing point: "Trust but Verify" applies to all types of testing.

Are you a gaming enthusiast and NFL fan looking for a software testing contract?  This job could be for you! Quality Analyst (6 - 12 month contract) - Madden Location: Orlando, Florida, United States

Test case generation improvements still lead the way in the future of AI and RPA:  AI for Overburdened Software Development Teams   February 25, 2020  by  Doug Black ...   Testing:  While automated software testing has been around for years, adding AI to the equation can both automate test runs and generate test cases, saving analysts’ time while running more test scenarios. One of these is Intel Capital-backed  Mesmer , which uses RPA to automate mobile app testing. Forrester Research  has reported  on a private equity firm using an AI-based application to automatically create more than 50 percent of the test validation cases used for one of its software projects.   ...

Test procedures store most of the value generated by your test organization.  The defects they generate and the quality of your software ultimately determine the value of your test organization. All test procedures written in a spreadsheet (XLS, GSheet) and in data-driven ready format provides the most flexibility for ease of automation and future manipulation.  Procedures can generate text natural language procedure (from the concatenation of data/procedure) for manual testing.  Separate tabs or worksheets should be used for unique scenarios.  Embrace the data-driven paradigm and change the way your organization views test procedures!

On February 14th we asked for input on standard test libraries .  One of the examples provided shared 20 "scenarios" for Login pages.  As a starting point to standardize procedures, we will use these 20 test items.   The test items are really a checklist without specific procedures and data.  They are organized validly into GUI/Functional, Security, Session, and Configuration/Browser categories.  A state-based approach is recommended for consistently progressing through a collection of data-driven tests. How to create common data-driven procedures from this checklist (and/or from project requirements): 1. Identify what each checklist item is providing for a test procedure (DATA, VERIFICATION as a standard or extended procedure, or a GUIDELINE). 2. Define a standard procedure for each unique VERIFICATION and the variable DATA needed for that procedure. 3. Define rows in the data table to verify each item. 4. Define all steps to execute each procedure. The plain text f

Test Strategy defines your organization's test process, gates, objectives, metrics.  It is maintained by the Test Process Owner and applies to all projects.  An example of a web test quality assurance organization: QA Strategy Our primary objective is to eliminate Escaped Defects and improve user satisfaction. • SQA (Software Quality Assurance) to own defect, escape reviews, and goal reports for the project. • Goals: • Add Frequency and Severity to defect reports     •   Frequency –       • Reproducible - every time (5 of 5)     • Intermittent – (2,3,4 of 5)     • One time – (1 of 5)     • To Classify (seen once but reproduction not attempted) • Severity –       • Critical (loss of data/financial integrity, downtime > 5 min)     • Major (incorrect non-financial data, 1 to 5 min downtime)     • Minor (other) • No Major or Critical Customer Escapes for any release • <10 Major or Critical Escapes found in Beta for any relea

What should you have in place prior to your next software project? 1. Test Strategy (process, gates, objectives, metrics) by Test Process Owner 2. Test Plans for each Project (who, what outline, when, where, how, needs) by Test Project Owners 3. Defect/Escapes Process (tracking and feedback of defects/escapes) 

Looking for industry standards for software testing?   My warning: Do not put all your effort into being 29119 compliant - this should be used as general knowledge and training for software testers.  Check out  ISO/IEC/IEEE 29119 Software Testing   and the wiki explanation .  

W3Schools ( https://www.w3schools.com/ ) is a great site for you to improve your web skills through interactive examples.  Great for manual testers to improve abilities for data-driven testing and automation.

Tweet That!  Test for Twitter   Posted on LinkedIn : Senior Test Engineer   Twitter Seattle, WA, US What You’ll Do Design, develop, maintain and execute functional, regression, integration and end-to-end tests Maintain and enhance automation frameworks that can be used by both Development and QA Perform automated and manual testing efforts for our products Help define, improve and implement QA best practices Represent QA at stand-ups, project meetings etc Who You Are B.S. in Computer Science or related discipline (or equivalent experience) with  5+ years of professional experience  in software QA engineering Experience developing test strategies, test plans, and test cases Experience with UI automation frameworks Experience driving quality through process, instrumentation, requirements and testing Experience developing test automation using Python, Java, etc. Have excellent communication, presentation, and interpersonal skills Efficient in effort estimat

Requesting your help in locating Test Library formats (text, script languages, CSV/XLS, JSON/XML, ...) and examples.  Some of these may be test checklists (e.g. things you could test by generic page or object: ~180+ Sample test cases for web and desktop apps  ) and others may test procedures (e.g. specific to a function or app/page: 20 scenarios for login and search  ). Thanks for your help!

Show some love to your software testers on this special day.  Whether they may verify your product functions or validate your system objectives, let them know you care and that they are special.  Happy February 14th!

If you are at a loss for how to improve your test process, start by rating the severity and impact of defects that escape your development and test process.  Group these escapes by product/system function, test issues (no test or scenario, the test was not run, missing data, defect found but scope/severity wrong, ...), and development/root cause issues so that you can then prioritize changes to attack.  Enjoy!

https://www.thespacereview.com/article/3881/1  (see the entire article for context) Starliner software setback by Jeff Foust Monday, February 10, 2020 "... Process escapes At its Thursday meeting, ASAP said it was concerned the two software errors were symptoms of a more fundamental problem. “The panel has a larger concern with the rigor of Boeing’s verification processes,” Hill said. ASAP called for reviews of Boeing’s software development and testing processes. “Further,” he added, “with confidence at risk for a spacecraft that is intended to carry humans in space, the panel recommends an even broader Boeing assessment of, and corrective actions in, Boeing’s SE&I [systems engineering and integration] processes and verification testing.” ..."

https://blogs.fangraphs.com/instagraphs/job-posting-yankees-baseball-systems-positions/ by  Meg Rowley   February 11, 2020 Please note, this posting contains three positions. Quality Assurance Engineer Position Overview: The New York Yankees organization is accepting applications for a Quality Assurance Engineer as part of their Baseball Systems department. Applicant should have experience working in QA/Testing roles, have strong understanding of different testing types (Functional Testing, Regression Testing, Compatibility Testing), have experience with automating testing frameworks, and knowledge of industry best practices around DevOps and Test Automation. STO NOTE: Proceed and apply at your own risk.  Source has not been verified.

A majority of organizations have widely varying tests that follow “no standard content”  alignment for manual tests and minimal consistency for automation.  As an engineering discipline we crave some apples for apples comparison and consistency.  To establish some common ground for overall process (and not test content yet), please review this article:  https://www.softwaretestingclass.com/how-to-optimize-the-testing-process-and-reduce-development-costs/amp/

In 2020, I don't think there should be much opposition to the view that we need to improve the quality of software development and testing.  Recent events at Boeing and the Iowa Caucus  are examples of software issues in the news (impacting all levels of software from embedded to apps and web development).  Software Test Optimization blog will share information on improving software development and testing in the news and will propose methods for improving software quality through development and test optimization.  Welcome!