Test Strategy example

Test Strategy defines your organization's test process, gates, objectives, metrics.  It is maintained by the Test Process Owner and applies to all projects.  An example of a web test quality assurance organization:

QA Strategy

Our primary objective is to eliminate Escaped Defects and improve user satisfaction.

• SQA (Software Quality Assurance) to own defect, escape reviews, and goal reports for the project.

• Goals:

• Add Frequency and Severity to defect reports

    • Frequency – 

    • Reproducible - every time (5 of 5)

    • Intermittent – (2,3,4 of 5)

    • One time – (1 of 5)

    • To Classify (seen once but reproduction not attempted)

• Severity –  

    • Critical (loss of data/financial integrity, downtime > 5 min)

    • Major (incorrect non-financial data, 1 to 5 min downtime)

    • Minor (other)

• No Major or Critical Customer Escapes for any release

• <10 Major or Critical Escapes found in Beta for any release

Gates for Development and Test

• Development – minimal testing and process checks

• Test – production ready under formal test >=1 week for primary projects

• Production – sanity check promoted changes are the same as tested in Test environment

Project Roles and Responsibilities

All Projects will have SQA leading:

• Test Plan for the project

• Defect, escape reviews, and goal reports for the project

• Coordinates execution and review of Tests

• Weekly reporting of status

Test Process 

Test Plans and Defects/Escapes process shall be created by SQA team by Project.  Test Plans should include the following test types:

• Positive /negative functional tests – make sure you cover both.

• Data and configuration coverage – separate the data that drives your tests from the procedures and keep track of user configurations.   (Includes Environments)

• Performance or Stress testing – include configurations and data for new minimal systems, as well as configurations and data that maximize stress and minimize available system resources.  Identify key features that are susceptible to stress in general and  specific stresses of a particular resource.

• Security testing

Test Procedures shall be completed and reviewed by teams prior to team test candidates.  

Test Project directories shall be under (….\SQA\<projects>).

Metrics

Test Procedure development - planned, in-progress, under-review, active with Manual and Automated attributes

Test Execution - planned, in-progress, under-review, Passed, Failed with Manual and Automated attributes

Escapes - defects found after any gate in our release process

F/F - Feature / Function Tree which is a tiered list of Minimal Marketable Features and their decomposition into test items

Short Term Plan (3 months – May, June, July)

• Prototype, review, and propose automation tools 

• In progress with <Selenium> and Microsoft VS/Test leading reviews

• <> also under review

• Work with IT on recommendations – In Progress

• Test Plans – for all new CRs released

• Test procedures – for all new CRs released
   All test procedures to be written in <Spreadsheet> in data-driven ready format.  Procedures can generate text natural language procedure (from concat of data/procedure).

Calendar Year-end Targets (tbd)

• Initial automation running daily (TBD % coverage map)

• Automation Ready – framework and infrastructure support ready for new features