Software Test Optimization

Anyone who knows me, knows I have a passion for software testing. Software testing is a diverse field lacking standardization, and software test standardization is a specific area I enjoy.  So when I found this article sharing data on people and skills from applications for a specific software testing position, it was a must read. I highly recommend you spend at least five minutes reading this and thinking about software test standardization:  https://www.kenst.com/2021/03/hiring-a-software-tester-an-analysis/  by Chris Kenst Reviewed by KLOMAR 

Welcome!  To clarify the most significant confusion our viewers face, let me clarify the current KLOMAR focus: KLOMAR can translate your automated software tests (Selenium, RPA, test data, and others) and English language manual software tests into new automation languages or standardized common language English manual tests. KLOMAR does not do language localization or linguistic services. Please let us know how we can assist - even if it is beyond our current focus.

 Data science, and understanding ubiquitous data streams, is clearly a hot technology topic.  The foundation of these exercises is having clean factored data (time, location, source, and all relevant attributes).  Often these data streams are in different formats and have different factors, or these rough streams require data mining to capture and catalog. How many of us have received emails and snail mail generated from dirty data sources?  I personally enjoy seeing these errors in my mail pieces and searching for clues in the defects presented. Do you have text and data conversion projects planned in 2021?  Do you have plans to verify and validate clean data?

 REVIEW:  How many developers does it take to test a product? September 28, 2020  by  Wayne Rash OVERVIEW: A collection of feedback from specific and named software organizations on how they form their development teams. Summary Quotes: " While in most cases, for every two to five developers, one tester would be fine..." "many variables come into play, ranging from  the availability of automation tools in testing  to the income reliability of the project, to how much of the testing can be automated" Software Criticality factor - "mission critical applications, including automotive, heavy truck, and mining equipment product development. A single failure could lead to death or disaster." Further thoughts from the reviewer:  We need a way to rate the efficiency and skills of test team.  AKA some warm bodies can reduce the efficiency of a test team.  We need consistency on how we generate test cases and determine coverage.  We need software ...

Full ariticle :  Five reasons test automation projects fail  from FierceElectronics.com by Ford Arnett If you are about to embark on a software test automation investment, this is a SHOULD READ, and if you aren't - this is a MUST READ to get you going down the automation path! Summary: " Test automation... has an alarmingly high failure rate. Often, this is due to unrealistic expectations, lack of understanding around automation, or not putting enough focus on value." Key Points: " 1. Forgetting automation software is still software" - manage it like other software development.  Have standards, schedules, set deliverables, version it, test it, and re-use - re-use - re-use (don't reinvent if you can help it). "2. Automating ‘just because’" - prioritize functions to be automated and start with high-value tests with the largest time savings. "3. No organizational support" - set the right expectations with all stakeholders. ...

A worthwhile read to understand where software test $40B USD$ are spent but nothing earth-shattering here (summary segmentation below is most of the story): Software Quality Assurance (SQA) Testing Market Set for Rapid Growth and Trend 2020-2026   by basavraj July 21, 2020 Software Quality Assurance (SQA) Testing Market Segment considering Production, Revenue (Value), Price Trend by  Test Consulting And Compliance Quality Assurance Testing Application And Software Testing Risk And Compliance Testing Covering OthersMarket segmentation, Market Segment by Consumption Growth Rate and Market Share by  Application: Artificial Intelligence Testing Cybersecurity Testing Blockchain Testing IoT Testing

Kobiton Press Release Summary of Software Test Survey: 55% said automation would improve software quality 76% are currently automating fewer than half of all software tests 85% said they’d like to automate at least a quarter of all test cases eventually 58% said their automation programs are relatively new or at least six months from beginning 26% are struggling with the process of evaluating and choosing the appropriate test automation tools  17% have found it challenging to train and acquire skilled automation engineers 73% are manually running at least 100 tests prior to each software release 51% are releasing software updates daily or weekly

Let's start the conversation with these questions: How would you describe your software test methodology? Agile, structured, ad hoc, custom, or other. Approximately what percentage of your software tests are automated? What are your software test automation tools? Which software test environments are all used? embedded, desktop, web, apps, IOT, other. What test levels are all formally documented in your software test scope: unit/module, build, functional, system (structure rather than test type) Do you have a formal test strategy or process document guiding your test planning? What are your priorities and goals for test improvements? (such as maintainability, new function efficiency, or defect reduction) After baselining your current test state, the next step is digging into a deeper review of a recent test project's test documentation and assets.  Contact KLOMAR to take the next step!

KLOMAR REVIEW: 2020 State of Testing Report  by Ben Linders, Lalitkumar Bhamare, and Joel Montvelisky Recommendation: skip this one - not worth the eight minutes to review these opinions and summaries. I think the summaries have good hooks but where's the beef?  My initial thought from summaries was, well, this isn't good news such as  some concerning trends here that point to a loss of focus on software testing.  Non-testers taking part in testing appears like an issue at the surface and more information on the distribution of who these non-testers are will review the extent of the concern for me. My overall summary is Software Testing strategies, methods, and organization trends are divergent right now and I believe that should concern software testers and consumers.

REVIEW: " Debunking Five Common Automated Testing Myths BY   ELYSIA LOC K  of DevOps Practice " Summary: Although " automated testing is imperative for DevOps maturity and success", many things hold us back from achieving success with automation, and among these (mis) beliefs and issues are: 1. " We Don’t Have the Time to Slow Down and Add Automation" - many organizations have proven the investment in smart automation pays dividends over time. A few general examples... 2.  " Automated Testing Is Too Difficult and Frankly, Just Not Worth It " -  start with high value low hanging fruit and make a solid foundation for growth. 3.   "We Have to Hire a Specialized Person" -    I will take the counterpoint here and disagree.  I believe that although most people in your org likely can learn and start to automate, THIS IS THE MOST IMPORTANT part of your automation strategy, DO NOT RE-INVENT THE WHEEL .    If you do not have a pe...

REVIEW: 3 Test Design Principles... by Melissa Benua This concise review of priorities and test strategy for continuous integration testing is highly recommended. Quick summary: 1. Reliability "If a test is only 99% reliable (one false report in every 100 test executions), and you run it 200 times a day, then your team will be investigating false-positive failures at least twice daily. "... 2. Importance ..."In a continuous integration system, the most precious resource you have to spend is engineers' time. Engineers have grown to expect results quickly, and they are unwilling to wait for something they perceive as wasting time. So ensure that you're getting relevant results back as quickly as possible."... 3. Specificity ..."But if you only have a budget of five minutes for an entire integration test pass, for example, and every integration test case takes 10 milliseconds, then you have time to run only 30,000 test cases. And if you ...

Do you need custom test software developed or need to convert tests from one tool or format to another?  We specialize in these test development projects!  Contact info@klomar.com or respond to this blog entry for an estimate.

Recruiting software testers into my previous organizations, I have always taken pride in identifying those with excellent test skills whether it be automation, customer validation, test planning, or any other skill required in a well-rounded test organization.  And believing I had a decent handle on those characters who may be suited for other technical positions but didn't have the drive and spine for my test org.  In my view, testers need to see things as broken and have a sense for when they see weakness there could be bigger problems ahead.  The water in that glass is 49.6% full and evaporating quickly! In contrast, software developers are notorious for believing all is good and no need to test in my code.  "If it compiles, ship it." That glass is at least 60% full! Clearly, the intersection of these viewpoints are brought together and everyone is asked to bring DATA for reality. Many jobs take a certain mindset and personality while on the job.  BUT ...

Test case generation improvements still lead the way in the future of AI and RPA:  AI for Overburdened Software Development Teams   February 25, 2020  by  Doug Black ...   Testing:  While automated software testing has been around for years, adding AI to the equation can both automate test runs and generate test cases, saving analysts’ time while running more test scenarios. One of these is Intel Capital-backed  Mesmer , which uses RPA to automate mobile app testing. Forrester Research  has reported  on a private equity firm using an AI-based application to automatically create more than 50 percent of the test validation cases used for one of its software projects.   ...

Test procedures store most of the value generated by your test organization.  The defects they generate and the quality of your software ultimately determine the value of your test organization. All test procedures written in a spreadsheet (XLS, GSheet) and in data-driven ready format provides the most flexibility for ease of automation and future manipulation.  Procedures can generate text natural language procedure (from the concatenation of data/procedure) for manual testing.  Separate tabs or worksheets should be used for unique scenarios.  Embrace the data-driven paradigm and change the way your organization views test procedures!

Test Strategy defines your organization's test process, gates, objectives, metrics.  It is maintained by the Test Process Owner and applies to all projects.  An example of a web test quality assurance organization: QA Strategy Our primary objective is to eliminate Escaped Defects and improve user satisfaction. • SQA (Software Quality Assurance) to own defect, escape reviews, and goal reports for the project. • Goals: • Add Frequency and Severity to defect reports     •   Frequency –       • Reproducible - every time (5 of 5)     • Intermittent – (2,3,4 of 5)     • One time – (1 of 5)     • To Classify (seen once but reproduction not attempted) • Severity –       • Critical (loss of data/financial integrity, downtime > 5 min)     • Major (incorrect non-financial data, 1 to 5 min downtime)     • Minor (other) • No Major or Critical...

What should you have in place prior to your next software project? 1. Test Strategy (process, gates, objectives, metrics) by Test Process Owner 2. Test Plans for each Project (who, what outline, when, where, how, needs) by Test Project Owners 3. Defect/Escapes Process (tracking and feedback of defects/escapes)