Imagine my QA team of 50+ members working on a large project, spending vast amounts of time on managing 4,000+ test scripts. The results were always the same, with formal QA testing running weeks’ late and more missed dates due to defects found too far down the process.
The perception that 4,000 automated functional test scripts were passing at 100 percent was enough to give my QA team the sense that all was well. You want regression testing? Sure, as long as development didn’t make too many UI modifications or workflow changes, all could be fine. This of course, was never the case. A change to a requirement would demand the arduous task of identifying and making code changes in the test scripts. What’s more, if a test script failed, then attention was spent on fixing the test script so the test would pass, rather than having the confidence that a defect had been found.
We’ve tried unsuccessfully to improve our QA processes for three years with no success. Our lack of automation strategy was crucial for the inefficiencies throughout the project. The QA team couldn’t make an immediate impact to the development process. The wall between Development, Release Management and QA was firmly in place.
Naturally, we saw a lot of top managers come and go at that time.
Finding a Solution
The sequential nature of our test scripts was too brittle. We needed an automation strategy that required less attention to identify and refactor the effected test scripts and more scalability to detect defects. We had to scale test coverage and find a way to provide immediate feedback quicker and more often to our stakeholders—development and release management.