Testing is the key part of the equation when it comes to discovering and understanding risk within a project. An untested component is an unknown and therefore cannot be released with confidence.
Recently 2i engaged with a client in order to help them sort out their testing approach. Testing was done manually at the end of each sprint, which led to parts of the release having to skip or miss testing. Having to release only a partially tested product is dangerous due to the uncertainty and lack of visibility of the risk. There is a fine line between being bold and being reckless. Understanding the risk allows you to be bold.
“To boldly go where no one has gone before”
In order to increase visibility, it was important to bring testing forward and get it done within the sprint, rather than at the end. With limited testing resources, we made the best use of the tools available.
A Pipeline was introduced to enable Continuous Integration with newly automated tests. Whenever a developer pushed an update to the project, the Pipeline would automatically build the system and run the tests.
This provided a broad coverage of the system while the testers were then free to focus on manually testing the other areas. This helped remove the uncertainty from areas that were previously left behind due to other priorities.
A problem we faced was all testing resources were being used to validate the new features being added which resulted in regression testing being ignored. By failing to test that a new update has not caused an adverse effect to an existing feature, introduces uncertainty in whether new bugs have been created. To combat this challenge, we used Cypress to introduce automated functional regression tests to cover this previously unexplored area. These tests were integrated into the development Pipeline, so they ran automatically every time an update was pushed through the sprint. This resulted in bugs being caught earlier and in time to be fixed for release, reducing the risk that an undiscovered issue will negatively affect a client
Another risk which often exists is website performance and up time. If the system is facing heavy traffic and performance suffers, or stops working entirely, there is a risk of losing business and causing reputational damage. To mitigate this risk, we used Postman to test their APIs. This was also used within the Pipeline. We were then able to convert the Postman test scripts into a performance testing script to be used with K6. By executing performance and load tests every build we could ensure continuity of service and be swiftly alerted to any performance issues introduced.
A new issue that has arisen in modern times with the increasing reliance on remote working, is how best to communicate the results of testing. Bugs and issues found are commonly discussed through messages or bug tickets, perhaps using a video call. This makes it harder for the testing team to quickly inform the developers of issues found. We minimised the communication gap by utilising the built-in test reporting capabilities of our Pipeline which ties test results to the build. With well-structured and named tests it made it clear to see what functionality had been broken. This allows developers to get started resolving the problem while the testers can investigate further if required. By improving communication between test and development, more bugs are able to be resolved quicker.
Uncertainty is risk and risk can be dangerous. Testing increases visibility which reduces uncertainty, helping reduce and understand risks. Making use of innovative solutions can make testing more reliable and continuous. Testing is the safeguard of quality and is important it is done well.
