How to focus on what’s REALLY important when you test…

“How long will testing take?” “We need to do exploratory testing!” “Who’s leading on integration testing?”

All valid questions and, likely, ones that we’ve heard being asked regularly through the years, but should everyone be looking at this differently?

Of the three risk categories (Technology, Operational and Business) we discussed in our whitepaper, “The Evolution of Risk”, we’ve already explored the Technology and Operational risks in previous blogs. In this blog I will focus on Business Risk; why testing risk is important and how testing has a major role in uncovering business risks and delivering successful business outcomes.

Business risk lies within the product that organisations deliver and, if not managed correctly, may lead to loss of revenue, impact brand value and reduce productivity to market. It is a threat to a company’s ability to achieve its financial goals.

Why test risks rather than testing types?

Software trends keep changing to fit in with an unpredictable, competitive market and therefore testing trends also adapt to fulfil software testing needs.

Let’s look at examples of Testing Types vs. Testing Risks.

Testing Risks help organisations answer difficult questions in delivery like ‘When to stop testing’ and it has some very real advantages over Testing Types, such as;

• Reduced cost
• Focused test ideas on significant and real risks to a business
• Testing can be tailored with more ideas around problematic areas
• Overall testing goals and strategies are visible to everyone

How do you identify business risks and focus on what’s important?

Let’s take the example above of a Fund Migration project.

We start with the Product under test and identify all the business outcomes that we aim to achieve by releasing the product to market.

Then, identify all the business risks against those business outcomes considering the factors that may have an adverse effect - Environment, Time, Regulatory Dates, Resources, Infrastructure requirements etc.

An effective impact assessment process can help prioritise significant business risks over risks that have negligible or no impact to the business. This will keep the focus on the most important risks as we won’t be able to test everything.

Test ideas can be produced on the agreed risks and then mapped against individual business outcomes for end-to-end traceability and coverage. Remember that some test ideas may lead to more risks that then need assessed for prioritisation!

In our example above, typical desirable business outcomes could be;

• BO #1 – The data held within the fund and the fund setup should match the design requirement
• BO #2 – All fund reports should run across the required systems (specify systems)
• BO #3 – All the system interfaces this fund is held on are enabled (specify interfaces)
• BO #4 – Fulfil third party system dependencies for migration (specify dependencies)

Now we will focus on the first business outcome, BO #1 and identify as many data risks as possible: Data completeness, correctness, timeliness, consistency, attributes, weekend processing, transformation logic, exception handling, data load, import, export, third party source dependencies, rounding up logic.

During the impact assessment process we may decide that the rounding up logic is not as important a risk in comparison to others and therefore, collectively, we agree to document the “rounding up logic” risk for future references and move on to action other risks that are more significant for the business.

We can then produce ideas against each risk - for example compare the data against current production system to prove that data is complete (data completeness) or verify any additional data that has been requested as a part of migration is correct as per design (data correctness).

We may identify further risks while testing the ideas, e.g. Is the data received within the agreed SLA?, and if not, this may lead to a new risk of load test on the new platform to make sure the performance is within the acceptable benchmark with the current migration approach.

This new risk will then go through the impact assessment process and so on.

What are the benefits of this approach?

There are a number of reasons why this is a great way to approach testing.

It encourages organisations to move away from traditional planning of test phases which sometimes, in bigger and more complex projects, end up causing duplication of work or overlooking some vital elements in the scope.

It also helps you tell your story in status reports, identify gaps in testing more easily and also to grow your lateral and critical thinking skills relating to how to test for a specific risk - there is always more than one solution!

Critically, it improves risk discovery and identifies risks that you might not have considered before enabling you to prioritise testing much more effectively.

Testing risks helps prevent bugs

There is common misconception around “preventing bugs” that needs clarification. Bug prevention does not mean we do not need testing, although it clearly helps to avoid discovering critical bugs at the later stages of delivery.

Working through test ideas will allow you to uncover more information around risk which can be used to review other elements such as the overall design approach or the code itself which could prevent issues further down the lifecycle.

Having reliable traceability throughout the development life cycle to ensure you match stakeholder expectations stops those nasty surprises later in the project where critical functionality isn’t delivered as expected.

Finally, establishing time and effort during the design and analysis phase to discover previously unknown risks allows you to review and expand testing ideas to learn about the risks from the beginning of the product development stage. Prevention rather than detection!

So what does it mean for testing?

With a traditional “test phase” approach, we often uncover critical risks during the peak of delivery and thus spend our money, time and resources trying to mitigate these.

With our progressive approach we can identify risks earlier in the lifecycle and allow for effective mitigation through test ideas.

However, there is a cultural aspect to take into consideration while implementing this testing style. Some organisations adopt a ‘risk means bad news’ stance and believe that exposing risks may cause delays to their projects (or worse). However, if we encourage risk identification at the right time, this will not only increase quality but will also increase confidence in the delivering success.

If this seems of an interest to you, then download our White Paper to explore more about the Business, Operational and Technology risks that can affect your organisation.

This document has references from Dan Ashby’s Risk based testing article.