In our point of view below, we demonstrate why your investment in test and QA can protect you from financial and reputational damage.
With the transition towards banks embracing digital delivery, customers are increasingly interacting and dependant on 24/7 banking services. High demand combined with poor delivery and change management can have a catastrophic consequence with immediate and wide reaching impact to customers. Test and Quality Assurance (QA) is still, incredibly, regarded as an expensive commodity. In our point of view below, we demonstrate why your investment in test and QA can protect you from financial and reputational damage.
Testing & Risk
It is rare to read an article on recent financial mergers and acquisitions without reference to the disastrous TSB migration that made the headlines during 2018 and significantly impacted TSB customers.
A tale which has become the tester’s shield against the rising sentiment of ‘you don’t need testers, you just need automation’, something that many an enterprise transformation initiative has pinned its cost reduction ambitions to.
I recently read with great interest, the article from www.increment.com (‘What broke the bank’) that deduced (through consultation and research) that the culprit was ‘insufficient testing’.
Of course, ‘insufficient testing’ itself does not result in a full-scale production failure event. ‘Insufficient testing’ primarily results in insufficient information and data on risk. Information and data on risk allows decision makers to understand and assess the risk to their customers and their business and then make more informed decisions.
So, of course, the cause of the failure wasn’t ‘insufficient testing’, it was a failure of those responsible to understand the risk and protect their customers until the evidence demonstrated that their customers would not be impacted - (“Banks have a duty of care to their customers, keeping their savings safe and maintaining the satisfactory operation of existing services”).
To date, the failure has cost TSB £330 million (“post migration costs including compensating customers, correcting fraudulent transactions and hiring help”) - a sobering financial cost to measure for any decision maker focused on delivery over quality.
Test Discipline
The article interviews an experienced consultant who states, “they would expect world class design rigour, test discipline, comprehensive operational proving, cut over trial runs and operational support set up”.
This statement underlines two issues that we continually see as a we engage with many organisations who are selling and promoting Agile and DevOps
- Testing has been devalued as a capability. It is now considered ‘embedded’ into test and DevOps teams without enterprise level governance of testing and assurance
- As a result of devolving testing to squads/DevOps, integration and end-to-end risk is not clearly understood and tested for, nor is it tested to any appropriate level of rigour
Striking a balance between being agile but still recognising the risks of enterprise level services and cross platform dependencies is a subject that receives little guidance in the Agile textbooks or DevOps articles.
Historically, this challenge has been addressed manually through large SIT teams testing P2P and E2E journeys, resulting in the need for alignment of a significant number of dependencies and a greater co-ordination of people. Missing dependencies and components are usually only ever highlighted when products are plugged together and tested end-to-end.
“Banking systems continue to become more complex. Internet and mobile have brought the customer closer to the main systems that keep the bank running. The number of ways you can touch a banks IT system has increased”.
Testing and QA needs to ensure it adequately covers both ‘product’ risk and ‘service’ risk. Whilst the potential journeys through both have exponentially increased with the multiple channel delivery model, so have the risks of failure.
As was the case with TSB, when you incorporate 1.3 billion corrupted customer migration records, somewhat not surprisingly, testing gets fairly complicated and failure becomes very costly.
Testing in the age of complexity
This takes us to what the article eventually states is the real root cause “What really broke the banks IT systems, was their complexity (new apps, micro services, active-active data centres)”. The TSB migration data was complex, and on evidence, it seems they didn’t work out exactly how to test it, to reflect the live conditions the migration would operate under. The article states “TSB had walked into a minefield and the bank seemingly had no idea”.
This is the challenge that faces all banks as Fintech disrupts the industry.
Like it or not, they have to become fully functioning tech companies. Senior leaders have to be confident in the technical solutions and delivery approaches they depend on to deliver the levels of service their customers expect (and recognise all the complexity and risk that comes with it).
Banks don’t have the luxury of failing fast. You can try, but it’s probably not going to be very fast and it may cost you £330 million.
The GAFA (Google, Apple, Facebook and Amazon) tech companies are tackling the problem from the other side and trying to address how a tech company can become a bank.
The stakes are high for traditional banks to retain and grow their customer base in an industry of increasing consumer choice and open banking. Quality, consistency of service and trust are a key differentiator between traditional banks and the new players.
Resilience and Risk
TSB provides a lesson to the industry that should be fresh in the mind of every financial institution embarking on transformation. Resilience and risk must be front and centre of their strategies to retain, protect and grow their customer base.
With this comes the assurance of doing business with a trusted and reputable financial institution The article states “The cost of applying resilience and redundancy has come down. Manage risk and fail gracefully. These things are all there, which can help the bank manage their risk and fail gracefully when disaster strikes”. Resilience requires you to first understand the potential risks you need to address. High quality, enterprise-wide Test and QA expertise is key to this.
Modern Test and QA capabilities can no longer trade on purely defect detection as their key objective.
Defect prevention by identifying risk early and supporting the design of resilient systems is key to reducing the impact on customers. This outcome will never be satisfied by introducing testing that is managed as a commodity and offered to the cheapest bidder.
At 2i we believe organisations should continue to invest in understanding the practices of testing and QA and continue to develop, embed, measure and monitor the effectiveness of these practices across the enterprise.
This should then allow decision makers to be confident in the feedback and data they receive from test and QA and enable them to protect their customers from unnecessary risk.
Quality is no longer optional – so what is your risk appetite and protection?
As a result of the TSB migration, the Bank of England issued a discussion paper on operational resilience.
The article states “The paper has a potential change to regulation – making individuals within a company responsible for what goes wrong with a company’s IT systems.” The paper is (here).
The risk to customers will become a personal risk for decision makers, increasing the need for absolute clarity and confidence in the quality position of your solutions.
In the era of disruption, you may find yourself associated with the wrong type of disruption headlines.
Testing and QA needs to be front and centre of your delivery strategy. Our team at 2i incorporate the knowledge and expertise of test and QA across many sectors, and particularly the financial sector, coupled with broad and deep experience of technology to navigate this minefield and ensure you establish the high quality assurance and testing capability you need.
