Share this post
AI can help teams build software faster. Assurance needs to keep pace, giving leaders the evidence to decide whether that change is ready, which risks need attention and whether faster delivery is improving business outcomes and performance.
Are customers finding it easier to use a service? Are colleagues spending less time on manual work? Is a programme delivering the benefits that justified its budget?
These questions sit at the heart of David Orr’s conversation with the Test Associates podcast. As 2i’s VP Consultancy & Solutions, David sees organisations under pressure to reduce costs, deliver change sooner and protect the services people depend on.
At 2i, we believe assurance should help leaders turn delivery progress into informed decisions. That means understanding what is working, where to focus effort and whether a change is creating value. Quality Intelligence brings that evidence together, giving organisations greater control and the confidence to move forward or change direction when they need to.
Watch the full interview
David Orr, VP Consultancy & Solutions, on AI, Quality Intelligence and delivering business value
Video produced by Test Associates — Awards, Conferences & Media, featuring David Orr, VP Consultancy & Solutions at 2i. With thanks to Vaishnavi Nashte, Marketing & PR Manager at Test Associates. This article draws on the conversation.
Agree what better looks like
Technology is involved in almost every business ambition. Yet business and technology teams can still judge progress differently: one looks for an improvement in service or revenue; the other reports what has been delivered.
David sees organisations adopting a product operating model to bring those teams together around the outcomes they need to achieve. They take ongoing responsibility for a product and its value to customers, users and the business. Assurance needs to reflect that shared responsibility, connecting evidence about quality to the outcomes the team is working towards.
That value needs to be specific. For a commercial organisation, it might mean more returning customers or increased revenue. For a public service, it could mean helping citizens access support more easily. For employees, it might mean less time spent working around a system.
Agreeing those measures early gives teams something meaningful to work towards. It also gives leaders a way to judge whether faster delivery is paying off. A change can work as specified and still fall short of the improvement the business needs.
Build on quality engineering
Faster development only shortens the route to value if the rest of delivery keeps up. If teams still wait for evidence about what a change means for the business, the gains made earlier are lost.
Quality assurance and quality engineering provide essential foundations: building quality in, preventing defects and avoiding rework. Quality Intelligence builds on those disciplines, bringing together business context, risk insight, quality engineers’ judgement and AI that accelerates their work.
QI makes those connections more explicit and consistent across delivery. Alongside understanding whether a change works, leaders need to know what it means for critical services, which risks remain and whether it supports the intended business outcome.
For teams, that means agreeing the decisions assurance needs to support before choosing the activities and measures. What must a release owner know to approve a change? Which business processes need protection? What evidence would show that the intended improvement has been achieved?
Those questions give quality engineering a clearer focus and make its findings more useful to the people accountable for delivery.
Focus assurance where it adds value
David describes using AI to assess how changes affect business processes, interfaces, applications and features. That helps teams identify where assurance effort is needed sooner.
At 2i, we use a ‘risk dial’ to help determine how much assurance a change needs. This takes account of the organisation’s priorities, the potential impact on services and any audit or compliance requirements. A change to a critical payment process needs greater scrutiny than a minor cosmetic update.
The real opportunity isn’t simply to test faster. It’s to understand which risks matter most, focus assurance where it adds value and give leaders the evidence to make confident decisions.
Putting that into practice means making the reasoning visible. Teams should be able to explain why a change needs greater scrutiny, which evidence is required and what uncertainty remains. If new information reveals a wider impact, the level of assurance should change with it.
That gives organisations greater control over where effort goes. It also helps avoid applying the same checks to every change regardless of its consequences.
AI can accelerate the work involved, from analysing requirements to generating tests. People remain responsible for judging whether the evidence is sufficient to proceed.
Make the release decision easier
A report can show plenty of activity without answering a leader’s question: are we ready to move forward?
Quality Intelligence brings information from delivery tools together to show how work is progressing, the quality being achieved, the evidence behind compliance requirements and the outcomes being delivered.
For a release, leaders need to understand which business processes are affected, what has been checked and what still needs attention. A high test pass rate means more when they can see whether the outstanding issues affect a critical customer journey or a minor feature.
That context gives them a clearer basis for proceeding while protecting live services and critical operations. It also makes the choices explicit: proceed with an understood risk, seek further evidence or address a specific concern.
As David points out, decision-making can itself become the bottleneck. Evidence needs to arrive in time, and someone must be ready and accountable to act on it.
He describes a progression from control to confidence and trust. Clear controls establish how work happens. Relevant evidence builds confidence in the results. Consistent scrutiny helps leaders judge what they can entrust to AI and where human oversight is needed. Measuring the outcome after release then helps establish whether the change delivered its intended value.
Apply the same scrutiny to AI
Using AI to support assurance also raises a question: who checks the AI itself?
While AI can help accelerate quality engineering activities, AI Assurance focuses on independently evaluating AI systems and their use, assessing whether they are reliable, explainable and appropriately governed.
David discusses 2i’s AI Assurance solution, which assesses accuracy, explainability, robustness and performance. It can assess AI introduced into business processes as well as AI used to build and check software.
That distinction matters when the same model helps interpret requirements, write code and generate tests. Teams need independent challenge to identify assumptions or errors that could otherwise pass through each stage.
They also need a record they can follow: what AI did, what was checked and why a decision was made. This supports the wider purpose of Quality Intelligence: giving people evidence they can scrutinise and trust when making decisions.
Start with context and a clear priority
AI needs to understand the job it has been given. David discusses specification-led development: setting out business processes, rules and data alongside requirements for security, resilience, performance and controls.
Getting that context right improves the prospect of a useful result first time. It takes people who understand both the technology and the service it needs to deliver. As assurance changes, critical thinking and business understanding remain essential to interpreting evidence and challenging AI outputs.
For organisations deciding where to begin, David recommends understanding current maturity before choosing an investment. Our Quality Intelligence maturity assessment looks across eight dimensions and five levels to identify strengths, gaps and opportunities for improvement.
The first priority might be clearer outcomes, better release evidence or explicit accountability for AI. The right choice depends on what is holding the team back.
Start with your next release: can you explain what it will improve, which services it could affect and what evidence you need to approve it?
If those answers are difficult to bring together, that gives you a practical place to start with Quality Intelligence.
How ready is your organisation for the next generation of quality?
Understand where your assurance approach stands today and where to focus next to strengthen decisions, protect critical services and deliver business value.
Explore our Quality Intelligence approach
For a more specific next step, speak to us about a Quality Intelligence maturity assessment.