In a previous 2i blog we described ‘Algorithms for High Performance DevOps’, an introduction to how an organisation can adopt a metrics-driven approach to DevOps maturity and transformation, which references insights from the manufacturing world most notably Lean and Six Sigma.
In his DevOps.com blog, Matt Dickens also explores the same area, providing a very helpful guide to ‘Choosing the Right Metrics for DevOps Adoption’. This follows on naturally as Matt describes how to populate the framework we suggest, with specific tools and practice related measurement methods intended to boost DevOps performance utilising those insights.
Matt starts with the key point that the transformation is an ongoing, long term process of transformation, and like always the success factor is having a clear end goal in mind and then breaking it down into a series of ‘bite size chunk’ steps to progress along that journey.
His article is very helpful because he provides an initial list for suggestions of what these steps might be for different organisations at different stages of their adoption, and then maps each step to the most effective tools and practices improvements to make to progress against that specific metric, such as:
- Deployment time
- Change failure rate
- Release cadence
- Lead time
- Recovery time
Maturity Model
This is a great approach to maturity model-driven transformation, as it embeds domain expertise in each of the modules that also relates to a broader overall journey featuring a logical link to the other modules too.
For example Matt defines the ‘Change Failure Rate’ as the proportion of releases that include a bug or error, with the immediate benefit of higher-quality code translating to savings in the long term as less time is spent finding and fixing bugs or reworking code.
He then maps these to a number of capability improvement journeys that would affect these metrics, such as Choosing your release tools, Adopting version control, Automating testing and Automating deployments (CI/CD).
This is linked to a module of adopting more test automation into the release pipeline, with a view to positively impacting this metric, and there’s a follow on recommendation where he makes the interesting point that deployment times aren’t actually the key metric improvement to be derived from CI/CD automation, but that rather it’s a goal of advancing ‘Release Cadence’.
In short it’s not how long it takes to deploy but rather how often the team are issuing releases. In traditional waterfall organisations this could be as infrequent as once every 3-6 months+, whereas the ideal of a high performance DevOps team is even many releases per day.
Thus automating testing is a stepping stone that enables a higher release frequency, and Matt proposes that improving that release frequency is the key measure of progress resulting from an investment into CI/CD automation, explaining that:
“rather than releasing huge batches of changes once a month, they might begin to release smaller packages every week. Ultimately, the goal is to have a steady stream of incremental improvements being released on-demand, and usually multiple times per day.
Accelerated release cadences then result in reduced lead times—the period between work being finished and finally released to production. This is beneficial for developers and their end users as more frequent releases create more opportunities to act on user feedback earlier in the process.”
In another 2i blog Going with the Flow, Senior Consultant Greg McKenna also describes this same approach of applying manufacturing throughput concepts to DevOps and software testing, also highlighting key improvements like reducing batch size, as a way of improving the overall capacity while reducing production defects.
Ultimately this is a process of taking a wholesale, systems-centric view of your entire ‘production line’ and improving its quality and capacity throughput, by adopting the right tools in the right ways at the right time, depending on your current progress along this transformation journey.
2i Services
2i can assist organisations to adopt these techniques to also develop the qualities, practices and approaches that your people need to form high-performing Agile Delivery Teams. As testing experts, we can help your organisation embed best practices throughout your DevOps life-cycle and infrastructure management.
This requires a holistic understanding of a large, complex enterprise environment, including multiple technologies, departments and workflow interactions. 2i specialises in mapping this complexity and from that defining a DevOps blueprint that synthesises them together to achieve faster throughput of successful code deployment.
