As we work with existing and new clients, we are still seeing increasing amounts of organisations looking to implement the principles of DevOps in the Enterprise.
This can often seem like a development initiative, that looks at DevOps as purely a technical solution. However, testing and QA is at the heart of DevOps and continuous delivery. Work can only move through the delivery flow, after careful test, inspection and measurement to ensure value is delivered to the customer and not risk.
The principles of DevOps are addressed by the 3 ways:
- The First Way: Principles of Flow
- The Second Way: Principles of Feedback
- The Third Way: Principles of Continuous Learning
In this article, we will look at The First Way: Principles of Flow.
This principle requires testing to no longer be considered a ‘stage’. We have long been hearing the term ‘Shift-left’. In DevOps, there is no left.
Flow describes one continuous system of work (without Silos), from idea to production. This requires a test and QA process that is embedded in the DevOps processes themselves. Quality needs to be ‘built in’.
Here are some of the key principles of ‘Flow’ and the role of testing and quality assurance practices required to enable ‘Flow’.
Making work visible
We have all by now seen Kanban and scrum boards dotted round offices which show the visible work of teams. DevOps isn’t all whiteboard and post it notes for show.
The purpose is for the teams to ensure the “work flows” from left to right as quickly as possible. To allow work to be assessed as being ‘Done’ (e.g. all unit tests passed, acceptance tests passed and automated etc), testing and QA practices are built into each and every stage of the work flow.
Be it Test Driven Development, automated unit test, acceptance tests etc. DevOps requires constant feedback on feature quality to enable the work to flow to the customer.
Limiting work in progress
For those seasoned testers who have been involved in large enterprise projects, they will be familiar with testing being called out as a ‘bottleneck’. This is often the result of testing being the first lens to bring focus to the gaps and issues impacting a large delivery.
By limiting work in progress, the blast radius is contained. Where work is stuck in one stage of the delivery process, it becomes obvious and the team ‘Stops the line’, focusing all team members to ‘swarm’, solve the issue and unblock the delivery pipeline. There is no value to the business of change stuck in testing.
Reducing batch sizes
Reducing batch sizes to enable faster delivery, requires testing to consider how to test small change rapidly, while ensuring the change footprint has not regressed other functions.
This requires great team collaboration to determine the right level of testing for change delivery, that ensures risk is tested for without the need to large scale regression testing on every change.
This prevents smaller batches of work being bottle necked in the delivery flow by tests that are not necessary and also makes it easier to find and resolve defects in the process.
Reducing the number of handoffs
In large enterprise deliveries, testing is often delivered by a separate team from feature development.
This requires the hand off across teams in the form of information, asset review, input to new tooling systems etc, which incur a time and effort cost. Where the testing is fully owned by the DevOps team, the cost of hand off is vastly reduced. Teams look to share information, use common collaborative tool sets, and reduce duplication of effort.
By owning test and quality as a team, effort and risk of poor/incorrect hand over to different teams is vastly reduced. This approach enables defect prevention, as opposed to trying “test quality in”.
Identifying and removing constraints in the process
This is where DevOps practices start to tackle waste and wait time in the delivery flow. Areas that have historically increased the cost of testing for Enterprises can be addressed through DevOps practices and accelerate test delivery.
On demand environments – Through use of Cloud, Virtualised Machines, Containers etc, environments can be deployed in a pre-configured state on demand and build and burn. Environment congestion no longer burns the testing budget.
Code Deployment – Automation of deployments reduces the testing dependency on separate teams. DevOps practices with tools such as Jenkins allow on-demand to build and deploy which is configured, invoked and orchestrated by the tool.
Test set up and run – It is key that testing keeps pace with the rate of development in DevOps. This is only possible with automated testing that is both broad and deep in terms of coverage.
Loosely coupled architecture – Overly tight architecture increases the risk of change delivery. The footprint of change may only be uncovered through significant discussion across teams, or at worst through defect discovery late in the release cycle. DevOps practices looks to loosely couple architecture to ensure that change can be delivered with a well understood and tested footprint.
Eliminate waste in the value stream
Value Stream Mapping – This approach captures all work required to deliver value to the customer. It provides end to end visibility of the design, development, test and operations effort required to deliver change and the lead time to get from idea to customer value. This enables the whole team to identify process and metrics for improvement. For example, ensuring testing is risk based with high automation would target reduction of time in test for a feature.
Defects – As we know, defects incur a significant cost of delay for delivering change to the customer. Incorrect, unclear or missing information creates additional and unnecessary effort. As test is embedded in DevOps teams, there should be team ownership of quality. All required dependant information to enable defect prevention should be embedded in the teams DevOps practices and this will ensure when defects are detected, they can be resolved at pace. Defects are a form of waste in the process, and the focus should always be on defect prevention.
So, to enable work to FLOW, DevOps addresses many areas that can be historically classified as the ‘cost of testing’ (environment set up, deployment, configuration and data set up).
This is the significant amount of effort spent to get to the point where a test can be executed and evaluate a result. This is when we provide the confidence that the change will deliver value and not risk to the customer and that the change can be deployed.
This is key to ‘Flow’.
In the next article, we will look at the ‘principles of feedback’ in DevOps.
References: The DevOps handbook: Kim, Humble, Debois and Willis
