In brief (for leaders):

DevOps promised faster releases and fewer failures together. A decade of DORA research says the two can travel together, most reliably when testing is designed into the flow of work
Where testing was added as a stage near the end of the pipeline, it became the speed bump the business now complains about
A green pipeline shows that the checks someone thought to write have passed. It does not show that the release is safe
Deployment frequency, change failure rate, rework and time to recover belong on one page. Reported alone, the first can flatter the other three

The Unexpected Item in the Bagging Area

Supermarkets across Britain made the same promise when they installed self-service tills. Faster checkout, shorter queues, fewer staff on the floor. For a basket of ordinary items, it delivers. Then comes the unexpected item in the bagging area, and the whole lane stops until someone walks over, taps a screen and waves you on.

The scanning was automated. The exception was not. The queue simply moved to the one place nobody had designed for.

Many DevOps programmes have the same shape. Deployments were automated, often brilliantly. Testing was added as a stage near the end and expected to cope with whatever arrived. The takeaway is this. Speed and safety are not a trade-off, and the evidence says organisations can have both. They arrive together most reliably when testing is designed into how work flows, rather than queued at the end of it. Where the design is wrong, the gate becomes the bottleneck, and the business concludes that quality slows it down. Quality is not the problem. Placement is.

What a Decade of Evidence Says About Speed and Safety

A widely held belief is that quality gates are the brake on delivery, and that going faster means loosening them. Unfortunately, the DORA research programme, now run by Google Cloud and built on more than a decade of annual surveys of technology professionals, points the other way. Its consistent message has been that teams need not trade throughput against stability. DORA also identifies test automation as one of the technical capabilities associated with stronger delivery performance, alongside continuous integration and working in small batches.

Two cautions apply. The data is survey-based, so it shows association rather than proof of cause. And the measure most closely tied to testing, change failure rate, has been the awkward one. DORA’s 2024 analysis described it as an outlier among the original four key metrics, and RedMonk’s reading of the report noted that the medium-performing group showed a lower change failure rate than the high-performing group. DORA’s response was to add rework rate, which counts the unplanned deployments needed to fix user-facing defects. The measurement is still being refined.

That is a reason to treat these numbers as signals to investigate rather than targets to hit. Nathen Harvey, who leads DORA at Google Cloud, has discussed Goodhart’s Law as a common pitfall of metrics: once a measure becomes a target, it stops being a good measure.

Three Design Flaws That Turn a Gate Into a Speed Bump

None of these is about people. All three are about how the process was built.

Testing as a stage rather than a property. When quality assurance is a stage, work queues at its door. Larger batches arrive less often, and each carries more risk. In a 2024 Quality Bits conversation, Alan Page recalled the advice that the best way to expose quality problems is to attempt continuous delivery. When a team cannot release continuously, the likely reason is how feedback was designed, not how hard anyone works.

Gates that cry wolf. A suite that fails for reasons unrelated to the product teaches teams one lesson: rerun it until it goes green. A gate nobody trusts is a speed bump. A gate everybody bypasses is decoration. A gate that blocks on noise is a tax. As the quality debt post in this series argued, flaky automation is often a symptom of underlying instability, not a tooling problem.

Checks mistaken for testing. A pipeline is superb at asking questions someone already thought to ask. James Bach and Michael Bolton separate that automated checking from testing, which is the investigation of what nobody thought to ask. A green pipeline therefore says that no known check failed. It is the green dashboard from the first post in this series, relocated into the delivery pipeline.

The Bill Nobody Presents

Take a deliberately simple illustration. An organisation makes 2,000 production deployments a year across its teams. At a change failure rate of 15%, 300 of them fail. If each failure consumes 30 hours of engineering, support and leadership time at a blended £70 an hour, that is £2,100 a failure and £630,000 a year. Bring the rate down to 5% and the bill falls to £210,000. The difference is £420,000, before a single customer is counted.

The more revealing number sits on the dashboard. If each failure is fixed with a further deployment, 300 of those 2,000 deployments are repairs. The dashboard reports 2,000 releases. The business received 1,700 changes and 300 corrections. Deployment frequency has been counting the clean-up as progress.

The assumptions are mine and illustrative. They are not DORA benchmarks. Substitute your own.

AI Narrows the Gate Further

Artificial intelligence adds pressure. DORA’s 2024 report estimated that a 25% increase in AI adoption was associated with a 1.5% fall in delivery throughput and a 7.2% fall in delivery stability, and pointed to larger batch sizes as a likely cause. The 2025 report shows the throughput association turning positive, yet AI adoption is still, in the report’s words, associated with increasing instability. DORA’s summary is that AI amplifies what is already there.

Feed more change, faster, into a poorly designed gate and you amplify the gate. The earlier post on AI and your testing strategy made the same point about test tooling: AI amplifies the practices you already have, blind spots included. The point here is narrower: more change is now arriving at the same queue.

Designing Testing Into the Flow

Last month’s post argued that headcount is an input, not an outcome. The same goes for pipelines. The organisations that get this right differ in design, not in tooling:

Shrink the batch. Small changes are quicker to test, easier to diagnose and cheaper to reverse.
Move feedback to where the change is made. Fast, trusted checks run by the team that wrote the change free skilled testers to ask what the checks cannot: what could harm a customer that nobody has thought of?
Reserve hard stops for real risk. Every gate should exist because a named business risk justifies it, and be retired when it stops doing so.
Design for testability. If the architecture makes a service hard to observe or isolate, every gate downstream will be slow, however good the team.
Measure the set. Report deployment frequency, change failure rate, rework and time to recover together, and trend how long a customer-affecting defect lives before anyone knows.

The Question Leadership Should Ask

THE LEADERSHIP QUESTION “Did DevOps make testing faster, or did it just make failures faster?”

Put it to engineering and test leadership together, not separately. Ask for three numbers on one page: change failure rate, the share of deployments that were fixes, and the time a customer-affecting defect stays undetected. If they cannot be produced, the pipeline is reporting activity again. If they can, ask which gate in the pipeline protects against a named business risk, and which exists because it always has.

The Path Forward

This is not an argument against DevOps, automation or speed. It is an argument against mistaking automated deployment for delivery confidence. Testing is not what slows the pipeline. Where testing sits in the process is often what slows it, and that is a design decision leadership owns.

Make it a standing rule that no board or steering group sees deployment frequency without change failure rate and rework alongside it. Speed that cannot show its failure rate has not been measured.

Sources

DORA / Google Cloud. (2024). Accelerate State of DevOps Report, and announcement summary. https://cloud.google.com/blog/products/devops-sre/announcing-the-2024-dora-report
DORA / Google Cloud. (2025). State of AI-assisted Software Development. https://dora.dev/dora-report-2025/ and announcement by Nathen Harvey, https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report
DORA. Capabilities (continuous integration, test automation, trunk-based development). https://dora.dev/capabilities/
Stephens, R. (2024). DORA Report 2024: A Look at Throughput and Stability. RedMonk. https://redmonk.com/rstephens/2024/11/26/dora2024/
Stephens, R. (2025). DORA 2025: Measuring Software Delivery After AI. RedMonk. https://redmonk.com/rstephens/2025/12/18/dora2025/
Harvey, N. (2024). DevOps Diaries, Episode 32: Using DORA to measure your team performance with confidence. Gearset. https://gearset.com/devops-podcast/episode-032-nathen-harvey
Page, A. (2024). Modern Testing with Alan Page. Quality Bits podcast. https://www.buzzsprout.com/2037134/14599858
Bach, J. & Bolton, M. Testing and Checking Refined. Satisfice. https://www.satisfice.com/blog/archives/856
Goodhart, C. (1975), as popularised by Strathern, M. (1997). ’Improving ratings’: audit in the British university system. European Review, 5(3).

Notes on limitations

DORA findings are survey-based associations, not controlled experiments. The AI estimates (per 25% increase in adoption) come from a subset of roughly 3,000 respondents in 2024.
DORA moved from performance tiers to team archetypes in 2025, so year-on-year comparison is limited. Change failure rate benchmarks quoted in secondary sources vary and are not used here.
The £ calculation is illustrative. The 15% and 5% rates, 30 hours per failure and £70 blended hourly cost are assumptions, not benchmarks.
The Alan Page reference paraphrases advice he recalled in the interview, not a view he originated.