If you write software you want to do a good job and you want to do it. Quickly. People may tell you that you can’t do both but we are here to tell you that you can continuous delivery works. It’s the best way that we’ve found so far to create great high quality software efficiently. This isn’t our opinion. This is what the data says and empirically the fact that this is the approach that many maybe most of the best software companies in the world.
Choose to take at its heart is an important surprising maybe even fundamental lesson. There’s no trade-off between working quickly and working with high quality. You can have your cake and eat it. You can have speed without compromising quality. This is somewhat surprising counter-intuitive perhaps but it’s borne out by the data you get better software by moving more quickly.
Most organizations begin software development with what seems like a very sensible assumption and then make some bad mistakes. It seems obvious self evidence that doing a good job takes time. High quality work comes at the cost of taking a lot of time. Then most commercial organizations make their big mistake. We don’t have a lot of time everyone talks about first mover advantage so let’s do low quality work to save time now when we say organization. We don’t just mean the man somewhere else we develop as a part of the organization too. We often do this to ourselves as well as having pressure put on us by others to cook corners.
There’s often a kind of social pressure within an organization. An assumption that everybody shares that you can save time by cutting corners. The trouble is that it really depends on the corners that you cut if you’ve ever skipped testing to save time you’ve made this bad trade. If you’ve ever done. A quick and dirty fix you’ve cut corners on quality.
We think of continuous delivery as a lean approach by that. We mean that we aim to work as efficiently as we can identifying and eliminating waste so cutting some kinds of corners. My mental model of lean thinking is very simple. We want to maximize our output while minimizing the work that we do to achieve it. A fundamental tenet of lean thinking is to build quality in this matters. Not only because we value quality in software but that quality matters if we want to move fast and let’s be clear about this.Not every organization does value quality even when they say they do quality is at the heart of our ability to move fast in a sustainable long-term way.
The effectiveness of software development is measured over long time periods not weeks unless you are writing something simple and throw away. That is the data is pretty clear. The state of devops reports repeatedly find that there is no trade-off between speed and quality that is teams who score well on speed. Consistently score well on quality. Teams that score poorly on quality consistently score poorly on speed too. It’s not just that you can create better software faster. It’s that if you want to create better software you need to move faster and if you want to move faster you need to create better software.
These are linked why is this it does sound counter-intuitive. If you want to do work with high quality. We’ll need to spend more time thinking coming up with better simpler solutions to the problems that face us we’ll need to spend more time. Testing. Things too to confirm that our quality is good clearly. This must slow us down well sure this is missing something important though the cost of rework. All that time we spend trying to figure out why we went why it went wrong.
It’s time spent in bug triage meetings time spent trying to figure out how to reproduce the problems that we found time spent supporting customers who are trying to work with broken software. This stuff comes at an enormous cost quoting the state of devops. Reports again teams with high scores in stability and throughput. Those are metrics that measure the quality of our work and the efficiency or speed with which we deliver it. They spend 44 percent more time on new features that’s not 44 more time if you ignore the time taken to do a good job that’s including all of the extra time it takes to do. A good job.
Quality software allows us to make progress more quickly not less so yes. It takes more time to do a good job but this time spent is more like an investment than a cost cutting corners on quality during the development process is rather like deciding to save time by not brushing your teeth you can certainly save a few minutes every day by not brushing your teeth but ultimately your teeth fall out.
When the hp laserjet team decided to switch their development approach to continuous delivery. They did an analysis of how they spent their time before they made the change and after here’s the before and after they spent a lot of time on testing continuous integration test automation and so on techniques focused on improving the quality of their work. But despite this they ended up spending eight times more time on new features.
So good quality code improves our speed what about the other way around what’s the impact of speed on quality well. One of the things that I mean when talking about speed is speed of feedback.We describe great software development has been focused on two things optimizing to be great at learning and working to manage the complexity of the systems that we build.We think that managing complexity is closely related to the technical quality of the systems that we build. I’ll talk about what I think quality means another day in the meantime.
We don’t yet properly understand the problem domain. We may not be wholly familiar with the tech that we’ve chosen to use or the team that we are working with or what our users really want of the system. We certainly don’t know what our design should be or if it will work out as we hope. All of these things are things that we must learn in nearly every software development.
So we think that we should focus on being excellent experts at learning to do that we need to adopt the tools of excellence in learning the ideas at the roots of science assume you’re wrong rather than write work iteratively control.The variables make progress in small steps. Gather feedback make progress as a series of experiments building incrementally on what you’ve already learned. Crucial.
Central to all of these ideas is the idea of making progress in small steps. Small units of work are simpler to understand have fewer places for mistakes to hide are more easily tested and they’re easier to step back from if we make a mistake and they’re lower risk. If we make progress in small steps. We have more opportunities to learn one of the reasons that organizations don’t work in. Small steps is because there’s a kind of overhead associated with making change in software.
If your release process depends on manual testing for example before you are comfortable with releasing changing to production. Then it’s slow expensive not very repeatable and rather risky. But also the cost is a fixed overhead.
Let’s imagine that it takes a week to do manual testing for your system before it’s ready. For release. It takes you a r week to do the release. Then it would be ridiculous to imagine releasing every hour or every day. So instead you’re going to do enough work to justify the overhead of the release process a few weeks. Maybe let’s say three so we do three weeks development followed by one week of testing and releasing and so release we’re releasing changes roughly once per month. Each month. We release about three weeks worth of work if I as a developer introduce a change that causes a problem in week one and if I’m relying on testing to find it best case is that I don’t find out for another three weeks.
The person doing the testing will need to record the bug they’ll need to try and describe it in a way that makes sense to the development team with enough detail so that the developers can reproduce it. So when the bug report comes into the dev development team.
We’ll have to figure out whether it makes sense or not whether it matters or not whether we can afford the time to fix it or not who’s best place to work on it and whether we can reproduce it. If not then maybe we’re going to get into some kind of defect tennis batting the bug backwards and forth between the developers and the testers in the three weeks that have passed I the original developer have moved on now. I’m working on something new and I probably don’t remember all of the details of what I broke.
It’s going to take me a little while to get back up to speed to figure out what’s wrong with this bug as well as that by this time all of the other developers and I have added loads more changes. Some of them may be built on top of the change that’s now causing this problem so this is going to be tricky to unpick. I’m also probably now under a fair bit of time pressure. We found the bug late in the process and we’re either in the middle of the testing week or just after and presumably. The release date is either looming or being delayed awaiting my fix and re-test.
There’s a lot of pressure on me to do a quick and dirty fix. Now so let’s now imagine that we are only going to concentrate on improving our speed nothing else. Let’s start by deciding that we’ll release change once a week or once per month or once every three months a time scale that’s shorter than we can sensibly achieve now one of the big problems will be the manual testing. It’s expensive in terms of time so if our aim is to force ourselves to speed up then we must address that so we replace the manual testing with automated tests.
Initially we do the same tests as before but automated they’re going to run more quickly because they’re automated that gives us an immediate speed up. Let’s say they run in one day instead of a week not very ambitious goal. I admit you can probably do better than that but it will do for the purposes of our example.
If we can run our tests in a day. We can probably run them every day perhaps overnight so now.I’m a developer and once again. I commit a change that breaks something the following day. I’m notified of the failure this time not by some second-hand description of the failure created by a tester but by the presence of a failing test. I’m probably going to be more easily able to figure out that I caused the problem because it was definitely caused by a commit during the previous working day or the day-befores test would have already failed. So.
I’ve fewer potential causes to look at there’s no translation step in trying to describe the failure with a failing test.There’s no question of reproducing the problem. Either we can simply run the test again and watch it fail. I’m not under any direct time pressure now so. I’m working with a simpler problem because there’s been less time to complicate it with more changes with less time pressure because even if it means that I can’t fix it today.
I can probably fix it tomorrow and I have a completely reproducible failure that I can work on which of these two processes do we think will produce higher quality work working with higher quality improves the speed with which we can make progress working to speed up forces us to eliminate waste in our processes optimize our approach improve the quality and speed of our feedback and moves the identification of problems from the end of a cycle to closer to the point to which the problem was introduced so making it easier to fix there’s no trade-off between speed and quality.
In fact it’s quite the reverse working with quality improves speed working with speed improves quality. It’s a virtuous circle.
