Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.
Ever since Mark Zuckerburg uttered the phrase “move fast and break things,” it has become the motto of many development teams. Companies wanting to be the next unicorn decided this must be the way to operate. The race was on to release more, ship faster, never stop. If moving slowly and methodically wasn’t working, doing more had to be the key to success.
These companies would cherry-pick statistics from reports such as the Accelerate: State of DevOps Report. The 2019 report showed elite companies have 46 times more frequent code deployments than low-performing teams and a 2,555 times faster lead time to move from code commit to deploy to support their initiatives to move faster.
The problem is moving fast doesn’t work across industries or for all teams. To effectively move fast, you need processes in place to support the velocity. The consequences of moving too fast and not being able to fix things when they break are high.
How We Got Here
What has gotten us to the point where we are moving too fast? In short, we have. How do you feel when it takes months or a week for a PR to be resolved compared to hours or days? As consumers and end-users of software, our expectations are continually rising. We have to ship quickly because that is what we as consumers expect. As customers, we are pushing companies, which in turn push their employees to meet the increasing expectations.
What are the consequences when you try to move fast and not just break things, but fail?
- It takes longer to resolve incidents.
- You lose customer confidence and sales.
- Employees burnout and you have a high turnover rate.
Let’s go back to the stats from the Accelerate: State of DevOps Report. Yes, elite teams ship faster. But, their changes are 1/7 as likely to fail and they recover from incidents 2,604 times faster than low-performing teams. It’s not just about moving fast and breaking things; it is about having the right systems and processes in place to support this way of working.
Setting Yourself Up for Success
You need two things to effectively move fast: a culture of psychological safety and smart investments in tooling. Employees need to feel empowered to speak up if things are moving too fast, if they are concerned about why a feature is being built and to identify gaps in the processes. They need to feel they won’t be blamed when something breaks. Building this requires empathy, open communication and teamwork. This psychological safety is the foundation of being able to move quickly and quickly recover when things break.
Next up is selecting the right tooling and processes. Invest in tools that make things easier. Tools should be useful, usable and change the underlying problems, not create more.
Think about the tools in place to quickly resolve incidents when something fails:
- Observability and monitoring tools to identify and notify when things go wrong.
- Incident management tools to route, track and escalate issues.
- Feature management tools to enable circuit breakers and load shedding to turn off features quickly.
The culture and tools are part of the equation; the final piece is having the right processes in place to effectively use the tools and support the people.
What are some processes you can implement to enable safety at speed?
- Schedule chaos days to understand how things break and know how to fix them.
- Release features via targeted rollouts, betas, or canary launches.
- Test code in production without exposing it to all users.
- Configure operational feature flags to dynamically change logging levels when an incident occurs.
- Run experiments to gather feedback and ensure features are moving in the right direction before a release.
You should attempt to move fast, you should try to break things—but only when you have the right protections and processes in place. Using a combination of the right tools and processes, you can deliver more value faster without sacrificing quality or your employees’ health.
