Today’s most successful organisations are fast and flexible. They can deploy new features for digital services to meet rapidly evolving customer demands and market opportunities. They capitalise on first-mover advantage and sustain innovation: improving digital services incrementally through weekly—or even daily—deployments.
For many businesses, maintaining or achieving rapid time-to-market is a constant challenge. If the same is true for you, you’re not alone. We constantly hear the same type of complaints from many different organisations, at every level:
“IT is too slow.”
“We can’t get new initiatives delivered quickly enough.”
“New features consistently run late in delivery or get bogged down in approvals processes.”
“We don’t react to the needs of our customers.”
“The tech team is a constant bottleneck.”
In many cases, it may not actually be the tech team causing the blockages that impede your ability to deliver value, at pace, consistently. It could be the way your tech team is structured that’s the issue. And adopting an alternative operating model—called You Build It You Run It—could be the answer to your problems.
We’ve helped clients make enormous gains in deployment throughput by adopting You Build It You Run It: from 10 deployments a year to 4000+, without a correlating increase in production incidents.
So, how do you manage deployments in a You Build It You Run It operating model, and how does it differ to your—likely—current deployment process (often referred to as ‘Ops Run It’)?
Deployment in ‘Ops Run It’: the common, conventional IT approach.
Many software teams use an operational model called ‘Ops Run It’. In this model, Operations teams are charged with the responsibility of deploying and supporting applications in production. The above graphic highlights a composite IT department, where Delivery and Operations function as silos.
In this approach there are many delivery teams building and testing software services. Once those services are built and tested, there’s a handover. In Operations, there’s a Change Management team running change-advisory board meetings (or CAB meetings) to ensure the services are stable and safe. Then, there’s an App Support Team doing the deployments.
How does ‘Ops Run It’ work in a day-to-day capacity?
- The Delivery team adds a delivery request into the change-management queue.
- Each delivery request is approved or rejected in a CAB meeting.
- An approved change is added into the deployment queue.
- The Application Support team perform the actual deployment at the scheduled time.
- Post-deployment validation testing will be performed by either the Operations App Support team or, in some cases, the Delivery team.
What are the drawbacks in this approach?
- It’s incredibly slow to approve a change request: in this operating model, deployments are mired in bureaucracy because it takes hours, days, or even weeks to approve a change.
- The act of deployment itself is slow-to-complete: it can take hours or days to actually complete deployment. Additionally, if the deployment tasks are not automated, the act of deployment can be risky and time consuming. In some cases, the architecture itself may prevent teams from being able to make changes while the system is running. This means all deployments must be timed to correlate with planned outages. If the team does not have automated production verification tests, then testing must be completed manually after each deployment. All of these factors—whether individually or in conjunction—can contribute to lengthy deployment times.
- It’s slow to synchronize knowledge between teams during handovers: every time the delivery team completes a piece of work, they need to conduct a handover process to pass the deliverable to Operations for approval and deployment. This involves a detailed summary of what was built, why it was built a specific way, and anything to be mindful of in support cases. Providing this context is time consuming and often arduous, given the Operations team was not involved in building the change and typically have very little context as to why it is important.
- You can’t sustain innovation with Ops Run It: while Ops Run It is suitable for commercial-off-the-shelf software (COTS) applications and foundational systems, it does not facilitate the rapid time-to-market required for sustained innovation. This is because it is slow to approve, implement and learn from new product features.
Deployment in You Build It You Run It: how to launch new product features to market every day.
The above graphic shows a hybrid model.
It’s important to note that You Build It You Run It and Ops Run It are not binary propositions. It’s a common misconception that you can simply choose one model over the other and employ it universally. In most cases, you’ll have different models working together, because different services have different requirements. You should also frequently review your operating model depending on the maturity or lifecycle of the product/system you’re supporting.
Here, the Delivery team employs the You Build It You Run It operating model for all activity involved with developing, testing, deploying and evolving digital services.
The same Delivery team could be using an Ops Run It model—in conjunction with a central Operations team—for work associated with commercial-off-the-shelf software (COTS), foundational systems, or even mature digital products that require little intervention.
How does You Build It You Run It work in a day-to-day capacity?
In You Build It You Run It, an on-call Delivery team has total ownership over developing, deploying, monitoring and supporting digital services. As a result, there is no hand-off from Development to Operations.
The Delivery team has all the skills and context needed to do this across the entire lifecycle. It’s made possible by a loosely-coupled service-oriented architecture, and an automated deployment process.
In the You Build It You Run It operating model, the on-call product team will determine for themselves whether what they want to do is a regular, low-risk change. We trust they’re in the best position to make this call, because these team members have built the service and best understand the intricacies of its codebase.
If the Delivery team deem it is a regular low-risk change, they use a pre-approved change request process, and it’s managed by the team as a standard change. This process is extremely fast when implemented correctly—updates can be deployed daily to help organisations sustain innovation and consistently deliver value.
If the Delivery team deem it is an irregular or high-risk change, it goes through a normal CAB process with the same change-management team from Operations.
When a deployment happens, the Delivery team perform it themselves—at a time of their own choosing—and report their successful changes to the Change Management team.
Critically, this removes the bottleneck you see in Ops Run It where the Delivery team is dependent on the App Support team.
In You Build It You Run It, we also typically see Delivery teams being much more considered and cautious with their deployment times than an Operations team might be. This is because the Delivery team themselves are responsible for fixing any problems that arise from the deployment.
Often, in scenarios where deployment is automated and the architecture is built in a way that doesn’t require downtime for deployments, Delivery teams will deploy during the day to give themselves a working day to monitor for issues and implement immediate fixes as they arise. This, in turn, means fixes are provisioned faster and services are more reliable in general.
What are the benefits of this approach?
In the right environment, there are many benefits to You Build It You Run It. The top three we typically see are:
- It’s much, much faster to approve change requests: in this operating model, an approved change request for a regular low-risk change can be obtained within minutes or even immediately. As opposed to days, weeks, or months.
- It’s much, much faster to complete deployments: the deployment process can be fully automated by the Product team using the deployment pipeline. Crucially, even if you choose not to automate this process, there is no handoff to any other team as part of the deployment workflow; the team is entirely self-sufficient.
- It’s much, much faster to synchronise knowledge: there’s no requirement for meetings or phone calls to facilitate a deployment handover for an Operations team. The only requirement is to synchronise knowledge within the Product team; through an approach like paired development, everyone on the team has far greater clarity of what’s happening at any point in time.
So how do I get started with You Build It You Run It?
Like many things, You Build It You Run It can be incredibly powerful when implemented in the right context. But it’s not necessarily the right approach for every organisation universally.
If you’re interested to learn more about You Build It You Run It—including whether it’s right for you, and how you can start to determine its suitability for your team —I’d love to set up a conversation.
We all know the traditional enterprise IT operating model. Delivery and operations teams are on opposite sides of a wall. A delivery team builds a software service, then hands it over to operations. This causes a lot of problems, and makes it difficult to achieve business outcomes.
In our latest playbook, Steve Smith and I have used our years of experience building digital services and digital platforms with our customers to describe how we’ve helped them to achieve superior IT performance. They’ve used an operating model that empowers product teams to own every aspect of digital service management, including deployments and on-call. It’s called You Build It You Run It.
Drawing on years of experience, we wanted the playbook to be something of real value. A deep piece of content that would explain how to implement the You Build It You Run It operating model for the constantly evolving digital services we work with.
But before you jump into how to implement You Build It You You Run It, I’d like to explain this model of technology operations. I’m going to do this with Simon Sinek’s Golden Circle model. Starting with the why, then looking at the how, and finally working out the what. Here goes.
Why is there a problem?
Our customers want to sustain their own innovation. They want to satisfy the constantly changing needs of their users. And they want to stay ahead in competitive markets. But, many of them are facing tough challenges that prevent this from happening.
- They can’t accelerate their time to market. They need to launch new features to customers at least weekly, and ideally daily. Accelerate by Nicole Forsgren et al found organisations with frequent deployments were twice as likely to exceed profitability expectations.
- They can’t reduce the cost of failure. They need to realise the opportunity costs, but can’t because failures take too long to resolve. A Fortune 1000 survey found the average cost of a critical failure was between $0.5M and $1M per hour.
- They can’t nurture a high-performance culture. They need to foster teams with a high degree of psychological safety and strong decision-making. A typology of organisational cultures by Ron Westrum demonstrated that a bureaucratic culture results in little trust, low collaboration, and poor quality decisions.
We think of an operating model as insurance for business outcomes. In our experience, the traditional IT operating model of one operations team running many digital services on behalf of many delivery teams just can’t achieve the throughput, reliability, and culture required to sustain innovation.
A different operating model is needed. One which offers the right insurance for your digital business outcomes.
How can we think differently?
We can reject the division between delivery and operations teams first recommended by the COBIT framework in 1996. Its pre-Internet rationale simply doesn’t hold true today. Instead, we can look at the entire lifecycle of a digital service from cradle to grave, and ask ourselves: what’s needed to simultaneously achieve high throughput, high reliability, and a learning culture?
With that lens, we want an operating model for digital services to have:
- A focus on customer outcomes, not software outputs
- A minimum of handoffs when building and running live services
- An emphasis on knowledge sharing and learning opportunities
- A powerful incentive to constantly prioritise operational concerns alongside product features
- A means to generate powerful insights, and disseminate them far and wide
This naturally led us to You Build It You Run It.
What is You Build It You Run It?
You Build It You Run It was coined back in 2016, when the CTO of Amazon Werner Vogels said in an interview:
“Giving developers operational responsibilities has greatly enhanced the quality of the services, both from a customer and a technology point of view. The traditional model is that you take your software to the wall that separates development and operations and throw it over and then forget about it. Not at Amazon. You build it, you run it. This brings developers into contact with the day-to-day operation of their software. It also brings them into day-to-day contact with the customer. This customer feedback loop is essential for improving the quality of the service.”
In its simplest form, You Build It You Run It is an operating model for modern engineering and technology operations. It’s the (long) name for on-call product teams owning all aspects of their digital services, from inception to decommissioning. Product teams do their own production deployments, launch features to live traffic, monitor customer behaviours, and respond to incidents themselves. You Build It You Run It transforms technology operations from reactive ticket management to proactive continuous improvement.
Adopting You Build It You Run It means comprehensive changes for people, processes, and technology. It requires the creation of cross-functional teams who are responsible for the development, testing, and production support of digital services. It means redefining roles, streamlining service management processes, and building a fully automated toolchain from deployment pipeline to incident management.
Our You Build It You Run It page has loads of resources on on-call product teams – case studies, conference talks, in-depth articles, and more. Plus our You Build It You Run It playbook gives you a deep dive into how to make it happen! Get in touch, and let us know what you think.