Our Thinking Fri 9th September, 2022
How to avoid developer burnout on-call
You Build It You Run It is an operating model that empowers product teams to own every aspect of digital service management. When done well, it accelerates your time to market, increases your service reliability, and grows a learning culture.
However, despite all the potential and opportunity for sustained innovation that You Build It You Run It can unlock, there are some pitfalls to avoid.
If you’re not mindful of these pitfalls, they can drain the confidence of your senior leadership, and ultimately put the success of You Build It You Run It at risk.
Given that product teams own every aspect of digital service management in the You Build It You Run It operating model—including production support for the service after building it—there is a risk of developer burnout. Or in other words, an overwhelming exhaustion as a result of constantly feeling ‘on call’.
In our recent You Build It You Run It playbook, my co-author Steve Smith and I take a deeper look into the risk of developer burnout, and how to guard against it.
What does burnout look like for operations on-call?
Picture the scene. You’ve just launched a new digital service to your customers, and your application support team—which consists of four operations analysts—is responsible for managing live service availability 24×7. The digital service is fragile and requires a high level of manual intervention.
The operations analysts manage an on-call rota between them. They’re often paged out of working hours to resolve live incidents, causing frequent disruptions.
Family birthdays, public holidays, sleeping patterns; they’re all disrupted as, and when, incidents occur.
Over time, as a result of being on-call and pulled in out-of-hours, your operations analysts gradually become demotivated, overworked, and fatigued.
This is what burnout looks like. Sadly, working in large organisations with multiple development teams, Steve and I have witnessed it many times.
Over time, and without the right attention, burnout can result in:
- Negative impacts on service availability
- Negative impact on time to restore services
- Negative impact on employee productivity
- Increase in employee churn, and perhaps most importantly
- Negative impact on the mental health of your team.
Why is burnout a consideration in You Build It You Run It?
You Build It You Run It is an operating model in which product teams build, deploy, operate, and support their own digital services. As a result, product team developers are on-call for their own digital services: both in-office and out-of-hours.
In my experience of working with software teams, product team developers are more vulnerable to burnout than operations analysts. That’s because on-call is usually an opt-in consideration for developers, as opposed to a contractual obligation for operations analysts. Developers can feel unprepared for on-call support, unhappy with their on-call remuneration, and/or afraid of callouts for unreliable downstream dependencies.
As a result, it may only be a minority of developers who carry the on-call responsibilities for their whole team. Steve and I refer to this as the limited on-call schedule pitfall.
We’ve seen product teams with only two developers on-call, and they rotate weekly while trying to balance BAU maintenance work with planned feature delivery. It’s not a healthy situation to be in.
A high number of incidents and/or a high percentage of BAU work can cause your senior leaders to doubt the effectiveness of the You Build It You Run It operating model.
However, the real problem is a culture of poor engineer practices, and a lack of technical quality.
How to avoid developer burnout in the You Build It You Run It operating model
When Steve and I discuss burnout with our customers, we describe four key areas to tackle:
- Prepare for on-call from day one. Ensure product team developers are well equipped to handle out of hours alerts. Set on-call expectations in product teams from the moment they start to build digital services. This maximises incentives for product team members to build operability into product features. It gives them time to adjust to a new mindset, in which they could be called out at 0300 to fix their own work.
- Re-architect digital services for adaptability. Aim for digital services that don’t require substantial human intervention, and are fast to fix on failure. Design loosely-coupled product teams and loosely-coupled digital services, to enable graceful degradation when a production failure inevitably occurs. This allows customers to partially use your digital services, and your business outcomes to be partially realised, while full availability is restored in the background.
- Ensure fair remuneration for on-call developers. Adjust developer contracts so they feel compensated for the disruption to their personal lives. Offer a level of pay to on-call developers for 24×7 support that recognises the inconvenience of out of hours support.
- Create a sustainable on-call schedule. Protect developers from spending too much time on-call out of hours. Encourage a product team to manage an out of hours on-call schedule, which safeguards the reliability of digital services and preserves work-life balance for product team members. How product team members help each other to manage the on-call schedule is vital to success. Some team members may be unable to do on-call for family or health reasons, and their decisions need to be respected. Some team members may be reluctant due to a lack of preparation, and any such problems need to be solved.
In summary, you can avoid developer burnout in You Build It You Run It when you embrace these four areas.
By adopting these practices, you can ensure you safeguard the motivation of your team members. In addition to the main benefit—a rested, engaged and contented team—this will help you to ensure high availability for your digital services, alongside a high level of feature throughput and a learning culture.
To find out more, you can continue our You Build It You Run It pitfalls series:
- 7 pitfalls to avoid with You Build It You Run It
- 5 ways to minimise your run costs with You Build It You Run It
- Why your operations manager shouldn’t be accountable for digital reliability
- How to manage BAU in product teams
- 4 ways to remove the treacle in change management
- Why product teams still need major incident management
- Stop trying to embed specialists in every product team
- How to avoid developer burnout on call – you are here!
Our You Build It You Run It page has loads of resources on on-call product teams – case studies, conference talks, in-depth articles, and more. Plus our You Build It You Run It playbook gives you a deep dive into how to make it happen! Get in touch, and let us know what you think.