In the previous post in our ‘Beyond DevOps’ series, Nick described how our People team is now focussing on collaborative DevOps/Operability Engineers. I’d like to follow that with an example of how Equal Experts has contributed to operability practices on a complex production system.

I’ve been working at HMRC Digital for the past 3 years as the Technical Delivery Manager
for the award-winning Tax Platform. Equal Experts has partnered with HMRC since its first
digital delivery centre opened in 2013, working in blended teams alongside HMRC Digital’s
own people and several other suppliers.

Throughout this collaboration, everyone has shared an emphasis on creating a highly
operable Platform-as-a-Service. This has really benefitted the teams running digital services,
and the teams managing the Tax Platform itself.

DevOps at HMRC Digital

HMRC is responsible for over two thirds of all government transactions, and in 2012 it
published its Digital Strategy. Since then HMRC Digital has created 5 digital delivery centres
across the UK, with 60 multidisciplinary teams responsible for more than 100 digital services
built to the Digital Service Standard. Those services run on the Tax Platform – a cloud-based, Platform-as-a-Service that provides a scalable, durable production environment.

The Digital Delivery Unpackaged guide from HMRC Digital is a great resource for understanding HMRC’s approach to digital delivery. At the outset, there was a Web Operations team responsible for cloud infrastructure, delivery enablement, and release engineering. Some teams expected Web Operations to be “the DevOps team”, taking care of all aspects of their live services. As demand on Web Operations grew over time, we worked with HMRC to clarify how DevOps applies to the Tax Platform.

Maximum operability with minimal friction

At Equal Experts, we’ve always seen DevOps as a philosophy of collaboration, and you
don’t need DevOps Engineers embedded on a team to have that philosophy.

At HMRC Digital, the goal is to give teams all the capabilities they need to develop, test, deploy, and operate their live services. This is sometimes known as “You Build It, You Run It”, and it’s been a key enabler to supporting so many teams on the Tax Platform. A Platform-as-a-Service offering maximum operability with minimal friction is easier for teams to operate safely and reliably.

Operability on the HMRC Tax Platform

Since 2015, the responsibilities of the WebOps team have been clarified by creating a number of smaller platform teams. Now Web Operations concentrate on cloud infrastructure, while other platform teams collaborate with service teams to understand their needs.

Those teams have created a number of operability-focussed tools:

1. A “New Service” button to initialise a GitHub repository with logging and metrics
   libraries
2. A “Logging Dashboards” DSL to create Kibana logging dashboards with request,
   error, and deployment data
3. A “Monitoring Dashboards” DSL to create Grafana monitoring dashboards with
   request, response time, and error graphs
4. An “Anomaly Detection” DSL to create Sensu alert configurations with default checks
   for errors and slow response times
5. A “Service Runbook” template to provide a standardised approach to documenting
   service operations and incident response

These tools encourage teams to think about operating their live services from the outset, anticipate abnormal operating conditions, and respond to incidents when they occur.

DevOps isn’t just about using infrastructure as code. Its real value comes when you embed operability into your ways of working. DevOps-style collaboration on operability practices at HMRC Digital have really paid off. The Tax Platform has been built to the needs of its digital services, and the citizens that rely on those services every day.

This is the fifth article in our multi-author series “Everyone Does Continuous Delivery”, which aims to explore DevOps and Continuous Delivery – and how they affect our culture and work. Keep an eye on the blog or follow us on Twitter for the latest updates.

Part 1: DevOps as a Constraint
Part 2: The value of Operability
Part 3: DevOps is just a conversation starter
Part 4: Looking for DevOps/Operability Engineers
Part 5: Operability for a Platform-as-a-Service (you are here!)