HM Revenue & Customs

A case of
pandemic agility.

How Equal Experts helped HMRC support the economy in four short weeks

In March 2020 the UK went into lockdown, causing the most brutal recession in living memory. The Government had to react quickly, with policies to help the people and businesses in the United Kingdom cope. These include the Coronavirus Job Retention Scheme, the Self Employment Income Support Scheme, and the Eat Out to Help Out Scheme. Once these initiatives were launched, His Majesty’s Revenue and Customs (HMRC) needed to deliver a whole new system, capable of dealing with huge spikes in traffic. And it had to be designed, implemented, and delivered in a matter of weeks. Here’s how Equal Experts helped make it possible.

This case study will help you to understand:

How good digital services can be launched to customers within days.

Why a digital platform should be treated as a product rather than a project.

The importance of culture in a digital platform ecosystem.

01

HOW EQUAL EXPERTS SUPPORTED HMRC TEAMS TO WORK AT PACE AND SCALE.

Working quickly - creating a team to deliver in just four weeks.

As the Chancellor of the Exchequer was rethinking the economic structure of the country, HMRC was equally busy gathering together several blended teams of data engineers to support a digital transformation. 

A mix of partners – including Equal Experts – enabled HMRC to jointly develop strong cloud technology architecture and improve the diversity of the organisation. They comprised experts technically capable of delivering a practical, straightforward system. A system that needed to withstand demand at levels never experienced before by HMRC.

  • 67,000

    number job claims within half an hour of the Job Retention Scheme going live

  • 440,000

    people applied for government grants via the Self Employment Income Scheme on the first day of its operation

The scale of the work

HMRC is an established government department, with a multitude of suppliers, hundreds of teams and thousands of engineers. 

A huge amount of work was required to build and maintain the platform and services needed to deliver financial support to more than 12 million employed and self-employed workers via the Coronavirus Job Retention Scheme, the Self-Employment Income Support Scheme and the Eat Out to Help Out Scheme.

Due to the timescales, Equal Experts worked with HMRC to enable the team structure to work differently on the Multi Tax Digital Platform (MDTP). Specialists from each discipline needed to be represented at team level. A platform engineer was embedded into each digital service team to safely “short-circuit” existing processes. This allowed us to eliminate risks early in the process and maximise on collaboration. 

This was adopted as a short term measure and within weeks of the first launch, we adapted digital platform tools and processes to make some of those “short-circuits” unnecessary.

Small technical teams with blended roles.

Cross-functional Digital Teams

 

THE MDTP – HOW A SINGLE CLOUD PLATFORM PROVIDES ROOM TO BUILD AND GROW.

02

THE MDTP - HOW A SINGLE CLOUD PLATFORM PROVIDES ROOM TO BUILD AND GROW.

Standing on the Shoulders of Giants.

The UK government’s rapid digital response to coronavirus (COVID-19) was a result of years of investment in people, governance and technology. The Multi-channel Digital Tax Platform is the cloud platform, hosted on Amazon Web Services (AWS), that was home to all the services provided. Originally evolving out of other services and deployed through shared lessons, this provided a “paved road” that allowed our digital service teams to write code to solve these HMRC problems on day one. 

Discover more about cloud based platforms in our Digital Platform playbook.

MDTP is home to:

  • 244

    Customer-facing applications

  • 850

    Production microservices

The cloud platform enabled the HMRC campaign for coronavirus. It held up to record traffic – peaking at 132 million page views in a single week – cementing the importance of having a PaaS in a single public cloud.

Understanding our work on the platform.

The MDTP is a cloud platform for over 244 customer-facing applications and 850 production microservices that have been built to support HMRC’s digital tax strategy. As one of HMRC’s partners working on the MDTP, we’ve helped provide an easy way for teams to build and deploy an application that can scale to handle millions  of requests. By building a Platform As A Service (PaaS), or platform-based service cloud computing service, the blended team has enabled HMRC to develop, run, and manage applications without the complexity of building and maintaining the infrastructure.

Why do we recommend you host in a single public cloud?

A Digital Platform is a type of bespoke PaaS: a set of streamlined tools, processes, and people which form platform capabilities to help you and your organisation rapidly meet the needs of your customers.

There are different types of bespoke PaaS, based on workload. The most common types we’ve seen before are Digital and Data Platforms. We advise your organisation to invest in a bespoke PaaS when a clear, homogenous workload starts to emerge across multiple teams.

Equal Experts strongly recommend you host a bespoke PaaS in a single public cloud. Amazon Web Services (AWS), Azure, or Google Cloud Platform are the main providers. They each offer a tried and tested, on-demand, cloud computing platform, with a wide range of reliable cloud services that can be provisioned instantly and billed on a pay-per-use basis. 

Discover more about designing Digital Platforms.

03

THE IMPORTANCE OF CULTURE IN PROVIDING FOCUS IN A PROJECT.

The HMRC response would not have been possible without the huge effort of the people who made it happen.

The response relied on digital capability and expertise to make policy into reality quickly – often going from concept to live in just a few weeks.

  • 4 weeks

    to launch all services

  • 97%

    customer satisfaction level

Collaborative ways of working were essential. 

It is difficult to comprehend how impressive the delivery of these services is. Equal Experts consultants were focused on delivery, working alongside a pool of very talented HMRC people. This was a collaborative environment where learning from each other to solve real problems meant that anything was possible. 

The culture of Equal Experts.

At Equal Experts, we trade on our ability to learn and share knowledge, rather than protecting or ‘guarding’ it. At its core, Equal Experts is a haven where this sharing happens freely and happily, between like-minded practitioners and our customers.

The People first approach.

The established culture and collaboration between digital service and digital platform teams was a foundation to the success of this project. Digital platform teams were already used to bi-directional feedback loops with digital service teams. This meant everyone was able to run these at high-speed, leveraging existing relationships and collaboration tools.

I am in awe of the problem-crunching fury with which HMRC and the Treasury created the furlough scheme and all the other means of support.

From Prime Minister Boris Johnson’s New Deal for Britain speech, June 2020
04

HOW TO EFFECTIVELY COLLABORATE AT SCALE.

Collaboration was the foundation to the success of this project.

Another shared obstacle was the increase in remote working. In a matter of days, HMRC had gone from being a mainly office based workforce to having 55,000 people working from home. The most important aspect of delivering a system at speed is the ability for engineers to be able to just ‘get on with it’. The work we’d already done with HMRC supported governance, best practice and a mandate for teams to use certain tools; this meant that any shift made to accommodate this radical change did not risk causing new technology problems or security concerns that might otherwise have delayed delivery.

The benefits of automating the setup.

  • Makes an engineer’s life easy by taking away pains of setup
  • Eases the burden of costs for companies
  • The setup itself will become an onboarding document and will be validated with every (new) team member trying to set up their machine.
  • Consistent setup across the team and thus helps in the effective pairing
  • Get rid of the “works on my machine” phenomenon
  • Helps in keeping the tools upgraded

Why we used Slack for communication.

Equal Experts already used Slack for effective communication with client teams. It is very easy to have instant conversations but also keep track of what happens elsewhere. A Slack channel could be created ad-hoc for a specific issue. Engineers, analysts and designers from multiple teams and specialisations can focus on collaborating and solving problems without endless meetings or endless email chains.

  • Questions for a specific team – the team uses public team channel as the first option.
  • Private Slack channels – for the day to day internal discussions that don’t need to be aired in the public.
  • A consistent naming convention, with good channel descriptions.

How to deliver at speed – Self-Service + Infrastructure-as-code.

The other really important aspect of working at speed is that as much as possible can be achieved using self-service. Logs, performance monitoring, automated alerting, a build pipeline that would automatically build and deploy to the non-production environments. It would even configure a Pull Request Builder that would run the tests before committing to the main branch.

An engineer’s point of view.

A really important aspect of working at speed is that as much as possible can be achieved using self-service. Here is a view from one of the engineers on the project:

“Within next to no time, I had all the supporting components in place. Logs, performance monitoring, automated alerting, a build pipeline that would automatically build and deploy to the non-production environments as soon as I committed to the main branch. It would even configure a Pull Request Builder that would run the tests before committing to the main branch.

All this was done in a matter of minutes, giving me time to focus on writing the service itself.”

You Build It, You Run It.

HMRC wanted to give the MDTP team all the capabilities they needed to develop, test, deploy, and operate their own live services. We supported HMRC to adopt a “You Build It, You Run It” approach, something we at Equal Experts use regularly to ‘upskill’ and empower our clients’ teams. This has been a key enabler to supporting so many teams on the MDTP. A Platform-as-a-Service offering maximum operability with minimal friction is easier for teams to operate safely and reliably.

The production support method of ‘You Build It, You Run It’ refers to developers supporting their own production services. You Build It You Run It unlocks weekly or more frequent deployments for Digital Service teams, as it eliminates any operational handoffs. It also incentivises Digital Service teams to balance their time between product and operational features. The net result is more frequent production deployments and more reliable live services.

Find out more about You Build It You Run It

05

INTERACTING WITH LEGACY SYSTEMS - UNDERSTANDING WHEN IT’S POSSIBLE - AND WHEN IT’S NOT.

Interacting with legacy systems.

Unless the project is built from scratch, there will always be a requirement to work around legacy systems. In this case, each digital service required interactions with legacy systems across HMRC. With HMRC’s technical experts, we analysed each system to understand whether it was ‘fit for purpose’. Where it was possible to work with the system, we supported service teams to leverage existing, well-defined interfaces to legacy systems. Where there was a potential risk of destabilising the whole process, the platform team quickly and iteratively built well-defined interfaces to new software systems. 

Organisations, over the years, build up tens or even hundreds of legacy IT systems (either off the shelf or bespoke). Typically, these systems are stand-alone or have limited ability to integrate with a wider enterprise. A digital platform can allow these disparate legacy systems to evolve into modern-looking, more flexible systems.

06

WHY CONTINUOUS DELIVERY IS AN EFFECTIVE, SECURE SOLUTION FOR CHANGE.

How technology initiatives, if done in the right way, can be successful and fun.

A digital platform is never “done” – and what has come before is not a predictor of what is to come. Although working to short timescales, this value still stood. Throughout the HMRC project, by adopting a continuous delivery approach, we were able to increase feedback cycles, de-risk production deployments, and ensure production systems could be operated in a reliable functioning condition. This enabled the performance and resilience required so that HMRC could deliver on the Government’s promise to the huge population of claimants accessing the Covid-19 services.

How does Continuous Delivery work?

  • We start projects by building a “walking skeleton,” automating the path to production.
  • This is an end-to-end slice of working functionality.
  • It proves the architecture, path to production, and a foundation to build upon, incrementally.
  • We then automate the process of building and testing software in a repeatable and consistent way, from development, through test and into production.
  • We incrementally adopt continuous delivery and build a deployment pipeline to give us confidence the same software is deployed and tested through each state.
  • This confidence allows us to release early and often, with small enhancements directly responding to customer feedback.

Want to know more?

Are you interested in this project? Or do you have one just like it? Get in touch. We'd love to tell you more about it.