May 13, 2025
With demand for digital services continuing to grow and AI offering opportunities to accelerate productivity, the public sector is facing increasing pressure to modernise and deliver more with less. Strengthening data foundations and building a data capability at scale will support faster, data-driven decision-making, streamline operations and, crucially, accelerate the adoption of AI.
However, improving an organisation’s data infrastructure alone won’t deliver faster decisions or create better outcomes for citizens. Some of the biggest barriers to unlocking the full potential of data aren’t technical. They’re human.
Culture, leadership and team structure play a defining role in whether data is seen as a strategic asset within an organisation or just a reporting tool. To unlock the real value of data public sector leaders must look beyond tools and rethink how data is embedded into everyday service delivery.
Earlier this year, the State of digital government review identified leadership as one of the five root causes of the issues public sector organisations face within digital and data, with these capabilities not seen as essential for policy formulation or operational delivery.
To be effective, modern data functions need to be understood as assets, capable of delivering wide-ranging benefits to a public sector organisation. Data needs to be represented and championed at the executive level to gain the buy-in and resources required for meaningful and ongoing change. Data leaders within government departments need to drive this conversation and reframe data as a driver of insight, productivity, and transformation to lay the groundwork for mature, organisation-wide data capability.
Organisations also have to move from isolated data departments to real-time organisation-wide data utilisation and rethink how data is gathered, shared, and used across the business, public sector teams
This may mean shifting from fragmented data silos to connected systems where data flows securely and consistently across departments to enable faster decision-making and joined-up service delivery. Providing access to clean, standardised data can help policy teams rapidly assess programme performance, or give frontline staff better insights into citizen needs in real time.
To achieve this, organisations need robust data pipelines that automate data ingestion, transformation, and sharing – a topic we’ll be exploring in more detail in a future blog in this series. In the meantime, our Data Pipelines Playbook explores the principles, best practices and pitfalls for designing and delivering these pipelines.
Many departments currently lack the team structures to make use of modern data platforms. Despite recent growth in the number of people in digital and data roles, the public sector still lags behind high-performing private organisations in the proportion of technical roles. The largest operational departments, including HMRC, the Home Office and the Department of Work and Pensions, average just 5% compared to 8-12% in regulated private sector industries.
This imbalance can often prevent true collaboration or innovation, with organisations failing to integrate digital and data colleagues into their operating model alongside policy and operational teams. Tackling it requires organisations to introduce cross-functional teams, bringing together a diverse set of roles and skills, such as data engineers, analysts, service designers and leaders, to enable end-to-end product delivery through shared goals and ownership
Embedding skilled data experts alongside people with delivery expertise and those with domain expertise leads to rapid practical impact. For HMRC’s Customer Insights Platform (CIP), a collaborative approach between programme, platform and product teams turned the CIP into a powerful, scalable platform that drives fraud prevention, customer insights, and smarter decision-making for UK government services.
With limited data expertise and resources in public sector organisations, increasing demands on their skills and knowledge can quickly lead to bottlenecks and slower decision-making. As data capabilities become more and more important, in particular with the acceleration of AI, empowering people and teams with data skills and repeatable, reliable processes is vital.
Organisations that promote self-service through better tools, training and “paved roads” can embed data use throughout teams and unblock data expertise bottlenecks. At John Lewis Partnership a “paved road” approach provided a repeatable and reusable process for building data ingestion pipelines, automating some of the more complicated tasks so that more teams could self-serve. This enabled more teams to build and operate their own data pipelines and allowed JLP data engineers to focus more on higher-value data engineering work. Within 18 months, the Partnership Data Platform was transformed, hosting 65 data products, and generating an estimated £22.1m cost saving.
Even with strong leadership and skilled teams, wider organisational issues can derail data initiatives. Slow approval processes, unclear governance and inflexible funding models often stifle innovation before it has a chance to flourish. Funding in particular is of concern for public sector leaders with only one in five respondents believing current models enabled the effective investment in and running of digital services.
Public sector departments will need to create and tailor processes, practices and funding models for wider use of data. This is especially of concern within governance, which can impact security, privacy, compliance, consistency and accessibility, amongst other things. Public sector organisations need to create permissive models with clear guardrails instead of restrictive controls to overcome this challenge. We explore this and other challenges in more detail, alongside best practices and guidance, in our Data Products playbook.
To realise the benefits of a modern data structure, public sector organisations must also address the cultural and leadership aspects of transformation. However, identifying issues within an organisation is not always simple.
A data health check provides an independent assessment of an organisation’s current data capabilities, challenges and opportunities. At Equal Experts, our tailored service also looks deeper at the human and organisational factors present and helps teams plan the next steps to greater data maturity, addressing both technical and cultural factors. We can provide unique support in:
Book a Data Health Check to assess your organisation’s current maturity and build the leadership, culture and capability needed to make data a true strategic asset in the public sector. Contact us today.
All business models have their pros and cons. But, when you consider the type of problems we help our clients to solve at Equal Experts, it’s worth thinking about the level of experience and the best consultancy approach to solve them.
If you’d like to find out more about working with us – get in touch. We’d love to hear from you.
