September 17, 2026
By now most of us have used LLMs. Many of us reach for them daily, for an ever-widening set of tasks. In a recent survey the share of US adults using AI was put at 62%, with 45% using it for their jobs. But the organisational impact keeps coming in below expectations. In the most recent PWC Global CEO survey, 50% of CEOs said that AI had delivered no cost or revenue benefits. Most tech leaders I speak to recognise this. There is a real desire to use AI to make a difference to their business, but actually moving the needle is hard.
There are a variety of reasons why organisations are not able to capture these benefits. One explanation that rings true for me is that we are using AI to speed up processes that were designed for a human world and have been adapted over time to meet the quirks of the people running that process. That is very sensible when you have people doing the work, but inefficient in a world where AI can do many things. A faster version of the wrong process is still the wrong process. I believe that real returns will only arrive when AI is central to how the work is designed, not an assistant bolted onto the side of it.
That’s what I mean by an AI-native organisation: one where AI is a first-class entity, and the people and processes are built around it rather than the other way round. Today, if you want AI as a first-class entity, you are almost certainly thinking in terms of agents.
So what would you actually need in place? This post covers the technical foundation: the components and ways of working needed to build, deploy and run agents safely. There is another half of redesigning the processes themselves, along with the operating model around them. I’m not covering that here – that can wait for another post.
The idea of agents has a long history in AI and typically means a piece of software which has goals and acts autonomously to meet those goals. Agents have become incredibly potent recently because the current generation of AI, which is based on LLMs, can be given instructions in ordinary language. You can create an agent through supplying some instructions in English to an LLM as a simple text file and the LLM is able to understand your intent and act on it.
These are specific pieces of functionality the agent can call when it judges they’ll help. Web search is the most common one; you might equally supply a tool for reading PDFs or querying your order database. The key property is that a tool is deterministic code for a defined task. You control exactly what it does, and you can put the usual engineering rigour and testing around it. The model can then choose whether to call it, it looks relevant for its task. It’s worth noting that this in itself is an indeterminacy. The model may choose to call the tool or may not – even though it would be very relevant.
Reusable instruction sets an agent can pull in when relevant. Some typical skills are a security review skill for a coding agent, or a research skill for compiling a report from web sources. Skills are like parts of a prompt that capture specific procedural knowledge rather than a whole new capability. They are loaded on demand so they don’t consume context until needed. Usefully, the agent skills specification format has become a de facto cross-vendor standard, so a skill you write is not locked to one agent framework.
An LLM is a function: you put text in, it gives text out and in itself it holds no memory between calls. The conversation feels continuous but it is an illusion maintained by re-sending parts of the history of your conversation each time. However the input window is finite, and much of it is already spent on system instructions, your current request, tool definitions and tool results. So you need something to manage what goes into that window. Something to keep what is pertinent, summarise key bits and drop the rest. This is one of the key engineering problems in agent design.
Memory is related but worth treating separately. Context management is about the current window. Memory on the other hand is about what persists across sessions. Things like knowing that you work for ACME Lawyers, or that this customer has an open complaint which the agent learned last Tuesday. They have different storage, retrieval and governance questions (not least: what happens when the memory is wrong, and who can see it).
These components – instructions, an LLM, tools, skills, context and memory – are the raw material of an agent. Bundled together inside a loop they are the raw materials of something that can act autonomously: it has a goal, acts, observes the result, decides what to do next, and repeats until it judges the goal has been met or hits a limit you have set.
You’ll want to interact with an agent: to ask it something; to see what it produced, and sometimes the agent will want to interact with you. It will need to seek clarification or want guidance on which of the next steps it should perform.. Because agents stand in for people or functions, chat interfaces like Messenger systems, WhatsApp and the like feel natural, and that’s where most teams start.
But chat isn’t the whole point, and some of the most valuable agents have no conversational surface at all. Against which do backend tasks run in a pipeline, on a queue, or inside a CI job. These agents will need to use the existing systems for communications between processes such as message queues like Kafka.
Agents should run with some autonomy, which today means on a schedule, on a human request, or in response to an event from another system. The event-driven case is under-used right now and may be the one with the most potential benefits for your business. An agent that is waiting for and reacts to something happening in your business is a more powerful automation than one that waits to be asked.
Building an agent is currently almost too easy, and there are more ways to do it than you’d like:
The right choice depends on your existing technical estate, the use case, and your data governance constraints more than on any feature comparison. Most of the agent frameworks manage LLM access, context, and triggering. It’s all part of the package.
You presumably want the agent to do something useful for your business, which means access to your data. If your structured data is well managed in a decent warehouse like Databricks, Snowflake you’re already ahead. You’ll also want unstructured content such as documents, PDFs, wikis etc. reachable via vector search or direct access to the content systems (perhaps via a tool).
Your agent will work better if the meaning (semantics) of your data is clear. An agent asked about “active customers this quarter” needs to know which of your four definitions of “active” applies, which of the three revenue columns is the one people mean, and which tables are deprecated. Humans navigate this in a variety of ways – often using tribal knowledge or being able to talk to the owners of the data, but an agent can’t do that. Data modelling activities like creating a semantic layer, or a canonical model alongside keeping your metadata up to data will make a big difference on how well your agent performs.
An agent takes actions. But is it allowed to? And under whose authority should it take them? Practically, you need: an identity for each agent, distinct from the humans who invoke it; permissions scoped to the minimum the agent needs; a way to act on behalf of a user that inherits their entitlements rather than exceeding them; and an audit trail recording what was done, by which agent, under whose authority, and with which data. You will need to find a path between locking down the permissions so that the agent is unable to do anything useful and providing it with service accounts with excessive access. Row and column-level controls in your data platform have to survive the trip through the MCP server or API.
This is an emerging area. Standards for agent identity are not yet established. But if your agent is going to do anything consequential you need to be clear about authorisation and ultimate responsibility.
The interface to that data matters as much as the data itself. MCP (Model Context Protocol) has become the default for providing data to AI centred solutions. MCP is kind of like an API for these. An MCP interface gives a description of the data alongside methods for accessing it. It is still evolving and recent revisions have pushed it decisively toward enterprise use. The protocol is now stateless, so servers can sit behind ordinary load balancers; requests can pause mid-execution to ask a user for confirmation; and the authorisation model has been hardened. MCP gateways are now becoming available which can help with governance for enterprises.
You don’t have to use MCP. Instead, you can use tools to obtain direct access to your data through an API or other connection or you may choose to implement and expose a semantic layer in your data platform. Whichever you choose, you’ll need the access controls implemented correctly.
How do you know that your agent is doing what you expected? You could just write some instructions and hope for the best, but if your agent is doing anything consequential you should evaluate its performance before trusting what it does. Usually creating an agent involves iterating around a set of instructions, tools etc. until you get the results you want consistently. Doing these “evals” involves creating test cases that you can use to measure the performance against expected outputs. Typically you create a set of sample instructions to the agent along with expected responses. An eval framework then compares these to validate the agent is giving the right responses.
Evals tell you how an agent behaves before you deploy it or after you change it but you also need protection while it’s running. Output guardrails check that responses don’t contain what they shouldn’t. But input is also an agent-specific risk. An agent that reads untrusted content (a web page, a customer email, a supplier PDF) can have instructions smuggled into it, and the model has no reliable way to distinguish your instructions from the ones it just read. Combine that with access to sensitive data and the ability to send information outward, and you have a genuine exfiltration path. The same caution applies to third-party MCP servers: an MCP server is code you’re granting access to your systems, and it deserves the same scrutiny as any other dependency.
None of this is a reason not to build and there are many tools like ArizePhoenix, DeepEval, PromptFoo, AWS Bedrock guardrails etc. which help with different aspects of evals and Runtime Safety, but you should be deliberate about which agents get which combination of untrusted input, sensitive data and outbound reach.
You will need to see what your agents are actually doing in production: the reasoning steps, which tools were called with what arguments, where runs failed, how long they took and what they cost. As well as being useful in diagnosing bad actions, these production traces are a great source for improving your eval cases.
Cost is worth confronting early, because agent economics is important and can get costly. Agent token usage is not just a matter of the input and output strings. Agents may run internal loops, try things out, do retries etc. that you are not aware of. Make sure you can capture and monitor these costs. It’s best to have per-agent cost attribution before you have thirty agents in place.
Sometimes you may want multiple agents working together. For example, for customer support you could have one agent that understands an incoming customer request which then passes on to other agents such as for order update, returns handling etc. to address the specific issue triaged by the initial agent. If you do want to take a multi-agent approach you will need a means of co-ordinating these agents. A2A is the leading protocol for this right now.
A word on multi-agent architectures. Multiple agents working together is an exciting picture but also the one most likely to get you into trouble. Coordinating two autonomous entities compounds the indeterminacy of each. It is much harder to identify the causes of failure when you have two entities behaving non deterministically. If the failures are intermittent, you can imagine the challenges trying to pinpoint the exact reason.
The useful discipline is to ask, every time: could this be a tool or a skill instead? A great many things reached for as “a second agent” are better expressed as a deterministic tool the first agent calls, or a skill it loads. You lose nothing and gain a system you can debug much more easily. I recommend going for multiple agents when the sub-problems genuinely need independent judgement and their own context.
Your agents should do what they are expected to and will need to be supported . But you should also enable experimentation of this new and exciting technology, so you need to create capabilities which support both these objectives whilst at the same time meeting your governance concerns.
We believe that like any software, agents need to be tested before they are made available for general consumption. You should employ some kind of AgentOps with CI/CD in which evals form part of the promotion process.
As mentioned above, it is probably easy for anyone in your organisation to build an agent, and to do it in a variety of ways. The risk is you end up with a zoo of partially completed agents, all built differently but which will be providing some business benefits to some people. To avoid this support and governance nightmare, you will almost certainly need to create recommendations on the approved ways to create agents in your organisation. In other areas of software having a golden-path/ paved road is an effective way to accelerate deployment of software, make support easy, and at the same time ensure governance concerns are met. However, as this is an area of rapid change, it is likely too soon to solidify on a single path and you may have to settle on recommendations rather than a more comprehensive factory approach.
In addition to the paved road, insist on one piece of metadata: every agent has a named owner. The history of IT is littered with examples of cool pieces of software which then became orphaned. Dashboards whose purpose nobody remembers, Web-apps which nobody feels able to switch off. This is happening now with agents. An agent with no owner should be a deployment blocker and a candidate for switching off.
There is a lot of excitement and potential around agents, so you should provide a means in which it is easy to experiment with agents to try out ideas. Otherwise teams will resort to shadow AI which will likely be outside of governance. So try to provide some kind of playground or sandpit in which people and teams can implement agents safely at a small scale with access to data.
Many aspects of this architecture can be reused for different needs. Skills for example, once developed, will be useful to a wide range of teams. And if you are taking a multi-agent approach then Agents can be reused for multiple purposes. So you will want registries where people can find and access skills, tools, agents etc. These already exist in the hyperscalers (GCP skill registry, AWS Agent Registry, Microsoft Foundry) and there are also standalone ones.
The above may look daunting, but you do not need all of this to begin with. If you are starting with nothing in place:
First, build one agent that matters. Find a real use case with a named owner. Provide data access through one interface and include evals from day one. Eval discipline is the difference between prototypes and production agents at scale so start building that discipline. Even a dozen test cases is better than none, and writing them forces you to state what “good” actually means. Try to build with CI/CD or some kind of simple AgentOps. For a first agent, insist on Human approval on anything consequential.
Once you have two or three agents, make sure you have done Identity and permissions properly, before the shortcuts become entrenched. It’s a good time to implement production tracing and per-agent cost attribution. Now may be a good time to enable a sandbox with real (or realistic) data, so that experimentation can happen in the open rather than as shadow AI.
Then start building the paved road and start your registries for skills, tools and agents. A word of caution on this step. This field is moving fast enough that solidifying too early costs more than it saves. Recommendations and defaults are a faster, more flexible way to address this than a factory right now.
And a closing thought on where I started. Everything above is the technical foundation. If you build all of it and point it at the processes you already have, you will get faster versions of those processes and roughly the disappointing returns everyone else is reporting. The harder question which processes should stop existing in their current form because an agent changes what’s possible is the one that actually separates AI-native from AI-assisted. That’s another post.
Because most organisations use AI to speed up processes that were designed for humans, rather than redesigning the process around what AI makes possible. A faster version of the wrong process is still the wrong process.
An AI-native organisation treats AI as a first-class part of how work gets done, building people and processes around agents rather than bolting AI onto workflows designed for humans.
An agent is software with a goal that uses an LLM to interpret instructions and act autonomously. A tool is deterministic code for a defined task, like a web search or database query, that the agent can choose to call.
Skills are reusable instruction sets an agent pulls in when relevant, such as a security review skill or a research skill. They capture procedural knowledge and load on demand, so they don’t consume context until needed.
Context is what an agent holds within its current conversation window. Memory is what persists across sessions, like knowing which organisation a user works for. The two raise different storage, retrieval and governance questions.
No. MCP (Model Context Protocol) is a common way to connect agents to data, but direct API access or a semantic layer in your data platform work too. What matters is that access controls are implemented correctly, whichever route you choose.
Evals are test cases with expected outputs used to measure whether an agent performs consistently. Without them, there’s no reliable way to know an agent is producing the right results before it’s trusted with real tasks.
Coordinating multiple autonomous agents compounds the unpredictability of each one, making failures harder to diagnose, especially when they’re intermittent. Many tasks people reach for a second agent to solve can be handled more reliably as a tool or skill instead.
Start with one agent solving a real problem, with a named owner, one data interface, and evals from day one. Add identity and permissions controls, cost tracking, and shared registries once you have a few agents running, not before.
Agent costs go beyond input and output text — agents can run internal loops, retries and exploratory steps you won’t see by default. Track and attribute cost per agent early, before you’re running dozens without visibility into what each one spends.
All business models have their pros and cons. But, when you consider the type of problems we help our clients to solve at Equal Experts, it’s worth thinking about the level of experience and the best consultancy approach to solve them.
If you’d like to find out more about working with us – get in touch. We’d love to hear from you.
