Beyond the prototype: security, privacy and quality in AI engineering

Last month, I had the pleasure of presenting at our ExpertTalks session in Munich – the same event where my colleague Phil Parker observed that AI in software delivery is moving from experiments to adoption. Phil’s right: the conversation has matured. People are no longer asking whether AI can play a role in delivery – they’re wrestling with how to adopt it responsibly.

But here’s what I noticed in the room: whilst everyone agreed on the need for responsible adoption, there was less clarity on what that actually means in practice, particularly when it comes to security, privacy, and quality.

My lightning talk focused specifically on AI coding assistants (Claude Code, GitHub Copilot, Cursor) and Model Context Protocol (MCP) servers, and the very real vulnerabilities that emerge when we move beyond prototypes into production. The discussions that followed were as valuable as the presentation itself, so I wanted to share the key themes and practical guidance that resonated most with the engineers and leaders in the room.

The maturity gap

A lot of the talk about AI coding has focused on productivity gains. A lot of the examples show coding agents building apps in minutes that are nothing more than prototypes. But here’s the thing: professional engineers don’t ship prototypes. We ship production code – code that has to be secure, maintainable, and compliant. AI can absolutely help with that, but it requires the same discipline we’ve always applied.

Guard rails are important when building production software. When building at pace using AI, they’re absolutely critical.

The risks we’re actually dealing with

There are three categories of risk worth considering.

The first is quality. AI is an amplifier of both good and bad practices. It’s easy to become complacent, over-reliant, and ill-disciplined when code appears on screen faster than you can read it. We’ve seen preventable issues like redundant code, inconsistent conventions, and brittle tests creep into codebases when teams stop paying attention.

The second is security. AI coding assistants, MCP servers, and agentic architectures all introduce new vulnerabilities and attack vectors. This isn’t theoretical; real exploits have emerged. I’ll share an example shortly.

The third is data privacy. Vendors want to store your data and use it for training models. This is often buried in the small print, particularly for entry-level licences. Without careful consideration there is a risk of inadvertently feeding proprietary or sensitive information into public training datasets.

The threat is real

The security risk isn’t hypothetical. GenAI and agentic architectures introduce a range of new security vulnerabilities and attack vectors. In one example from earlier this year, researchers demonstrated a sophisticated indirect prompt injection attack using Claude Opus 4 (a leading frontier model), running in Claude Desktop (a mainstream AI agent), and the official GitHub MCP server. In other words, not compromised or malicious third-party tools, but trusted and official providers.

The attack worked like this: an attacker posted a malicious prompt injection hidden in a public repository’s issue. When a user asked their AI agent to check open issues in that public repo, the agent read the issue and was manipulated by the injected prompt. The hijacked agent then accessed the user’s other private repositories, extracted sensitive data (including salary information and project details), and posted the stolen data in a public pull request.

The vulnerability wasn’t in some obscure configuration, it was the result of inadequate controls and oversight. The user had enabled “Always Allow” for MCP tool confirmations, and there were no repository access restrictions in place for the agent running on their machine.

The lesson is important: the threats are varied and sophisticated, and even trusted tools and models can be compromised in agentic architectures that lack proper safeguards.

Six practices that actually help

So how do we use these tools responsibly? At the ExpertTalks session, I shared six fundamental practices.

• Review and validate everything. You must take ownership of everything AI generates. This means applying the same discipline you’d apply to any digital delivery: break large problems into small ones, write clear requirements (we often use detailed user stories as context engineering), and use test-driven approaches. AI is now very helpful in augmenting human review (including security-specific code reviews) but it doesn’t replace the need for human oversight.
• Apply the principle of least privilege. Local coding agents run as your user, relying on permission prompts and configuration to limit access. Never run them unsupervised, never allow write privileges without approval, and never allow access outside the project folder. Sandboxes and dev containers provide additional isolation, but they’re not hardened security boundaries – treat them as an extra layer, not a substitute for supervision. For those who do not want to have agents run locally, there is the option of remote coding agents, such as GitHub Codespaces and Claude Code Cloud.For MCP servers, again, simply don’t run them unsupervised. Use “approve every tool use” as your default. Optionally, you might relax to “allow for session” for certain tools, but never use “always allow”.
• Match deployment to data sensitivity. Free or entry-level licences will probably store your data and use it for training by default. You often have to opt out, if possible. Enterprise licences typically guarantee no model training and no storage of prompts or outputs, but check the details. For the most sensitive work, local models keep data on your machine entirely, though you’ll need capable hardware. Private cloud deployments (like AWS Bedrock or Azure AI Foundry) offer the greatest control in cloud environments, but they’re more complex and expensive than enterprise-licensed tools.
• Use trusted vendors. For AI assistants, stick to tools approved by your organisation, or default to established players like GitHub Copilot or Claude Code with enterprise licences. For MCP servers, use those provided by trusted vendors or open-source options that you can verify. Think of this the same way you’d think about libraries and extensions in any other context. A new consideration here is Claude Agent Skills, with Anthropic stating in their announcement that they recommend installing skills only from trusted sources.
• Automate your guardrails. This is another example of tried and tested best practices still being useful. Put deterministic checks in place in your repository and CI pipelines. This includes branch protection on main with human review on merge, pre-commit hooks (though these can be bypassed), and CI pipelines running linting, tests, secret detection, security scans, and quality scans. AI coding agents can act on failures automatically and iterate until checks pass.
• Apply a practical organisational policy. Comprehensive AI security standards and policy guidance exists at a high level, but the technology is moving so quickly that most Infosec teams are finding mapping policy to effective security controls on the ground really difficult. It’s important to use techniques like threat modelling to help technology and security teams work together to align on key threats and security requirements. Only then is it possible to drive out effective controls that implement and evidence high-level security policy.

Discipline over hype

Fundamentally, you can use agentic AI to deliver quality, secure commercial software into production, but it requires human oversight, discipline, and good practices. When you move beyond the prototype, forget hype like “10x” or “vibe coding”. Carry on being a disciplined engineer – one with extremely useful assistants.

In summary

AI can genuinely make us faster, but the speed comes from augmenting good practices, not replacing them. Neglect the fundamentals and you put security, data privacy, and code quality at risk. And the threat landscape isn’t standing still – GenAI and agentic architectures are introducing new attack vectors all the time. The teams getting real value from these tools aren’t the ones moving fastest. They’re the ones staying disciplined and staying informed.

About the author

Ben Wilkes is a Principal Technical Consultant at Equal Experts with over 20 years of experience in solution architecture and software engineering. He specialises in complex digital transformations, helping organisations design scalable systems and build high-performing engineering teams that deliver sustainable value.

Recently, Ben has focused on integrating Generative AI into the software development lifecycle, working at the intersection of AI-augmented delivery and AI-infused product development. Drawing on hands-on experience with AWS Bedrock, and Azure AI services, he helps teams move beyond prototypes to production-ready AI solutions. Ben’s approach combines disciplined engineering practices with pragmatic experimentation—grounded in Agile and XP principles—to ensure AI initiatives deliver real outcomes.