LLM and agentic AI threat model: from prompt injection to data exfiltration
The realistic attack paths against LLM and agent-based systems, and the controls that actually reduce each one.
The structural problem
Language models do not reliably separate instructions from data. When a system feeds a model content it does not control — a web page, an inbound email, a supplier document, a ticket comment — that content can influence the model as though it were an instruction.
This is why the risk grows sharply once the model can act. An assistant that summarises a malicious document produces a poor summary. An agent that reads the same document and holds credentials can be steered into doing something on the attacker's behalf.
Direct and indirect prompt injection
Direct injection is a user attempting to override system instructions. Indirect injection hides instructions inside content the system ingests later, which is far more dangerous because no human reads it first.
Reduce it by treating all retrieved and ingested content as untrusted, isolating it from the instruction channel, stripping or neutralising active content, and never letting ingested content alone authorise a privileged action.
Data exfiltration paths
Common paths include retrieval over an index whose permissions are broader than the user's, an agent that can send email or post to external services, rendered output that fetches an attacker-controlled URL carrying data in the query string, and verbose logging that stores sensitive content in a lower-trust system.
Controls: enforce per-user permission filtering at retrieval time, restrict outbound network destinations, disable automatic fetching of remote resources in rendered output, and classify and protect AI logs at the sensitivity of the data they contain.
Excessive agency and tool abuse
Every tool granted to an agent is an attack surface. Grant the minimum tool set, prefer read-only where possible, require explicit human confirmation for irreversible or externally visible actions, and rate-limit tool invocation.
Add a kill switch that revokes the agent's credentials immediately, and rehearse using it.
Want this assessed against your environment?
Send us the specifics and a senior advisor will respond within one business day.
Supply chain and model risk
Consider the provenance of models, plugins, extensions and prompt libraries, and pin versions where the platform allows it. A silent model upgrade can change behaviour that your evaluations previously validated.
Re-run your evaluation set after any model or prompt change, and treat unexplained behaviour change as an incident until proven otherwise.
Detection and response
Monitor for anomalous tool usage, unusual retrieval volume, outbound destinations not on the allow list, and spikes in refusals or unusual output patterns. Include AI systems in your incident response plan with named owners and containment steps.
Assume some injection attempts will succeed. Containment design — narrow permissions, reversible actions, strong logging — is what turns a successful injection into a non-event.
Sources and further reading
- OWASP Top 10 for Large Language Model Applications
- NIST AI Risk Management Framework
- ISO/IEC 27001 information security management systems
- NIST SP 800-53 security and privacy controls
This article provides general information and decision support. It is not legal advice, audit assurance, certification advice or a guarantee of outcome.
Related reading
Securing enterprise AI adoption: a practical AI security control set
The AI security controls that matter first — identity, data boundaries, model access, logging, human oversight and supplier assurance.
Read articleAI risk assessment: how to assess an AI system before it ships
A repeatable AI risk assessment covering purpose, data, model behaviour, integration, human oversight, failure modes and evidence.
Read articleISO 42001 vs ISO 27001: how the two management systems interlock
What each standard covers, where they overlap, and how to run one integrated management system instead of two parallel programmes.
Read article