Skip to content
The methodOpen data
Back to the collection

Design Patterns for Securing LLM Agents against Prompt Injections

Luca Beurer-Kellner, Beat Buesser, Ana-Maria Creţu

Show all 14 authors

Edoardo Debenedetti, Daniel Dobos, Daniel Fabian, Marc Fischer, David Froelicher, Kathrin Grosse, Daniel Naeff, Ezinwanne Ozoani, Andrew Paverd, Florian Tramèr, Václav Volhejn

2025 · arXiv

Security & cryptographyLanguage models
Read the paper
HN score
2.5
Mentions
4
Readers
4
Months
4
What’s in a score?

From the abstract

As AI agents powered by Large Language Models (LLMs) become increasingly versatile and capable of addressing a broad spectrum of tasks, ensuring their security has become a critical challenge. Among the most pressing threats are prompt injection attacks, which exploit the agent's resilience on natural language inputs -- an especially dangerous threat when agents are granted tool access or handle sensitive information. In this work, we propose a set of principled design patterns for building AI agents with provable resistance to prompt injection. We systematically analyze these patterns, discuss their trade-offs in terms of utility and security, and illustrate their real-world applicability through a series of case studies.

Metadata: arXiv. Abstracts retain their original authors’ rights.

In the conversation

Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels

4 selected excerpts · 4 total mentions

simonw
Strong recommendation · rule
There's a crucial difference here. When you were working as a pentester, how often did you find a security hole and report it and the response was "it is impossible for us to fix that hole"? If you find an XSS or a SQL injection, that means someone made a mistake and the mistake can be fixed. That's not the case for prompt injections. My favorite paper on prompt injection remedies is this one: https://arxiv.org/abs/2506.08837 Two quotes from that paper: > once an LLM agent has ingested untrusted input, it must be constrained so that it is impossible for that input to trigger any consequential actions—that is, actions with negative side effects on the system or its…
Read the full comment
environment. The paper also mentions how detection systems "cannot guarantee prevention of all attacks": > Input/output detection systems and filters aim to identify potential attacks (ProtectAI.com, 2024) by analyzing prompts and responses. These approaches often rely on heuristic, AI-based mechanisms — including other LLMs — to detect prompt injection attempts or their effects. In practice, they raise the bar for attackers, who must now deceive both the agent’s primary LLM and the detection system. However, these defenses remain fundamentally heuristic and cannot guarantee prevention of all attacks.
Original comment Supabase MCP can leak your entire SQL database
benlivengood
Recommendation · rule
I believe you've covered some working solutions in your presentation. They limit LLMs to providing information/summaries and taking tightly curated actions. There are currently no fully general solutions to data exfiltration, so things like local agents or computer use/interaction will require new solutions. Others are also researching in this direction; https://security.googleblog.com/2025/06/mitigating-prompt-in... and https://arxiv.org/html/2506.08837v2 for example. CaMeL was a great paper, but complex. My personal perspective is that the best we can do is build secure frameworks that LLMs can operate within, carefully controlling their inputs and interactions with…
Read the full comment
untrusted third party components. There will not be inherent LLM safety precautions until we are well into superintelligence, and even those may not be applicable across agents with different levels of superintelligence. Deception/prompt injection as offense will always beat defense.
Original comment My Lethal Trifecta talk at the Bay Area AI Security Meetup
amonks
Neutral mention · rule
That’s a pretty scary answer, to be honest. Regardless, here’s the CaMeL paper. Defeating Prompt Injections by Design (2025): https://arxiv.org/abs/2503.18813 Here’s a paper offering a survey of different mitigation techniques, including CaMeL. Design Patterns for Securing LLM Agents against Prompt Injections (2025): https://arxiv.org/abs/2506.08837 And here’s a high-level overview of the state of prompt injection from 'simonw (who coined the term), which includes links to summaries of both papers above: https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/
Original comment Launch HN: Slashy (YC S25) – AI that connects to apps and does tasks

Keep following the thread.

More in Security & cryptography

More from the same topics.