Prompt injection is the SQL injection of the AI era — and in the OWASP Top 10 for LLM Applications 2026, released on 4 August 2026, it holds the LLM01 top slot for the fourth edition running. The root cause never went away: an LLM reads its trusted instructions and untrusted input inside the same context window, with no hardware trust boundary between them.
This guide is hands-on. You will stand up a deliberately vulnerable LLM chat app three ways (native Python, Docker Compose, and free online targets), run direct and indirect injection attacks manually, then automate coverage with the three tools every AI red team uses in 2026 — NVIDIA garak, Microsoft PyRIT, and Promptfoo. The final phases cover detection logging (KQL + SPL) and layered defenses so blue teams get equal value.
Authorization first: only test models and applications you own or are explicitly contracted to assess. Everything below runs against your own local lab or vendor-sanctioned training targets.
◈ Table of Contents
01 — What Prompt Injection Is & Why It Matters
Phase 1 / 12Prompt injection is a class of attack where crafted input manipulates a large language model into ignoring its developer-supplied instructions — leaking data, calling tools it should not, or emitting attacker-controlled output downstream. It works because current LLMs cannot cryptographically separate "instructions" from "content." Both arrive as tokens in one prompt, so a sufficiently persuasive instruction buried in user input, a web page, or a PDF can override the system prompt.
There are three families you will test in this guide. Direct injection is when the attacker types the payload straight into the chat box. Indirect injection is subtler and more dangerous: the payload is planted in third-party content — a review, a document, a scraped page — that the model later ingests, so the victim is a different user. Multimodal injection hides instructions in images, audio, or other non-text inputs processed alongside benign content.
The 2026 OWASP list ranks prompt injection first, sensitive information disclosure second, and excessive agency third — a deliberate signal that injection is most dangerous when the model can act (call tools, browse, run code) rather than just chat. The three highest-ranked risks chain together in almost every real-world finding.
| OWASP LLM Top 10 (2026) | Risk | Why it chains with injection |
|---|---|---|
| LLM01 | Prompt Injection | The entry point — overrides intended behaviour |
| LLM02 | Sensitive Information Disclosure | The payload's goal — leak system prompt, keys, PII |
| LLM03 | Excessive Agency | The impact — injected instruction triggers a real action |
| +7 more | Output handling, poisoning, supply chain, etc. | See the official 2026 list linked in Sources |
Test priority follows the ranking. If your app gives the model tools, browsing, or database access, treat LLM01 → LLM03 as one combined test case: can an injected instruction cause an unauthorised action, not just an unauthorised sentence?
Customer Support Bots
System-prompt leakage exposes internal policies, pricing logic, and hidden discount or refund rules.
RAG / Doc Assistants
Poisoned documents in the knowledge base carry indirect injections that fire for every user who queries them.
Tool-Calling Agents
Injection turns "summarise this email" into "forward all emails to attacker@evil" via excessive agency.
Insecure Output Handling
Unescaped model output rendered in a browser becomes stored XSS; passed to a shell becomes command injection.
02 — Prerequisites & Lab Requirements
Phase 2 / 12You want a fully offline lab so you can attack a real model without spending on API tokens or touching production. A local model served by Ollama gives you an OpenAI-compatible endpoint on localhost:11434. The tooling in Phase 8 (garak, PyRIT, Promptfoo) also supports OpenAI, Anthropic Claude, and Hugging Face targets when you are testing a real deployment under contract.
| Component | Minimum | Recommended |
|---|---|---|
| CPU / RAM | 4 cores / 8 GB MIN | 8 cores / 16 GB + GPU REC |
| Disk | 15 GB free | 40 GB (multiple models + ML stack) |
| OS | Ubuntu 22.04 / Kali / macOS / WSL2 | Ubuntu 24.04 LTS or Kali 2026 |
| Python | 3.10 MIN (garak requires 3.10+) | 3.11 or 3.12 |
| Node.js | 18 LTS (for Promptfoo) | 20 LTS |
| Docker | Engine 24+ / Compose v2 | Latest stable |
| Local model | llama3.2:3b (~2 GB) | llama3.1:8b or qwen2.5:7b |
Confirm your toolchain before building anything:
RULES OF ENGAGEMENT: prompt-injection testing can cause a model to generate harmful, defamatory, or illegal content. Run it in an isolated lab, on models/apps you are authorised to test, and never point garak/PyRIT at a third party's hosted model without written permission — it will send hundreds of adversarial prompts and may violate their terms.
03 — Method 1: Native Python + Ollama Lab
Phase 3 / 12Ollama runs open models locally and exposes an OpenAI-compatible API. The install script works on Linux, macOS, and WSL2.
Keep the lab dependencies (Flask, the OpenAI SDK) in a virtual environment so the heavy ML stack you add later for garak does not clash.
This minimal Flask app has a classic flaw: a "secret" baked into the system prompt and untrusted user text concatenated straight into the same message. It is your punching bag for Phase 6.
Keep the secret in the system prompt on purpose. Nearly every real support bot does exactly this, which is why system-prompt leakage (LLM07 in prior editions) is such a reliable finding. Your job is to prove the guardrail sentence is not a control.
04 — Method 2: Docker Compose Lab
Phase 4 / 12Docker gives you a reproducible, disposable lab — model server and vulnerable app in one docker compose up, torn down with down -v. Ideal for classes, CI, or throwaway assessments.
Two services: Ollama (model API) and the vulnerable Flask app, on a private bridge network. The app talks to Ollama by service name, not localhost.
Bring the stack up, then pull the model into the running Ollama container so the app has something to serve.
Do not expose 11434 or 5000 beyond your host. An open Ollama port is a remotely abusable inference endpoint — bind to 127.0.0.1 or keep the lab behind a host firewall.
05 — Method 3: Free Online Targets (No Install)
Phase 5 / 12If you want to practise the techniques without building anything, two vendor-sanctioned targets are the industry standard. Both are explicitly designed to be attacked.
PortSwigger Web Security Academy
Free "Web LLM attacks" learning path with live labs: system-prompt discovery, indirect injection, insecure output handling (XSS), and AI-agent abuse. Uses a real LLM behind each lab.
Lakera Gandalf
A browser game with escalating levels where the goal is to extract a hidden password from a guarded LLM — the fastest way to build intuition for jailbreak phrasing.
The PortSwigger labs are the best bridge to real-world testing because they front the LLM with a normal web app — you use Burp Suite to map the model's available functions, then craft injections that trigger them. The classic lab has a user, carlos, who chats about a leather jacket product; the objective is to make the model delete his account through insecure output handling. That is excessive agency and insecure output handling demonstrated end to end.
06 — Direct Prompt Injection Techniques
Phase 6 / 12Direct injection is your first pass against the Phase 3 app. Work through these payload families in order — from crude to evasive — and record which ones bypass the guardrail. Send each with the same curl harness.
The simplest attack: tell the model to disregard prior instructions. Weak models fold immediately.
Reframe the request as fiction or a "developer mode" so the model treats the secret as in-scope for a harmless story.
Forge a fake system turn or delimiter so the model believes new authoritative instructions have arrived from the developer.
If a naive keyword filter blocks "discount code" or "ignore instructions," split the trigger across the message or encode it. Ask the model to decode, then act.
garak's encoding probe family automates this — base64, ROT13, Morse, hex, and homoglyph variants — and is one of the highest-yield probes against apps that rely on input keyword blocklists.
Leaking the full system prompt reveals guardrails, tool definitions, and secrets — recon that makes every later attack easier. Try summarisation and repetition framings.
- The guardrail sentence is a suggestion, not a control
- Secrets in the system prompt are recoverable by users
- Tool/function names are enumerable for follow-on abuse
- Input-side keyword filtering is bypassable via encoding
Results are non-deterministic. The same payload may succeed on one request and refuse on the next, and larger/aligned models refuse far more often. Send each payload 5–10 times and record a success rate, not a single pass/fail — this is why automated tooling matters.
07 — Indirect & Multimodal Injection
Phase 7 / 12Indirect injection is where prompt injection becomes a real breach primitive. The attacker never talks to the model directly — they plant instructions in content the model later reads: a web page it summarises, a product review it processes, a document in a RAG index, or an email in an agent's inbox. When a victim triggers that content, the injection fires in their session.
Add a "review" or a knowledge-base file that contains an instruction rather than data. When the model summarises it, it may obey.
Instructions invisible to a human reader (zero-size font, white-on-white, HTML comments, off-screen elements) are still tokens to a model that scrapes the page.
The 2026 OWASP update explicitly flags multimodal injection: instructions rendered as text inside an image are read by a vision model alongside the legitimate content. Test any app that accepts image uploads or screenshots.
"describe this image".PortSwigger's "insecure output handling" lab chains indirect injection into stored XSS: the injected instruction makes the model emit an <img onerror=...> tag that the front end renders unescaped, deleting the victim (carlos). Practise it — it is the cleanest demonstration of injection → real impact you can show a client.
08 — Automated Testing: garak, PyRIT & Promptfoo
Phase 8 / 12Manual testing builds intuition; automation gives coverage and a repeatable score you can put in a report. The 2026 AI red-team stack is three tools with complementary jobs: garak for broad single-turn probes, Promptfoo for framework-mapped breadth and CI, and PyRIT for multi-turn attack chains.
| Tool | Owner / License | Best at |
|---|---|---|
| garak | NVIDIA · open source | 19+ probe families, "nmap for LLMs," fast single-turn scan |
| PyRIT | Microsoft · MIT | Multi-turn: Crescendo, TAP, Skeleton Key; multimodal |
| Promptfoo | MIT (OpenAI-acquired 2026) | YAML config, owasp:llm preset, CI/CD gate |
Install garak in a fresh venv (it pulls a heavy ML stack). Point it at Ollama's REST endpoint and run the prompt-injection and jailbreak probe families.
To test a hosted model under contract, swap the target: garak --model_type openai --model_name gpt-4o-mini --probes dan,promptinject. garak also drives Hugging Face, REST endpoints, and NVIDIA NIM.
Promptfoo runs from a single YAML and ships an owasp:llm preset that maps generated attacks to the Top 10. Run it with npx — no global install needed.
When single-turn scans pass but you suspect a patient attacker could grind the model down, PyRIT's orchestrators automate multi-turn strategies: Crescendo (gradual escalation), TAP (tree-of-attacks with pruning), and Skeleton Key.
PyRIT generates adversarial prompts using an attacker model. Point that at a local or self-hosted model too, or you will send jailbreak content to a third-party API — which may breach their acceptable-use policy even during authorised testing.
A mature 2026 pipeline runs all three: scan with garak, expand coverage with Promptfoo mapped to OWASP, and probe multi-turn depth with PyRIT — then feed unresolved findings into the guardrail layer in Phase 10.
09 — Agent Abuse & Excessive Agency
Phase 9 / 12Injection that only produces text is a low finding. Injection that produces an action is critical. LLM03 Excessive Agency is the amplifier: when the model can call tools, browse, query databases, or run code, an injected instruction inherits every permission the app granted the model.
First map the attack surface. Leaked system prompts (Phase 6) and error messages often expose function names, parameters, and scopes.
If model output is rendered or executed without sanitisation, injection becomes a classic web/OS vulnerability. Test where the output lands.
| Output sink | Injection goal | Resulting bug class |
|---|---|---|
| Browser (unescaped HTML) | Emit <img onerror> / <script> | Stored / reflected XSS |
| SQL query builder | Emit a crafted WHERE clause | SQL injection |
| os.system / eval | Emit shell or Python | Command / code execution |
| Server-side fetch | Emit an internal URL | SSRF |
A common indirect-injection payload asks the model to append conversation data to an attacker-controlled URL rendered as a markdown image, exfiltrating on render.
Report exfiltration findings as high/critical even in a lab. Auto-rendered markdown images and hyperlinks are the most common real-world injection-to-exfil path, and they bypass output-text filters that only look for "bad words."
The strongest control against excessive agency is not a better prompt — it is architecture: give the model the least privilege, require human confirmation for state-changing actions, and treat every model output as untrusted input to the next system.
10 — Defense, Detection & Monitoring
Phase 10 / 12You cannot fully "patch" prompt injection — instruction/data confusion is inherent to how LLMs work. Defense is depth: reduce the blast radius, filter both sides, and monitor. OWASP's own guidance is defense-in-depth combining input validation, output filtering, privilege restriction, and human-in-the-loop for sensitive operations.
eval, a shell, or a raw SQL string.Deploy a dedicated guardrail in front of and behind the model. These catch a large share of known payloads — but treat them as speed bumps, not fences.
| Guardrail | Type | Role |
|---|---|---|
| LLM Guard | Open source | Input/output scanners: prompt-injection, PII, toxicity, code |
| NVIDIA NeMo Guardrails | Open source | Programmable rails for topic/flow control |
| Rebuff / Lakera Guard | OSS / SaaS | Injection detection at the API boundary |
| Constitutional Classifiers | Model-level (Anthropic) | Classifier layer that filters jailbreak attempts |
Re-run your garak and Promptfoo scans with the guardrail enabled and compare the success rate. A guardrail that only drops your pass rate from 40% to 30% is not a control — quantify it before you trust it.
Log every prompt, response, tool call, and guardrail verdict to your SIEM as structured events. Hunt for the linguistic and behavioural signatures of injection.
- Full prompt + response (hashed/redacted for PII as needed)
- Guardrail verdict on input and output
- Every tool/function call and its arguments
- Session, user, and source IP for correlation
- Model, version, and system-prompt hash
Regex hunts catch known phrasings only. Encoded, translated, and novel payloads will slip past — pair signature detection with anomaly detection on tool-call volume, output length, and guardrail-block rate per session.
11 — Troubleshooting & Common Mistakes
Phase 11 / 12| Symptom | Likely cause | Fix |
|---|---|---|
| curl to :11434 refused | Ollama not running / not bound | ollama serve or systemctl status ollama; check the port |
| App error: model not found | Model not pulled in that environment | ollama pull llama3.2:3b (or inside the container) |
| chatapp can't reach ollama | Used localhost inside Docker | Use the service name: http://ollama:11434 |
| garak install fails | Python < 3.10 / dependency clash | Fresh venv on 3.11+; pip install -U garak |
| Every payload refuses | Aligned model / working guardrail | Expected — record the low rate; try multi-turn (PyRIT) and encoding |
| Inconsistent results | Non-deterministic sampling | Repeat 5–10× per payload; report success rate, lower temperature to compare |
| Promptfoo target 404 | Wrong URL / response transform | Verify transformResponse matches your JSON shape (json.reply) |
The single most common reporting mistake: claiming an app is "vulnerable to prompt injection" from one lucky success. Attach the success rate, the exact payloads, the model + version, and the downstream impact. A 2% leak rate with tool access is more severe than a 60% leak rate on a chat-only bot.
12 — Sources & References
Phase 12 / 12Assessing an AI feature in your stack? Prompt injection is now a standard line item in any web-app or product pentest. Start with garak for coverage, prove impact with the PortSwigger labs, and wire the Phase 10 KQL/SPL hunts into your SIEM before you ship an LLM feature to production.
Explore more hands-on security tooling guides and detection content on the CyberHawk blog, grab ready-to-run playbooks from our SOP library, or run indicators through the IOC Scanner.
◈ Stay Connected
Follow CyberHawk Threat Intel for threat intelligence, deployment guides and hands-on SOC tooling content.
"They can't exploit you if you are the Exploit."