In brief. promptmap points at your own LLM application's HTTP endpoint and fires a library of known injection and jailbreak payloads — system-prompt extraction, secret exfiltration, instruction override, delimiter confusion, indirect injection — then reports which ones got through. Detection is layered: a canary secret and rule checks run with no setup, and an optional LLM judge adds nuance. It exits non-zero on a breach, so it drops into CI as a guardrail regression test.
Why I built it
Most of my research is about whether an AI system can be talked into doing something it shouldn't, and whether anyone would notice. That question is just as practical for anyone shipping an LLM feature: your system prompt and your guardrails are a security boundary, and like any boundary they deserve to be tested on purpose rather than discovered in production. promptmap is the testing side of that — a harness you run against your own app before someone else runs the attacks for you.
How it works
You describe your app's HTTP API in a small JSON config — the URL, the request body with a {{prompt}} placeholder, and where the model's reply lives in the response. The tool sends each attack through that endpoint and judges the result. The attack library is grouped by technique and easy to extend with cases specific to your own app.
Detection runs in layers so it is useful out of the box and sharper when you want it:
- Canary — a unique secret is planted (via a
{{canary}}placeholder you put in your system prompt); if it appears in a response, that is unambiguous proof of leakage. - Rule — regular-expression markers for compliance and the absence of a refusal. Fast and free, but heuristic.
- LLM judge — an optional second model scores whether the response actually complied with the attack's goal. Best for the nuanced harmful-compliance cases, off unless you enable it and provide a key.
Using it
# see both outcomes with no setup
python -m promptmap --demo vulnerable
python -m promptmap --demo hardened
# generate a config template, edit it, then run
python -m promptmap --init > target.json
python -m promptmap --config target.json
# add the optional LLM judge
python -m promptmap --config target.json --judgeBecause it exits non-zero when any attack succeeds, you can wire it into a pipeline so a guardrail regression fails the build:
- run: python -m promptmap --config target.jsonScope and honesty about limits
This is a defensive tool — run it against systems you own or are authorized to test. It tests single-shot injection well; multi-turn and tool-mediated injection through real tool calls are only partially covered. The rule layer is heuristic and can miss a cleverly-worded compliance or over-flag a response that merely quotes the attack, which is exactly why the canary and the optional judge exist. Treat the report as triage, and confirm the failures.