This is a portfolio project, not a hardened production deployment. It is a RAG system over regulatory text, framed as "RAG you'd put in front of a regulator," so it owes an explicit threat model rather than an implicit one. This document states what is defended, what is not, and where the existing controls live in the code.
Found something? Open a GitHub issue (omit any exploit payload from the public description) or contact the maintainer via the profile on github.com/dkling-it. There is no production service or user data behind this repo; the surface is the code and the demo Space.
The interesting attack surface for a retrieval system is the retrieval channel: content that reaches the model as "context" is attacker-influenceable in the general case (indirect prompt injection / corpus poisoning). 2026 research shows a handful of crafted documents can flip RAG output most of the time, so the posture here is assume-breach: design so a poisoned or adversarial passage cannot silently become an authoritative-looking answer.
| Threat | This project's exposure | Control |
|---|---|---|
| Indirect prompt injection (instructions embedded in a retrieved passage) | Low in demo (static bundled corpus); real on the live path if the source text is tampered with | The extractive demo answerer does not interpret retrieved text as instructions. The live Claude answerer is instructed to answer only from [n]-cited context, and citation enforcement (src/rag_eu_ai_act/generate/citations.py) drops any claim that doesn't resolve to a retrieved passage; an injected instruction that isn't itself the cited regulatory text cannot become a grounded claim |
| Corpus poisoning (tampering with the ingested EU AI Act text) | Live ingestion fetches EUR-Lex text on demand | Data lineage: every live index build writes a manifest with a source hash, chunking parameters, and embedding model (src/rag_eu_ai_act/ingest/manifest.py). A changed source hash is detectable and auditable; the corpus is public EUR-Lex (CELEX 32024R1689), so ground truth is independently verifiable |
| Citation fabrication / hallucinated authority | Core risk for legal RAG (documented 17-43% hallucination rates in commercial legal tools) | Grounding is enforced, not requested: ungrounded answers surface is_grounded=false instead of being hidden, and attribution-precision (src/rag_eu_ai_act/generate/attribution.py) measures whether a cited passage actually supports its claim, not merely that the citation resolves |
| Secret exposure | Anthropic API key on the live path | Secrets are read only from the environment / .env (RAG_ANTHROPIC_API_KEY); never committed. CI and the demo run with no key, no network, no model download: the default attack surface is empty |
| Supply chain | Python dependencies | Version-constrained via pyproject.toml; uv.lock is generated locally for reproducible dev installs but isn't committed. The hermetic test gate (make check) runs offline |
Stated plainly so they aren't mistaken for silent coverage:
- No input-sanitisation / instruction-stripping layer on retrieved content. The defense today is architectural (grounding + citation enforcement), not a content firewall. A retrieval-content trust score is a candidate next step.
- No rate limiting / authn on the FastAPI service: it is a demo, not a multi-tenant endpoint.
- No adversarial eval yet (injection-laced questions, poisoned-passage fixtures). The offline eval set tests retrieval/grounding correctness, not robustness under attack; a natural extension of the existing harness.
The product boundary (retrieve, ground, enforce citations, drop the
unsupported) is also the primary injection containment: the system's worst-case
output is a missing or flagged-ungrounded answer, not a confident fabricated
one. That property is measured (grounded-rate, attribution-precision), not
asserted. See EVAL.md.