Add RAG sensitive data exposure#19
Open
luks-santos wants to merge 8 commits into
Open
Conversation
|
Warning Review limit reached
More reviews will be available in 50 minutes and 49 seconds. Learn how PR review limits work. Your organization has run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughA new RAG Sensitive Data Exposure lab is introduced with a FAISS+SQLite vector store, three vulnerability levels, FastAPI endpoints, and a full frontend UI for interactive testing and secret validation. ChangesRAG Sensitive Data Exposure Lab
Sequence Diagram(s)sequenceDiagram
participant Browser as Browser / User
participant Controller as RAG Controller
participant Lab as Lab Pipeline
participant Store as Vector Store
participant Embedder as Embedding Service
participant LLM as LLM Chat
Browser->>Controller: POST /level{N} with user_input
Controller->>Lab: evaluate_level(level, user_input, model)
Lab->>Lab: validate input length & denylist
Lab->>Store: ensure_level_indexed(level)
Store->>Embedder: embed all doc chunks
Embedder->>Store: embeddings + metadata
Store->>Store: index via FAISS
Lab->>Embedder: embed user_input
Lab->>Store: search(embedding, top_k)
Store->>Lab: ranked matches + metadata
Lab->>Lab: format context from matches
Lab->>LLM: call with system_prompt + context
LLM->>Lab: model output
Lab->>Lab: detect secret in output
Lab->>Controller: response with retrieved_docs, leak reason
Controller->>Browser: JSON: {retrieved_docs, input_accepted, secret_exposed}
Browser->>Browser: render docs list, show leak feedback
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
luks-santos
changed the title
Lexical denylist (password, secret, admin) bypassed by semantic paraphrase retrieval. - Add L2 corpus and denylist input handler - Add level2 endpoint and locale entries - Make the facade level-aware
…tive-data-exposure
luks-santos
force-pushed
the
rag-sensitive-data-exposure
branch
from
d3b5626 to
5ef3d01
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/controllers/rag_data_exposure_controller.py`:
- Around line 47-53: The controller currently treats any non-"validate" action
as "generate"; add an explicit allowlist check for action values ("validate" or
"generate") and reject others with an error response. In the handler around the
action variable, validate that action is one of {"validate","generate"} and
return a 4xx error (or raise the appropriate HTTPException) for invalid values;
keep existing calls to validate_rag_data_exposure_secret when action ==
"validate" and to evaluate_rag_data_exposure_level when action == "generate"
(use the same user_input/model extraction). Ensure you reference the action
variable, validate_rag_data_exposure_secret, and
evaluate_rag_data_exposure_level when implementing this guard so bad clients
fail fast instead of following the wrong path.
- Around line 42-44: In the _handle_level method move the JSON parsing (await
request.json()) inside the existing try/except so malformed JSON can't bubble
out; specifically, call await request.json() within the try block at the start
of _handle_level, validate/normalize action = str(data.get("action",
"generate")).strip().lower() there, and in the except handle
JSONDecodeError/ValueError by returning a controlled error dict (and/or
appropriate status payload) instead of letting an unhandled 500 occur.
In `@src/framework/registry.py`:
- Around line 99-100: The branch comparing controller_name uses a hyphenated
string ("rag-sensitive-data-exposure") but other dispatches expect underscored
names (e.g., "rag_sensitive_data_exposure"), causing mismatches; update the
dispatch logic in the function that handles controller_name (the branch that
calls validate_rag_data_exposure_secret) to normalize controller_name (e.g.,
controller_name = controller_name.replace('-', '_') or compare against both
forms) before the if checks so the compare will match and
validate_rag_data_exposure_secret(level_number, candidate_secret) is reached for
registrations named rag_sensitive_data_exposure.
In `@src/service/vulnerabilities/rag_data_exposure_lab.py`:
- Around line 195-231: The current flow appends vectors to FAISS before
inserting rows into SQLite which can cause FAISS/DB divergence on
unique-constraint failures; change the order so you build the DB rows and
perform the INSERT (using self._connect() and handling
sqlite3.IntegrityError/unique-constraint races by skipping or deduping duplicate
rows) before mutating FAISS, then compute start_vector_id = int(index.ntotal),
call faiss.normalize_L2(matrix) and index.add(matrix); apply the same
swap-and-tolerate-duplicate-insert logic to the other identical block referenced
around lines 461-471 (use symbols: index.add, faiss.normalize_L2,
start_vector_id, self._connect, rows, documents).
In `@src/static/facade/rag_data_exposure_template.html`:
- Around line 11-21: The textarea (id="ragExposurePrompt") and the secret input
lack programmatic labels; add accessible labels by either inserting <label
for="ragExposurePrompt">Prompt</label> tied to the textarea and a corresponding
<label for="..."> for the secret input, or by adding clear aria-label attributes
to those elements, and if needed use a visually-hidden utility class to keep
visual layout unchanged; ensure the labels reference the exact element ids used
in this template so screen readers can announce the fields.
In `@src/static/facade/rag_data_exposure_template.js`:
- Around line 72-78: The UI currently accepts any integer level >=1 (from the
level parsing code) but endpointForLevel only implements level1..level3, causing
404s for higher levels; clamp the parsed level to the supported range (1..3) or
clamp inside endpointForLevel so that any incoming level is reduced to
Math.min(Math.max(level, 1), 3) before building the URL. Update the level
parsing function or endpointForLevel (referencing endpointForLevel and the
level-parsing code that sets const level = Number(match[1])) to enforce this
clamp so requests always target /level1, /level2, or /level3.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 14462f52-df10-4037-8223-dc7e7b71afe4
📒 Files selected for processing (13)
locale/messages_us.propertiessrc/app.pysrc/controllers/rag_data_exposure_controller.pysrc/framework/decorators.pysrc/framework/registry.pysrc/service/vulnerabilities/__init__.pysrc/service/vulnerabilities/docs/RAG_DATA_EXPOSURE/LEVEL1/documents.jsonsrc/service/vulnerabilities/docs/RAG_DATA_EXPOSURE/LEVEL2/documents.jsonsrc/service/vulnerabilities/docs/RAG_DATA_EXPOSURE/LEVEL3/documents.jsonsrc/service/vulnerabilities/rag_data_exposure_lab.pysrc/static/facade/rag_data_exposure_template.csssrc/static/facade/rag_data_exposure_template.htmlsrc/static/facade/rag_data_exposure_template.js
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/controllers/rag_data_exposure_controller.py`:
- Around line 44-47: The handler assumes request.json() returns a dict and calls
data.get(), which will raise if JSON root is a list/string; update the code
around data = await request.json() to validate that data is a dict (e.g.,
isinstance(data, dict)) before using data.get("action", ...), and if it isn't,
return a structured error (e.g., {"error": "Invalid JSON root type: expected
object"}) so the existing action handling (action variable and the action not in
{"generate","validate"} check) only runs on valid input.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 10128d38-5302-4a02-9134-8cdfeb79cb2d
📒 Files selected for processing (5)
src/controllers/rag_data_exposure_controller.pysrc/framework/registry.pysrc/service/vulnerabilities/rag_data_exposure_lab.pysrc/static/facade/rag_data_exposure_template.htmlsrc/static/facade/rag_data_exposure_template.js
🚧 Files skipped from review as they are similar to previous changes (4)
- src/static/facade/rag_data_exposure_template.html
- src/framework/registry.py
- src/static/facade/rag_data_exposure_template.js
- src/service/vulnerabilities/rag_data_exposure_lab.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
This PR implements a RAG Sensitive Data Exposure lab.
The implementation uses a
RagDataExposureVectorStore, which combines FAISS with SQLite to store vectors by namespace (one namespace per level) and persist them for later retrieval.In the interface, it adds a facade that displays the retrieved segments and validates the leaked secret against the model's responses.
How It Works
vector_id) for each namespace.Levels
Each level builds on the same pipeline, changing only the defense that sits in front of retrieval.
Level 1 — Direct Retrieval
A direct query embeds close to the sensitive chunk, FAISS retrieves it, and the model leaks the secret straight back.
Level 2 — Semantic Paraphrase Bypass
The denylist is purely lexical, but retrieval is semantic — so a paraphrase (e.g. "internal recovery value", "privileged access") still lands on the sensitive chunk and leaks it.
Level 3 — Misclassified Low-Sensitivity Document
A document tagged as
lowstill contains one chunk with a secret, so document-level tags are too coarse and the sensitive chunk passes the filter.Closes #8
Summary by CodeRabbit
New Features
Documentation