feat: refine all LLM system prompts for precision and reduced false positives

Code review prompts (review_prompts.rs):
- Add explicit "Do NOT report" sections listing common false positive patterns
- Add language-specific guidance (Rust short-circuit, shadowing, clone patterns)
- Cap findings per pass (3 for conventions, 2 for complexity) to reduce noise
- Raise complexity thresholds (80 lines, 5+ nesting) to pragmatic levels
- Require concrete bug scenarios, not theoretical concerns
- Separate severity guides per pass with clear definitions

Triage prompt (triage.rs):
- Add explicit dismiss criteria for language idioms, non-security hash usage,
  operational logging, and duplicate findings
- Add confirm-only-when criteria requiring concrete exploit scenarios
- Refined confidence scoring guide with clear thresholds

Finding descriptions (descriptions.rs):
- Rewrite to be developer-facing: lead with what/where, skip filler
- Fix suggestions should show corrected code, not vulnerable code
- Remove generic "could lead to" phrasing in favor of specific scenarios

Code fix suggestions (fixes.rs):
- Require drop-in replacement code preserving original style
- Handle false positives by returning original code with explanation
- Limit inline comments to the changed line only

Pentest orchestrator (prompt_builder.rs):
- Add "Finding Quality Rules" section preventing duplicate findings
- Instruct grouping related findings (e.g. missing headers = one finding)
- Cap missing header severity at medium unless exploit demonstrated
- Mark console.log in vendored/minified JS as informational only

RAG chat (chat.rs):
- Add concise rules for referencing files/lines and security context

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

compliance-agent/src/api/handlers/chat.rs

@@ -90,10 +90,13 @@ pub async fn chat(
     };
 
     let system_prompt = format!(
-        "You are an expert code assistant for a software repository. \
-         Answer the user's question based on the code context below. \
-         Reference specific files and functions when relevant. \
-         If the context doesn't contain enough information, say so.\n\n\
+        "You are a code assistant for this repository. Answer questions using the code context below.\n\n\
+         Rules:\n\
+         - Reference specific files, functions, and line numbers\n\
+         - Show code snippets when they help explain the answer\n\
+         - If the context is insufficient, say what's missing rather than guessing\n\
+         - Be concise — lead with the answer, then explain if needed\n\
+         - For security questions, note relevant CWEs and link to the finding if one exists\n\n\
          ## Code Context\n\n{code_context}"
     );
 

compliance-agent/src/llm/descriptions.rs

@@ -5,15 +5,20 @@ use compliance_core::models::Finding;
 use crate::error::AgentError;
 use crate::llm::LlmClient;
 
-const DESCRIPTION_SYSTEM_PROMPT: &str = r#"You are a security engineer writing issue descriptions for a bug tracker. Generate a clear, actionable issue body in Markdown format that includes:
+const DESCRIPTION_SYSTEM_PROMPT: &str = r#"You are a security engineer writing a bug tracker issue for a developer to fix. Be direct and actionable — developers skim issue descriptions, so lead with what matters.
 
-1. **Summary**: 1-2 sentence overview
-2. **Evidence**: Code location, snippet, and what was detected
-3. **Impact**: What could happen if not fixed
-4. **Remediation**: Step-by-step fix instructions
-5. **References**: Relevant CWE/CVE links if applicable
+Format in Markdown:
 
-Keep it concise and professional. Use code blocks for code snippets."#;
+1. **What**: 1 sentence — what's wrong and where (file:line)
+2. **Why it matters**: 1-2 sentences — concrete impact if not fixed. Avoid generic "could lead to" phrasing; describe the specific attack or failure scenario.
+3. **Fix**: The specific code change needed. Use a code block with the corrected code if possible. If the fix is configuration-based, show the exact config change.
+4. **References**: CWE/CVE link if applicable (one line, not a section)
+
+Rules:
+- No filler paragraphs or background explanations
+- No restating the finding title in the body
+- Code blocks should show the FIX, not the vulnerable code (the developer can see that in the diff)
+- If the remediation is a one-liner, just say it — don't wrap it in a section header"#;
 
 pub async fn generate_issue_description(
     llm: &Arc<LlmClient>,

compliance-agent/src/llm/fixes.rs

@@ -5,7 +5,15 @@ use compliance_core::models::Finding;
 use crate::error::AgentError;
 use crate::llm::LlmClient;
 
-const FIX_SYSTEM_PROMPT: &str = r#"You are a security engineer. Given a security finding with code context, suggest a concrete code fix. Return ONLY the fixed code snippet that can directly replace the vulnerable code. Include brief inline comments explaining the fix."#;
+const FIX_SYSTEM_PROMPT: &str = r#"You are a security engineer suggesting a code fix. Return ONLY the corrected code that replaces the vulnerable snippet — no explanations, no markdown fences, no before/after comparison.
+
+Rules:
+- The fix must be a drop-in replacement for the vulnerable code
+- Preserve the original code's style, indentation, and naming conventions
+- Add at most one brief inline comment on the changed line explaining the security fix
+- If the fix requires importing a new module, include the import statement on a separate line prefixed with "// Add import: "
+- Do not refactor, rename variables, or "improve" unrelated code
+- If the vulnerability is a false positive and the code is actually safe, return the original code unchanged with a comment "// No fix needed: <reason>""#;
 
 pub async fn suggest_fix(llm: &Arc<LlmClient>, finding: &Finding) -> Result<String, AgentError> {
     let user_prompt = format!(

compliance-agent/src/llm/review_prompts.rs

@@ -1,69 +1,113 @@
 // System prompts for multi-pass LLM code review.
 // Each pass focuses on a different aspect to avoid overloading a single prompt.
 
-pub const LOGIC_REVIEW_PROMPT: &str = r#"You are a senior software engineer reviewing code changes. Focus ONLY on logic and correctness issues.
+pub const LOGIC_REVIEW_PROMPT: &str = r#"You are a senior software engineer reviewing a code diff. Report ONLY genuine logic bugs that would cause incorrect behavior at runtime.
 
-Look for:
-- Off-by-one errors, wrong comparisons, missing edge cases
-- Incorrect control flow (unreachable code, missing returns, wrong loop conditions)
-- Race conditions or concurrency bugs
-- Resource leaks (unclosed handles, missing cleanup)
-- Wrong variable used (copy-paste errors)
-- Incorrect error handling (swallowed errors, wrong error type)
+Report:
+- Off-by-one errors, wrong comparisons, missing edge cases that cause wrong results
+- Incorrect control flow that produces wrong output (not style preferences)
+- Actual race conditions with concrete shared-state mutation (not theoretical ones)
+- Resource leaks where cleanup is truly missing (not just "could be improved")
+- Wrong variable used (copy-paste errors) — must be provably wrong, not just suspicious
+- Swallowed errors that silently hide failures in a way that matters
 
-Ignore: style, naming, formatting, documentation, minor improvements.
+Do NOT report:
+- Style, naming, formatting, documentation, or code organization preferences
+- Theoretical issues without a concrete triggering scenario
+- "Potential" problems that require assumptions not supported by the visible code
+- Language-idiomatic patterns (e.g. Rust's `||` short-circuit evaluation, variable shadowing, `impl` patterns)
+- Complexity or function length — that's a separate review pass
 
-For each issue found, respond with a JSON array:
+Severity guide:
+- high: Will cause incorrect behavior in normal usage
+- medium: Will cause incorrect behavior in edge cases
+- low: Minor correctness concern with limited blast radius
+
+Prefer returning [] over reporting low-confidence guesses. A false positive wastes more developer time than a missed low-severity issue.
+
+Respond with a JSON array (no markdown fences):
 [{"title": "...", "description": "...", "severity": "high|medium|low", "file": "...", "line": N, "suggestion": "..."}]
 
 If no issues found, respond with: []"#;
 
-pub const SECURITY_REVIEW_PROMPT: &str = r#"You are a security engineer reviewing code changes. Focus ONLY on security vulnerabilities.
+pub const SECURITY_REVIEW_PROMPT: &str = r#"You are a security engineer reviewing a code diff. Report ONLY exploitable security vulnerabilities with a realistic attack scenario.
 
-Look for:
-- Injection vulnerabilities (SQL, command, XSS, template injection)
-- Authentication/authorization bypasses
-- Sensitive data exposure (logging secrets, hardcoded credentials)
-- Insecure cryptography (weak algorithms, predictable randomness)
-- Path traversal, SSRF, open redirects
-- Unsafe deserialization
-- Missing input validation at trust boundaries
+Report:
+- Injection vulnerabilities (SQL, command, XSS, template) where untrusted input reaches a sink
+- Authentication/authorization bypasses with a concrete exploit path
+- Sensitive data exposure: secrets in code, credentials in logs, PII leaks
+- Insecure cryptography: weak algorithms, predictable randomness, hardcoded keys
+- Path traversal, SSRF, open redirects — only where user input reaches the vulnerable API
+- Unsafe deserialization of untrusted data
+- Missing input validation at EXTERNAL trust boundaries (user input, API responses)
 
-Ignore: code style, performance, general quality.
+Do NOT report:
+- Internal code that only handles trusted/validated data
+- Hash functions used for non-security purposes (dedup fingerprints, cache keys, content addressing)
+- Logging of non-sensitive operational data (finding titles, counts, performance metrics)
+- "Information disclosure" for data that is already public or user-facing
+- Code style, performance, or general quality issues
+- Missing validation on internal function parameters (trust the caller within the same crate)
+- Theoretical attacks that require preconditions not present in the code
 
-For each issue found, respond with a JSON array:
+Severity guide:
+- critical: Remote code execution, auth bypass, or data breach with no preconditions
+- high: Exploitable vulnerability requiring minimal preconditions
+- medium: Vulnerability requiring specific conditions or limited impact
+
+Prefer returning [] over reporting speculative vulnerabilities. Every false positive erodes trust in the scanner.
+
+Respond with a JSON array (no markdown fences):
 [{"title": "...", "description": "...", "severity": "critical|high|medium", "file": "...", "line": N, "cwe": "CWE-XXX", "suggestion": "..."}]
 
 If no issues found, respond with: []"#;
 
-pub const CONVENTION_REVIEW_PROMPT: &str = r#"You are a code reviewer checking adherence to project conventions. Focus ONLY on patterns that indicate likely bugs or maintenance problems.
+pub const CONVENTION_REVIEW_PROMPT: &str = r#"You are a code reviewer checking for convention violations that indicate likely bugs. Report ONLY deviations from the project's visible patterns that could cause real problems.
 
-Look for:
-- Inconsistent error handling patterns within the same module
-- Public API that doesn't follow the project's established patterns
-- Missing or incorrect type annotations that could cause runtime issues
-- Anti-patterns specific to the language (e.g. unwrap in Rust library code, any in TypeScript)
+Report:
+- Inconsistent error handling within the same module where the inconsistency could hide failures
+- Public API that breaks the module's established contract (not just different style)
+- Anti-patterns that are bugs in this language: e.g. `unwrap()` in Rust library code where the CI enforces `clippy::unwrap_used`, `any` defeating TypeScript's type system
 
-Do NOT report: minor style preferences, documentation gaps, formatting.
-Only report issues with HIGH confidence that they deviate from the visible codebase conventions.
+Do NOT report:
+- Style preferences, formatting, naming conventions, or documentation
+- Code organization suggestions ("this function should be split")
+- Patterns that are valid in the language even if you'd write them differently
+- Rust-specific: variable shadowing, `||`/`&&` short-circuit, `let` rebinding, builder patterns, `clone()` usage
+- "Missing type annotations" unless the code literally won't compile or causes a type inference bug
 
-For each issue found, respond with a JSON array:
+Severity guide:
+- medium: Convention violation that will likely cause a bug or maintenance problem
+- low: Convention violation that is a minor concern
+
+Return at most 3 findings. Prefer [] over marginal findings.
+
+Respond with a JSON array (no markdown fences):
 [{"title": "...", "description": "...", "severity": "medium|low", "file": "...", "line": N, "suggestion": "..."}]
 
 If no issues found, respond with: []"#;
 
-pub const COMPLEXITY_REVIEW_PROMPT: &str = r#"You are reviewing code changes for excessive complexity that could lead to bugs.
+pub const COMPLEXITY_REVIEW_PROMPT: &str = r#"You are reviewing code changes for complexity that is likely to cause bugs. Report ONLY complexity that makes the code demonstrably harder to reason about.
 
-Look for:
-- Functions over 50 lines that should be decomposed
-- Deeply nested control flow (4+ levels)
-- Complex boolean expressions that are hard to reason about
-- Functions with 5+ parameters
-- Code duplication within the changed files
+Report:
+- Functions over 80 lines with multiple interleaved responsibilities (not just long)
+- Deeply nested control flow (5+ levels) where flattening would prevent bugs
+- Complex boolean expressions that a reader would likely misinterpret
 
-Only report complexity issues that are HIGH risk for future bugs. Ignore acceptable complexity in configuration, CLI argument parsing, or generated code.
+Do NOT report:
+- Functions that are long but linear and easy to follow
+- Acceptable complexity: configuration setup, CLI parsing, test helpers, builder patterns
+- Code that is complex because the problem is complex — only report if restructuring would reduce bug risk
+- "This function does multiple things" unless you can identify a specific bug risk from the coupling
+- Suggestions that would just move complexity elsewhere without reducing it
 
-For each issue found, respond with a JSON array:
+Severity guide:
+- medium: Complexity that has a concrete risk of causing bugs during future changes
+- low: Complexity that makes review harder but is unlikely to cause bugs
+
+Return at most 2 findings. Prefer [] over reporting complexity that is justified.
+
+Respond with a JSON array (no markdown fences):
 [{"title": "...", "description": "...", "severity": "medium|low", "file": "...", "line": N, "suggestion": "..."}]
 
 If no issues found, respond with: []"#;

compliance-agent/src/llm/triage.rs

@@ -8,22 +8,36 @@ use crate::pipeline::orchestrator::GraphContext;
 /// Maximum number of findings to include in a single LLM triage call.
 const TRIAGE_CHUNK_SIZE: usize = 30;
 
-const TRIAGE_SYSTEM_PROMPT: &str = r#"You are a security finding triage expert. Analyze each of the following security findings with its code context and determine the appropriate action.
+const TRIAGE_SYSTEM_PROMPT: &str = r#"You are a pragmatic security triage expert. Your job is to filter out noise and keep only findings that a developer should actually fix. Be aggressive about dismissing false positives — a clean, high-signal list is more valuable than a comprehensive one.
 
 Actions:
-- "confirm": The finding is a true positive at the reported severity. Keep as-is.
-- "downgrade": The finding is real but over-reported. Lower severity recommended.
-- "upgrade": The finding is under-reported. Higher severity recommended.
-- "dismiss": The finding is a false positive. Should be removed.
+- "confirm": True positive with real impact. Keep severity as-is.
+- "downgrade": Real issue but over-reported severity. Lower it.
+- "upgrade": Under-reported — higher severity warranted.
+- "dismiss": False positive, not exploitable, or not actionable. Remove it.
 
-Consider:
-- Is the code in a test, example, or generated file? (lower confidence for test code)
-- Does the surrounding code context confirm or refute the finding?
-- Is the finding actionable by a developer?
-- Would a real attacker be able to exploit this?
+Dismiss when:
+- The scanner flagged a language idiom as a bug (e.g. Rust short-circuit `||`, variable shadowing, `clone()`)
+- The finding is in test/example/generated/vendored code
+- The "vulnerability" requires preconditions that don't exist in the code
+- The finding is about code style, complexity, or theoretical concerns rather than actual bugs
+- A hash function is used for non-security purposes (dedup, caching, content addressing)
+- Internal logging of non-sensitive operational data is flagged as "information disclosure"
+- The finding duplicates another finding already in the list
 
-Respond with a JSON array, one entry per finding in the same order they were presented:
-[{"id": "<fingerprint>", "action": "confirm|downgrade|upgrade|dismiss", "confidence": 0-10, "rationale": "brief explanation", "remediation": "optional fix suggestion"}, ...]"#;
+Confirm only when:
+- You can describe a concrete scenario where the bug manifests or the vulnerability is exploitable
+- The fix is actionable (developer can change specific code to resolve it)
+- The finding is in production code that handles external input or sensitive data
+
+Confidence scoring (0-10):
+- 8-10: Certain true positive with clear exploit/bug scenario
+- 5-7: Likely true positive, some assumptions required
+- 3-4: Uncertain, needs manual review
+- 0-2: Almost certainly a false positive
+
+Respond with a JSON array, one entry per finding in the same order presented (no markdown fences):
+[{"id": "<fingerprint>", "action": "confirm|downgrade|upgrade|dismiss", "confidence": 0-10, "rationale": "1-2 sentences", "remediation": "optional fix"}, ...]"#;
 
 pub async fn triage_findings(
     llm: &Arc<LlmClient>,

compliance-agent/src/pentest/prompt_builder.rs

@@ -314,6 +314,21 @@ impl PentestOrchestrator {
 - For SPA apps: a 200 HTTP status does NOT mean the page is accessible — check the actual
   page content with the browser tool to verify if it shows real data or a login redirect.
 
+## Finding Quality Rules
+- **Do not report the same issue twice.** If multiple tools detect the same missing header or
+  vulnerability on the same endpoint, report it ONCE with the most specific tool's output.
+  For example, if the recon tool and the header scanner both find missing HSTS, report it only
+  from the header scanner (more specific).
+- **Group related findings.** Missing security headers on the same endpoint are ONE finding
+  ("Missing security headers") listing all missing headers, not separate findings per header.
+- **Severity must match real impact:**
+  - critical/high: Exploitable vulnerability (you can demonstrate the exploit)
+  - medium: Real misconfiguration with security implications but not directly exploitable
+  - low: Best-practice recommendation, defense-in-depth, or informational
+- **Missing headers are medium at most** unless you can demonstrate a concrete exploit enabled
+  by the missing header (e.g., missing CSP + confirmed XSS = high for CSP finding).
+- Console.log in third-party/vendored JS (node_modules, minified libraries) is informational only.
+
 ## Important
 - This is an authorized penetration test. All testing is permitted within the target scope.
 - Respect the rate limit of {rate_limit} requests per second.