feat: refine all LLM system prompts for precision and reduced false positives
Some checks failed
CI / Check (pull_request) Successful in 10m8s
CI / Detect Changes (pull_request) Has been skipped
CI / Deploy Agent (pull_request) Has been cancelled
CI / Deploy Dashboard (pull_request) Has been cancelled
CI / Deploy Docs (pull_request) Has been cancelled
CI / Deploy MCP (pull_request) Has been cancelled

Code review prompts (review_prompts.rs):
- Add explicit "Do NOT report" sections listing common false positive patterns
- Add language-specific guidance (Rust short-circuit, shadowing, clone patterns)
- Cap findings per pass (3 for conventions, 2 for complexity) to reduce noise
- Raise complexity thresholds (80 lines, 5+ nesting) to pragmatic levels
- Require concrete bug scenarios, not theoretical concerns
- Separate severity guides per pass with clear definitions

Triage prompt (triage.rs):
- Add explicit dismiss criteria for language idioms, non-security hash usage,
  operational logging, and duplicate findings
- Add confirm-only-when criteria requiring concrete exploit scenarios
- Refined confidence scoring guide with clear thresholds

Finding descriptions (descriptions.rs):
- Rewrite to be developer-facing: lead with what/where, skip filler
- Fix suggestions should show corrected code, not vulnerable code
- Remove generic "could lead to" phrasing in favor of specific scenarios

Code fix suggestions (fixes.rs):
- Require drop-in replacement code preserving original style
- Handle false positives by returning original code with explanation
- Limit inline comments to the changed line only

Pentest orchestrator (prompt_builder.rs):
- Add "Finding Quality Rules" section preventing duplicate findings
- Instruct grouping related findings (e.g. missing headers = one finding)
- Cap missing header severity at medium unless exploit demonstrated
- Mark console.log in vendored/minified JS as informational only

RAG chat (chat.rs):
- Add concise rules for referencing files/lines and security context

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Sharang Parnerkar
2026-03-29 22:57:37 +02:00
parent ff088f9eb4
commit da4084ee78
6 changed files with 152 additions and 63 deletions

View File

@@ -5,15 +5,20 @@ use compliance_core::models::Finding;
use crate::error::AgentError;
use crate::llm::LlmClient;
const DESCRIPTION_SYSTEM_PROMPT: &str = r#"You are a security engineer writing issue descriptions for a bug tracker. Generate a clear, actionable issue body in Markdown format that includes:
const DESCRIPTION_SYSTEM_PROMPT: &str = r#"You are a security engineer writing a bug tracker issue for a developer to fix. Be direct and actionable — developers skim issue descriptions, so lead with what matters.
1. **Summary**: 1-2 sentence overview
2. **Evidence**: Code location, snippet, and what was detected
3. **Impact**: What could happen if not fixed
4. **Remediation**: Step-by-step fix instructions
5. **References**: Relevant CWE/CVE links if applicable
Format in Markdown:
Keep it concise and professional. Use code blocks for code snippets."#;
1. **What**: 1 sentence — what's wrong and where (file:line)
2. **Why it matters**: 1-2 sentences — concrete impact if not fixed. Avoid generic "could lead to" phrasing; describe the specific attack or failure scenario.
3. **Fix**: The specific code change needed. Use a code block with the corrected code if possible. If the fix is configuration-based, show the exact config change.
4. **References**: CWE/CVE link if applicable (one line, not a section)
Rules:
- No filler paragraphs or background explanations
- No restating the finding title in the body
- Code blocks should show the FIX, not the vulnerable code (the developer can see that in the diff)
- If the remediation is a one-liner, just say it — don't wrap it in a section header"#;
pub async fn generate_issue_description(
llm: &Arc<LlmClient>,