diff --git a/README.md b/README.md index c7da379..9433058 100644 --- a/README.md +++ b/README.md @@ -28,9 +28,9 @@ ## About -Compliance Scanner is an autonomous agent that continuously monitors git repositories for security vulnerabilities, GDPR/OAuth compliance patterns, and dependency risks. It creates issues in external trackers (GitHub/GitLab/Jira) with evidence and remediation suggestions, reviews pull requests, and exposes a Dioxus-based dashboard for visualization. +Compliance Scanner is an autonomous agent that continuously monitors git repositories for security vulnerabilities, GDPR/OAuth compliance patterns, and dependency risks. It creates issues in external trackers (GitHub/GitLab/Jira/Gitea) with evidence and remediation suggestions, reviews pull requests with multi-pass LLM analysis, runs autonomous penetration tests, and exposes a Dioxus-based dashboard for visualization. -> **How it works:** The agent runs as a lazy daemon -- it only scans when new commits are detected, triggered by cron schedules or webhooks. LLM-powered triage filters out false positives and generates actionable remediation. +> **How it works:** The agent runs as a lazy daemon -- it only scans when new commits are detected, triggered by cron schedules or webhooks. LLM-powered triage filters out false positives and generates actionable remediation with multi-language awareness. ## Features @@ -41,31 +41,38 @@ Compliance Scanner is an autonomous agent that continuously monitors git reposit | **CVE Monitoring** | OSV.dev batch queries, NVD CVSS enrichment, SearXNG context | | **GDPR Patterns** | Detect PII logging, missing consent, hardcoded retention, missing deletion | | **OAuth Patterns** | Detect implicit grant, missing PKCE, token in localStorage, token in URLs | -| **LLM Triage** | Confidence scoring via LiteLLM to filter false positives | -| **Issue Creation** | Auto-create issues in GitHub, GitLab, or Jira with code evidence | -| **PR Reviews** | Post security review comments on pull requests | -| **Dashboard** | Fullstack Dioxus UI with findings, SBOM, issues, and statistics | -| **Webhooks** | GitHub (HMAC-SHA256) and GitLab webhook receivers for push/PR events | +| **LLM Triage** | Multi-language-aware confidence scoring (Rust, Python, Go, Java, Ruby, PHP, C++) | +| **Issue Creation** | Auto-create issues in GitHub, GitLab, Jira, or Gitea with dedup via fingerprints | +| **PR Reviews** | Multi-pass security review (logic, security, convention, complexity) with dedup | +| **DAST Scanning** | Black-box security testing with endpoint discovery and parameter fuzzing | +| **AI Pentesting** | Autonomous LLM-orchestrated penetration testing with encrypted reports | +| **Code Graph** | Interactive code knowledge graph with impact analysis | +| **AI Chat (RAG)** | Natural language Q&A grounded in repository source code | +| **Help Assistant** | Documentation-grounded help chat accessible from every dashboard page | +| **MCP Server** | Expose live security data to Claude, Cursor, and other AI tools | +| **Dashboard** | Fullstack Dioxus UI with findings, SBOM, issues, DAST, pentest, and graph | +| **Webhooks** | GitHub, GitLab, and Gitea webhook receivers for push/PR events | +| **Finding Dedup** | SHA-256 fingerprint dedup for SAST, CWE-based dedup for DAST findings | ## Architecture ``` -┌─────────────────────────────────────────────────────────────┐ -│ Cargo Workspace │ -├──────────────┬──────────────────┬───────────────────────────┤ -│ compliance- │ compliance- │ compliance- │ -│ core │ agent │ dashboard │ -│ (lib) │ (bin) │ (bin, Dioxus 0.7.3) │ -│ │ │ │ -│ Models │ Scan Pipeline │ Fullstack Web UI │ -│ Traits │ LLM Client │ Server Functions │ -│ Config │ Issue Trackers │ Charts + Tables │ -│ Errors │ Scheduler │ Settings Page │ -│ │ REST API │ │ -│ │ Webhooks │ │ -└──────────────┴──────────────────┴───────────────────────────┘ - │ - MongoDB (shared) +┌──────────────────────────────────────────────────────────────────────────┐ +│ Cargo Workspace │ +├──────────────┬──────────────────┬──────────────┬──────────┬─────────────┤ +│ compliance- │ compliance- │ compliance- │ complian-│ compliance- │ +│ core (lib) │ agent (bin) │ dashboard │ ce-graph │ mcp (bin) │ +│ │ │ (bin) │ (lib) │ │ +│ Models │ Scan Pipeline │ Dioxus 0.7 │ Tree- │ MCP Server │ +│ Traits │ LLM Client │ Fullstack UI │ sitter │ Live data │ +│ Config │ Issue Trackers │ Help Chat │ Graph │ for AI │ +│ Errors │ Pentest Engine │ Server Fns │ Embedds │ tools │ +│ │ DAST Tools │ │ RAG │ │ +│ │ REST API │ │ │ │ +│ │ Webhooks │ │ │ │ +└──────────────┴──────────────────┴──────────────┴──────────┴─────────────┘ + │ + MongoDB (shared) ``` ## Scan Pipeline (7 Stages) @@ -84,11 +91,16 @@ Compliance Scanner is an autonomous agent that continuously monitors git reposit |-------|-----------| | Shared Library | `compliance-core` -- models, traits, config | | Agent | Axum REST API, git2, tokio-cron-scheduler, Semgrep, Syft | -| Dashboard | Dioxus 0.7.3 fullstack, Tailwind CSS | +| Dashboard | Dioxus 0.7.3 fullstack, Tailwind CSS 4 | +| Code Graph | `compliance-graph` -- tree-sitter parsing, embeddings, RAG | +| MCP Server | `compliance-mcp` -- Model Context Protocol for AI tools | +| DAST | `compliance-dast` -- dynamic application security testing | | Database | MongoDB with typed collections | -| LLM | LiteLLM (OpenAI-compatible API) | -| Issue Trackers | GitHub (octocrab), GitLab (REST v4), Jira (REST v3) | +| LLM | LiteLLM (OpenAI-compatible API for chat, triage, embeddings) | +| Issue Trackers | GitHub (octocrab), GitLab (REST v4), Jira (REST v3), Gitea | | CVE Sources | OSV.dev, NVD, SearXNG | +| Auth | Keycloak (OAuth2/PKCE, SSO) | +| Browser Automation | Chromium (headless, for pentesting and PDF generation) | ## Getting Started @@ -151,20 +163,35 @@ The agent exposes a REST API on port 3001: | `GET` | `/api/v1/sbom` | List dependencies | | `GET` | `/api/v1/issues` | List cross-tracker issues | | `GET` | `/api/v1/scan-runs` | Scan execution history | +| `GET` | `/api/v1/graph/:repo_id` | Code knowledge graph | +| `POST` | `/api/v1/graph/:repo_id/build` | Trigger graph build | +| `GET` | `/api/v1/dast/targets` | List DAST targets | +| `POST` | `/api/v1/dast/targets` | Add DAST target | +| `GET` | `/api/v1/dast/findings` | List DAST findings | +| `POST` | `/api/v1/chat/:repo_id` | RAG-powered code chat | +| `POST` | `/api/v1/help/chat` | Documentation-grounded help chat | +| `POST` | `/api/v1/pentest/sessions` | Create pentest session | +| `POST` | `/api/v1/pentest/sessions/:id/export` | Export encrypted pentest report | | `POST` | `/webhook/github` | GitHub webhook (HMAC-SHA256) | | `POST` | `/webhook/gitlab` | GitLab webhook (token verify) | +| `POST` | `/webhook/gitea` | Gitea webhook | ## Dashboard Pages | Page | Description | |------|-------------| -| **Overview** | Stat cards, severity distribution chart | -| **Repositories** | Add/manage tracked repos, trigger scans | -| **Findings** | Filterable table by severity, type, status | +| **Overview** | Stat cards, severity distribution, AI chat cards, MCP status | +| **Repositories** | Add/manage tracked repos, trigger scans, webhook config | +| **Findings** | Filterable table by severity, type, status, scanner | | **Finding Detail** | Code evidence, remediation, suggested fix, linked issue | -| **SBOM** | Dependency inventory with vulnerability badges | -| **Issues** | Cross-tracker view (GitHub + GitLab + Jira) | -| **Settings** | Configure LiteLLM, tracker tokens, SearXNG URL | +| **SBOM** | Dependency inventory with vulnerability badges, license summary | +| **Issues** | Cross-tracker view (GitHub + GitLab + Jira + Gitea) | +| **Code Graph** | Interactive architecture visualization, impact analysis | +| **AI Chat** | RAG-powered Q&A about repository code | +| **DAST** | Dynamic scanning targets, findings, and scan history | +| **Pentest** | AI-driven pentest sessions, attack chain visualization | +| **MCP Servers** | Model Context Protocol server management | +| **Help Chat** | Floating assistant (available on every page) for product Q&A | ## Project Structure @@ -173,19 +200,24 @@ compliance-scanner/ ├── compliance-core/ Shared library (models, traits, config, errors) ├── compliance-agent/ Agent daemon (pipeline, LLM, trackers, API, webhooks) │ └── src/ -│ ├── pipeline/ 7-stage scan pipeline -│ ├── llm/ LiteLLM client, triage, descriptions, fixes, PR review -│ ├── trackers/ GitHub, GitLab, Jira integrations -│ ├── api/ REST API (Axum) -│ └── webhooks/ GitHub + GitLab webhook receivers +│ ├── pipeline/ 7-stage scan pipeline, dedup, PR reviews, code review +│ ├── llm/ LiteLLM client, triage, descriptions, fixes, review prompts +│ ├── trackers/ GitHub, GitLab, Jira, Gitea integrations +│ ├── pentest/ AI-driven pentest orchestrator, tools, reports +│ ├── rag/ RAG pipeline, chunking, embedding +│ ├── api/ REST API (Axum), help chat +│ └── webhooks/ GitHub, GitLab, Gitea webhook receivers ├── compliance-dashboard/ Dioxus fullstack dashboard │ └── src/ -│ ├── components/ Reusable UI components -│ ├── infrastructure/ Server functions, DB, config -│ └── pages/ Full page views +│ ├── components/ Reusable UI (sidebar, help chat, attack chain, etc.) +│ ├── infrastructure/ Server functions, DB, config, auth +│ └── pages/ Full page views (overview, DAST, pentest, graph, etc.) +├── compliance-graph/ Code knowledge graph (tree-sitter, embeddings, RAG) +├── compliance-dast/ Dynamic application security testing +├── compliance-mcp/ Model Context Protocol server +├── docs/ VitePress documentation site ├── assets/ Static assets (CSS, icons) -├── styles/ Tailwind input stylesheet -└── bin/ Dashboard binary entrypoint +└── styles/ Tailwind input stylesheet ``` ## External Services @@ -193,10 +225,12 @@ compliance-scanner/ | Service | Purpose | Default URL | |---------|---------|-------------| | MongoDB | Persistence | `mongodb://localhost:27017` | -| LiteLLM | LLM proxy for triage and generation | `http://localhost:4000` | +| LiteLLM | LLM proxy (chat, triage, embeddings) | `http://localhost:4000` | | SearXNG | CVE context search | `http://localhost:8888` | +| Keycloak | Authentication (OAuth2/PKCE, SSO) | `http://localhost:8080` | | Semgrep | SAST scanning | CLI tool | | Syft | SBOM generation | CLI tool | +| Chromium | Headless browser (pentesting, PDF) | Managed via Docker | --- diff --git a/compliance-agent/src/api/handlers/help_chat.rs b/compliance-agent/src/api/handlers/help_chat.rs new file mode 100644 index 0000000..78a64cf --- /dev/null +++ b/compliance-agent/src/api/handlers/help_chat.rs @@ -0,0 +1,187 @@ +use std::path::{Path, PathBuf}; +use std::sync::OnceLock; + +use axum::extract::Extension; +use axum::http::StatusCode; +use axum::Json; +use serde::{Deserialize, Serialize}; +use walkdir::WalkDir; + +use super::dto::{AgentExt, ApiResponse}; + +// ── DTOs ───────────────────────────────────────────────────────────────────── + +#[derive(Debug, Deserialize)] +pub struct HelpChatMessage { + pub role: String, + pub content: String, +} + +#[derive(Debug, Deserialize)] +pub struct HelpChatRequest { + pub message: String, + #[serde(default)] + pub history: Vec, +} + +#[derive(Debug, Serialize)] +pub struct HelpChatResponse { + pub message: String, +} + +// ── Doc cache ──────────────────────────────────────────────────────────────── + +static DOC_CONTEXT: OnceLock = OnceLock::new(); + +/// Walk upward from `start` until we find a directory containing both +/// `README.md` and a `docs/` subdirectory. +fn find_project_root(start: &Path) -> Option { + let mut current = start.to_path_buf(); + loop { + if current.join("README.md").is_file() && current.join("docs").is_dir() { + return Some(current); + } + if !current.pop() { + return None; + } + } +} + +/// Read README.md + all docs/**/*.md (excluding node_modules). +fn load_docs(root: &Path) -> String { + let mut parts: Vec = Vec::new(); + + // Root README first + if let Ok(content) = std::fs::read_to_string(root.join("README.md")) { + parts.push(format!("\n{content}")); + } + + // docs/**/*.md, skipping node_modules + for entry in WalkDir::new(root.join("docs")) + .follow_links(false) + .into_iter() + .filter_entry(|e| { + !e.path() + .components() + .any(|c| c.as_os_str() == "node_modules") + }) + .filter_map(|e| e.ok()) + { + let path = entry.path(); + if !path.is_file() { + continue; + } + if path + .extension() + .and_then(|s| s.to_str()) + .map(|s| !s.eq_ignore_ascii_case("md")) + .unwrap_or(true) + { + continue; + } + + let rel = path.strip_prefix(root).unwrap_or(path); + if let Ok(content) = std::fs::read_to_string(path) { + parts.push(format!("\n{content}", rel.display())); + } + } + + if parts.is_empty() { + tracing::warn!( + "help_chat: no documentation files found under {}", + root.display() + ); + } else { + tracing::info!( + "help_chat: loaded {} documentation file(s) from {}", + parts.len(), + root.display() + ); + } + + parts.join("\n\n---\n\n") +} + +/// Returns a reference to the cached doc context string, initialised on +/// first call via `OnceLock`. +fn doc_context() -> &'static str { + DOC_CONTEXT.get_or_init(|| { + let start = std::env::current_exe() + .ok() + .and_then(|p| p.parent().map(Path::to_path_buf)) + .unwrap_or_else(|| PathBuf::from(".")); + + match find_project_root(&start) { + Some(root) => load_docs(&root), + None => { + // Fallback: try current working directory + let cwd = std::env::current_dir().unwrap_or_else(|_| PathBuf::from(".")); + if cwd.join("README.md").is_file() { + return load_docs(&cwd); + } + tracing::error!( + "help_chat: could not locate project root from {}; doc context will be empty", + start.display() + ); + String::new() + } + } + }) +} + +// ── Handler ────────────────────────────────────────────────────────────────── + +/// POST /api/v1/help/chat — Answer questions about the compliance-scanner +/// using the project documentation as grounding context. +#[tracing::instrument(skip_all)] +pub async fn help_chat( + Extension(agent): AgentExt, + Json(req): Json, +) -> Result>, StatusCode> { + let context = doc_context(); + + let system_prompt = if context.is_empty() { + "You are a helpful assistant for the Compliance Scanner project. \ + Answer questions about how to use and configure it. \ + No documentation was loaded at startup, so rely on your general knowledge." + .to_string() + } else { + format!( + "You are a helpful assistant for the Compliance Scanner project. \ + Answer questions about how to use, configure, and understand it \ + using the documentation below as your primary source of truth.\n\n\ + Rules:\n\ + - Prefer information from the provided docs over general knowledge\n\ + - Quote or reference the relevant doc section when it helps\n\ + - If the docs do not cover the topic, say so clearly\n\ + - Be concise — lead with the answer, then explain if needed\n\ + - Use markdown formatting for readability\n\n\ + ## Project Documentation\n\n{context}" + ) + }; + + let mut messages: Vec<(String, String)> = Vec::with_capacity(req.history.len() + 2); + messages.push(("system".to_string(), system_prompt)); + + for msg in &req.history { + messages.push((msg.role.clone(), msg.content.clone())); + } + messages.push(("user".to_string(), req.message)); + + let response_text = agent + .llm + .chat_with_messages(messages, Some(0.3)) + .await + .map_err(|e| { + tracing::error!("LLM help chat failed: {e}"); + StatusCode::INTERNAL_SERVER_ERROR + })?; + + Ok(Json(ApiResponse { + data: HelpChatResponse { + message: response_text, + }, + total: None, + page: None, + })) +} diff --git a/compliance-agent/src/api/handlers/mod.rs b/compliance-agent/src/api/handlers/mod.rs index c860312..7902b26 100644 --- a/compliance-agent/src/api/handlers/mod.rs +++ b/compliance-agent/src/api/handlers/mod.rs @@ -4,6 +4,7 @@ pub mod dto; pub mod findings; pub mod graph; pub mod health; +pub mod help_chat; pub mod issues; pub mod pentest_handlers; pub use pentest_handlers as pentest; diff --git a/compliance-agent/src/api/routes.rs b/compliance-agent/src/api/routes.rs index feeb51f..e2f0ec0 100644 --- a/compliance-agent/src/api/routes.rs +++ b/compliance-agent/src/api/routes.rs @@ -99,6 +99,8 @@ pub fn build_router() -> Router { "/api/v1/chat/{repo_id}/status", get(handlers::chat::embedding_status), ) + // Help chat (documentation-grounded Q&A) + .route("/api/v1/help/chat", post(handlers::help_chat::help_chat)) // Pentest API endpoints .route( "/api/v1/pentest/lookup-repo", diff --git a/compliance-dashboard/assets/main.css b/compliance-dashboard/assets/main.css index 52547e5..4dff597 100644 --- a/compliance-dashboard/assets/main.css +++ b/compliance-dashboard/assets/main.css @@ -3645,3 +3645,205 @@ tbody tr:last-child td { .wizard-toggle.active .wizard-toggle-knob { transform: translateX(16px); } + +/* ═══════════════════════════════════════════════════════════════ + HELP CHAT WIDGET + Floating assistant for documentation Q&A + ═══════════════════════════════════════════════════════════════ */ + +.help-chat-toggle { + position: fixed; + bottom: 24px; + right: 28px; + z-index: 50; + width: 48px; + height: 48px; + border-radius: 50%; + background: var(--accent); + color: var(--bg-primary); + border: none; + cursor: pointer; + display: flex; + align-items: center; + justify-content: center; + box-shadow: 0 4px 20px rgba(0, 200, 255, 0.3); + transition: transform 0.15s, box-shadow 0.15s; +} +.help-chat-toggle:hover { + transform: scale(1.08); + box-shadow: 0 6px 28px rgba(0, 200, 255, 0.4); +} + +.help-chat-panel { + position: fixed; + bottom: 24px; + right: 28px; + z-index: 51; + width: 400px; + height: 520px; + background: var(--bg-secondary); + border: 1px solid var(--border-bright); + border-radius: 16px; + display: flex; + flex-direction: column; + overflow: hidden; + box-shadow: 0 12px 48px rgba(0, 0, 0, 0.5), var(--accent-glow); +} + +.help-chat-header { + display: flex; + align-items: center; + justify-content: space-between; + padding: 14px 18px; + border-bottom: 1px solid var(--border); + background: var(--bg-primary); +} +.help-chat-title { + display: flex; + align-items: center; + gap: 8px; + font-family: 'Outfit', sans-serif; + font-weight: 600; + font-size: 14px; + color: var(--text-primary); +} +.help-chat-close { + background: none; + border: none; + color: var(--text-secondary); + cursor: pointer; + padding: 4px; + border-radius: 6px; + display: flex; +} +.help-chat-close:hover { + color: var(--text-primary); + background: var(--bg-elevated); +} + +.help-chat-messages { + flex: 1; + overflow-y: auto; + padding: 16px; + display: flex; + flex-direction: column; + gap: 12px; +} + +.help-chat-empty { + display: flex; + flex-direction: column; + align-items: center; + justify-content: center; + height: 100%; + text-align: center; + color: var(--text-secondary); + font-size: 13px; + gap: 8px; +} +.help-chat-hint { + font-size: 12px; + color: var(--text-tertiary); + font-style: italic; +} + +.help-msg { + max-width: 88%; + animation: helpMsgIn 0.15s ease-out; +} +@keyframes helpMsgIn { + from { opacity: 0; transform: translateY(6px); } + to { opacity: 1; transform: translateY(0); } +} +.help-msg-user { + align-self: flex-end; +} +.help-msg-assistant { + align-self: flex-start; +} +.help-msg-content { + padding: 10px 14px; + border-radius: 12px; + font-size: 13px; + line-height: 1.55; + word-wrap: break-word; +} +.help-msg-user .help-msg-content { + background: var(--accent); + color: var(--bg-primary); + border-bottom-right-radius: 4px; +} +.help-msg-assistant .help-msg-content { + background: var(--bg-elevated); + color: var(--text-primary); + border: 1px solid var(--border); + border-bottom-left-radius: 4px; +} +.help-msg-assistant .help-msg-content code { + background: rgba(0, 200, 255, 0.1); + padding: 1px 5px; + border-radius: 3px; + font-family: 'JetBrains Mono', monospace; + font-size: 12px; +} +.help-msg-loading { + padding: 10px 14px; + border-radius: 12px; + background: var(--bg-elevated); + border: 1px solid var(--border); + border-bottom-left-radius: 4px; + color: var(--text-secondary); + font-size: 13px; + animation: helpPulse 1.2s ease-in-out infinite; +} +@keyframes helpPulse { + 0%, 100% { opacity: 0.6; } + 50% { opacity: 1; } +} + +.help-chat-input { + display: flex; + align-items: center; + gap: 8px; + padding: 12px 14px; + border-top: 1px solid var(--border); + background: var(--bg-primary); +} +.help-chat-input input { + flex: 1; + background: var(--bg-elevated); + border: 1px solid var(--border); + border-radius: 8px; + padding: 10px 14px; + color: var(--text-primary); + font-size: 13px; + font-family: 'DM Sans', sans-serif; + outline: none; + transition: border-color 0.15s; +} +.help-chat-input input:focus { + border-color: var(--accent); +} +.help-chat-input input::placeholder { + color: var(--text-tertiary); +} +.help-chat-send { + width: 36px; + height: 36px; + border-radius: 8px; + background: var(--accent); + color: var(--bg-primary); + border: none; + cursor: pointer; + display: flex; + align-items: center; + justify-content: center; + transition: opacity 0.15s; +} +.help-chat-send:disabled { + opacity: 0.4; + cursor: not-allowed; +} +.help-chat-send:not(:disabled):hover { + background: var(--accent-hover); +} diff --git a/compliance-dashboard/src/app.rs b/compliance-dashboard/src/app.rs index eb850ea..cb276b5 100644 --- a/compliance-dashboard/src/app.rs +++ b/compliance-dashboard/src/app.rs @@ -44,8 +44,6 @@ pub enum Route { PentestSessionPage { session_id: String }, #[route("/mcp-servers")] McpServersPage {}, - #[route("/settings")] - SettingsPage {}, } const FAVICON: Asset = asset!("/assets/favicon.svg"); diff --git a/compliance-dashboard/src/components/app_shell.rs b/compliance-dashboard/src/components/app_shell.rs index f4d8893..c5c3c43 100644 --- a/compliance-dashboard/src/components/app_shell.rs +++ b/compliance-dashboard/src/components/app_shell.rs @@ -1,6 +1,7 @@ use dioxus::prelude::*; use crate::app::Route; +use crate::components::help_chat::HelpChat; use crate::components::sidebar::Sidebar; use crate::components::toast::{ToastContainer, Toasts}; use crate::infrastructure::auth_check::check_auth; @@ -21,6 +22,7 @@ pub fn AppShell() -> Element { Outlet:: {} } ToastContainer {} + HelpChat {} } } } diff --git a/compliance-dashboard/src/components/help_chat.rs b/compliance-dashboard/src/components/help_chat.rs new file mode 100644 index 0000000..c79f912 --- /dev/null +++ b/compliance-dashboard/src/components/help_chat.rs @@ -0,0 +1,198 @@ +use dioxus::prelude::*; +use dioxus_free_icons::icons::bs_icons::*; +use dioxus_free_icons::Icon; + +use crate::infrastructure::help_chat::{send_help_chat_message, HelpChatHistoryMessage}; + +// ── Message model ──────────────────────────────────────────────────────────── + +#[derive(Clone, Debug)] +struct ChatMsg { + role: String, + content: String, +} + +// ── Component ──────────────────────────────────────────────────────────────── + +#[component] +pub fn HelpChat() -> Element { + let mut is_open = use_signal(|| false); + let mut messages = use_signal(Vec::::new); + let mut input_text = use_signal(String::new); + let mut is_loading = use_signal(|| false); + + // Send message handler + let on_send = move |_| { + let text = input_text().trim().to_string(); + if text.is_empty() || is_loading() { + return; + } + + // Push user message + messages.write().push(ChatMsg { + role: "user".into(), + content: text.clone(), + }); + input_text.set(String::new()); + is_loading.set(true); + + // Build history for API call (exclude last user message, it goes as `message`) + let history: Vec = messages() + .iter() + .rev() + .skip(1) // skip the user message we just added + .rev() + .map(|m| HelpChatHistoryMessage { + role: m.role.clone(), + content: m.content.clone(), + }) + .collect(); + + spawn(async move { + match send_help_chat_message(text, history).await { + Ok(resp) => { + messages.write().push(ChatMsg { + role: "assistant".into(), + content: resp.data.message, + }); + } + Err(e) => { + messages.write().push(ChatMsg { + role: "assistant".into(), + content: format!("Error: {e}"), + }); + } + } + is_loading.set(false); + }); + }; + + // Key handler for Enter to send + let on_keydown = move |e: KeyboardEvent| { + if e.key() == Key::Enter && !e.modifiers().shift() { + e.prevent_default(); + let text = input_text().trim().to_string(); + if text.is_empty() || is_loading() { + return; + } + messages.write().push(ChatMsg { + role: "user".into(), + content: text.clone(), + }); + input_text.set(String::new()); + is_loading.set(true); + + let history: Vec = messages() + .iter() + .rev() + .skip(1) + .rev() + .map(|m| HelpChatHistoryMessage { + role: m.role.clone(), + content: m.content.clone(), + }) + .collect(); + + spawn(async move { + match send_help_chat_message(text, history).await { + Ok(resp) => { + messages.write().push(ChatMsg { + role: "assistant".into(), + content: resp.data.message, + }); + } + Err(e) => { + messages.write().push(ChatMsg { + role: "assistant".into(), + content: format!("Error: {e}"), + }); + } + } + is_loading.set(false); + }); + } + }; + + rsx! { + // Floating toggle button + if !is_open() { + button { + class: "help-chat-toggle", + onclick: move |_| is_open.set(true), + title: "Help", + Icon { icon: BsQuestionCircle, width: 22, height: 22 } + } + } + + // Chat panel + if is_open() { + div { class: "help-chat-panel", + // Header + div { class: "help-chat-header", + span { class: "help-chat-title", + Icon { icon: BsRobot, width: 16, height: 16 } + "Help Assistant" + } + button { + class: "help-chat-close", + onclick: move |_| is_open.set(false), + Icon { icon: BsX, width: 18, height: 18 } + } + } + + // Messages area + div { class: "help-chat-messages", + if messages().is_empty() { + div { class: "help-chat-empty", + p { "Ask me anything about the Compliance Scanner." } + p { class: "help-chat-hint", + "e.g. \"How do I add a repository?\" or \"What is SBOM?\"" + } + } + } + for (i, msg) in messages().iter().enumerate() { + div { + key: "{i}", + class: if msg.role == "user" { "help-msg help-msg-user" } else { "help-msg help-msg-assistant" }, + div { class: "help-msg-content", + dangerous_inner_html: if msg.role == "assistant" { + // Basic markdown rendering: bold, code, newlines + msg.content + .replace("**", "") + .replace("\n\n", "

") + .replace("\n- ", "
- ") + .replace("`", "") + } else { + msg.content.clone() + } + } + } + } + if is_loading() { + div { class: "help-msg help-msg-assistant", + div { class: "help-msg-loading", "Thinking..." } + } + } + } + + // Input area + div { class: "help-chat-input", + input { + r#type: "text", + placeholder: "Ask a question...", + value: "{input_text}", + disabled: is_loading(), + oninput: move |e| input_text.set(e.value()), + onkeydown: on_keydown, + } + button { + class: "help-chat-send", + disabled: is_loading() || input_text().trim().is_empty(), + onclick: on_send, + Icon { icon: BsSend, width: 14, height: 14 } + } + } + } + } + } +} diff --git a/compliance-dashboard/src/components/mod.rs b/compliance-dashboard/src/components/mod.rs index db96afa..594e07e 100644 --- a/compliance-dashboard/src/components/mod.rs +++ b/compliance-dashboard/src/components/mod.rs @@ -3,6 +3,7 @@ pub mod attack_chain; pub mod code_inspector; pub mod code_snippet; pub mod file_tree; +pub mod help_chat; pub mod page_header; pub mod pagination; pub mod pentest_wizard; diff --git a/compliance-dashboard/src/components/sidebar.rs b/compliance-dashboard/src/components/sidebar.rs index b8a3177..227ccaa 100644 --- a/compliance-dashboard/src/components/sidebar.rs +++ b/compliance-dashboard/src/components/sidebar.rs @@ -52,11 +52,6 @@ pub fn Sidebar() -> Element { route: Route::PentestDashboardPage {}, icon: rsx! { Icon { icon: BsLightningCharge, width: 18, height: 18 } }, }, - NavItem { - label: "Settings", - route: Route::SettingsPage {}, - icon: rsx! { Icon { icon: BsGear, width: 18, height: 18 } }, - }, ]; let docs_url = option_env!("DOCS_URL").unwrap_or("/docs"); diff --git a/compliance-dashboard/src/infrastructure/help_chat.rs b/compliance-dashboard/src/infrastructure/help_chat.rs new file mode 100644 index 0000000..6286854 --- /dev/null +++ b/compliance-dashboard/src/infrastructure/help_chat.rs @@ -0,0 +1,59 @@ +use dioxus::prelude::*; +use serde::{Deserialize, Serialize}; + +// ── Response types ── + +#[derive(Debug, Clone, Serialize, Deserialize, Default)] +pub struct HelpChatApiResponse { + pub data: HelpChatResponseData, +} + +#[derive(Debug, Clone, Serialize, Deserialize, Default)] +pub struct HelpChatResponseData { + pub message: String, +} + +// ── History message type ── + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct HelpChatHistoryMessage { + pub role: String, + pub content: String, +} + +// ── Server function ── + +#[server] +pub async fn send_help_chat_message( + message: String, + history: Vec, +) -> Result { + let state: super::server_state::ServerState = + dioxus_fullstack::FullstackContext::extract().await?; + + let url = format!("{}/api/v1/help/chat", state.agent_api_url); + let client = reqwest::Client::builder() + .timeout(std::time::Duration::from_secs(120)) + .build() + .map_err(|e| ServerFnError::new(e.to_string()))?; + + let resp = client + .post(&url) + .json(&serde_json::json!({ + "message": message, + "history": history, + })) + .send() + .await + .map_err(|e| ServerFnError::new(format!("Help chat request failed: {e}")))?; + + let text = resp + .text() + .await + .map_err(|e| ServerFnError::new(format!("Failed to read response: {e}")))?; + + let body: HelpChatApiResponse = serde_json::from_str(&text) + .map_err(|e| ServerFnError::new(format!("Failed to parse response: {e}")))?; + + Ok(body) +} diff --git a/compliance-dashboard/src/infrastructure/mod.rs b/compliance-dashboard/src/infrastructure/mod.rs index 490c63b..84ceb89 100644 --- a/compliance-dashboard/src/infrastructure/mod.rs +++ b/compliance-dashboard/src/infrastructure/mod.rs @@ -5,6 +5,7 @@ pub mod chat; pub mod dast; pub mod findings; pub mod graph; +pub mod help_chat; pub mod issues; pub mod mcp; pub mod pentest; diff --git a/compliance-dashboard/src/pages/mod.rs b/compliance-dashboard/src/pages/mod.rs index bdc9281..c54217a 100644 --- a/compliance-dashboard/src/pages/mod.rs +++ b/compliance-dashboard/src/pages/mod.rs @@ -16,7 +16,6 @@ pub mod pentest_dashboard; pub mod pentest_session; pub mod repositories; pub mod sbom; -pub mod settings; pub use chat::ChatPage; pub use chat_index::ChatIndexPage; @@ -36,4 +35,3 @@ pub use pentest_dashboard::PentestDashboardPage; pub use pentest_session::PentestSessionPage; pub use repositories::RepositoriesPage; pub use sbom::SbomPage; -pub use settings::SettingsPage; diff --git a/compliance-dashboard/src/pages/settings.rs b/compliance-dashboard/src/pages/settings.rs deleted file mode 100644 index 0f41fee..0000000 --- a/compliance-dashboard/src/pages/settings.rs +++ /dev/null @@ -1,142 +0,0 @@ -use dioxus::prelude::*; - -use crate::components::page_header::PageHeader; - -#[component] -pub fn SettingsPage() -> Element { - let mut litellm_url = use_signal(|| "http://localhost:4000".to_string()); - let mut litellm_model = use_signal(|| "gpt-4o".to_string()); - let mut github_token = use_signal(String::new); - let mut gitlab_url = use_signal(|| "https://gitlab.com".to_string()); - let mut gitlab_token = use_signal(String::new); - let mut jira_url = use_signal(String::new); - let mut jira_email = use_signal(String::new); - let mut jira_token = use_signal(String::new); - let mut jira_project = use_signal(String::new); - let mut searxng_url = use_signal(|| "http://localhost:8888".to_string()); - - rsx! { - PageHeader { - title: "Settings", - description: "Configure integrations and scanning parameters", - } - - div { class: "card", - div { class: "card-header", "LiteLLM Configuration" } - div { class: "form-group", - label { "LiteLLM URL" } - input { - r#type: "text", - value: "{litellm_url}", - oninput: move |e| litellm_url.set(e.value()), - } - } - div { class: "form-group", - label { "Model" } - input { - r#type: "text", - value: "{litellm_model}", - oninput: move |e| litellm_model.set(e.value()), - } - } - } - - div { class: "card", - div { class: "card-header", "GitHub Integration" } - div { class: "form-group", - label { "Personal Access Token" } - input { - r#type: "password", - placeholder: "ghp_...", - value: "{github_token}", - oninput: move |e| github_token.set(e.value()), - } - } - } - - div { class: "card", - div { class: "card-header", "GitLab Integration" } - div { class: "form-group", - label { "GitLab URL" } - input { - r#type: "text", - value: "{gitlab_url}", - oninput: move |e| gitlab_url.set(e.value()), - } - } - div { class: "form-group", - label { "Access Token" } - input { - r#type: "password", - placeholder: "glpat-...", - value: "{gitlab_token}", - oninput: move |e| gitlab_token.set(e.value()), - } - } - } - - div { class: "card", - div { class: "card-header", "Jira Integration" } - div { class: "form-group", - label { "Jira URL" } - input { - r#type: "text", - placeholder: "https://your-org.atlassian.net", - value: "{jira_url}", - oninput: move |e| jira_url.set(e.value()), - } - } - div { class: "form-group", - label { "Email" } - input { - r#type: "email", - value: "{jira_email}", - oninput: move |e| jira_email.set(e.value()), - } - } - div { class: "form-group", - label { "API Token" } - input { - r#type: "password", - value: "{jira_token}", - oninput: move |e| jira_token.set(e.value()), - } - } - div { class: "form-group", - label { "Project Key" } - input { - r#type: "text", - placeholder: "SEC", - value: "{jira_project}", - oninput: move |e| jira_project.set(e.value()), - } - } - } - - div { class: "card", - div { class: "card-header", "SearXNG" } - div { class: "form-group", - label { "SearXNG URL" } - input { - r#type: "text", - value: "{searxng_url}", - oninput: move |e| searxng_url.set(e.value()), - } - } - } - - div { style: "margin-top: 16px;", - button { - class: "btn btn-primary", - onclick: move |_| { - tracing::info!("Settings save not yet implemented - settings are managed via .env"); - }, - "Save Settings" - } - p { - style: "margin-top: 8px; font-size: 12px; color: var(--text-secondary);", - "Note: Settings are currently configured via environment variables (.env file). Dashboard-based settings persistence coming soon." - } - } - } -} diff --git a/docs/features/deduplication.md b/docs/features/deduplication.md new file mode 100644 index 0000000..5dcf9ae --- /dev/null +++ b/docs/features/deduplication.md @@ -0,0 +1,61 @@ +# Finding Deduplication + +The Compliance Scanner automatically deduplicates findings across all scanning surfaces to prevent noise and duplicate issues. + +## SAST Finding Dedup + +Static analysis findings are deduplicated using SHA-256 fingerprints computed from: + +- Repository ID +- Scanner rule ID (e.g., Semgrep check ID) +- File path +- Line number + +Before inserting a new finding, the pipeline checks if a finding with the same fingerprint already exists. If it does, the finding is skipped. + +## DAST / Pentest Finding Dedup + +Dynamic testing findings go through two-phase deduplication: + +### Phase 1: Exact Dedup + +Findings with the same canonicalized title, endpoint, and HTTP method are merged. Evidence from duplicate findings is combined into a single finding, keeping the highest severity. + +**Title canonicalization** handles common variations: +- Domain names and URLs are stripped from titles (e.g., "Missing HSTS header for example.com" becomes "Missing HSTS header") +- Known synonyms are resolved (e.g., "HSTS" maps to "strict-transport-security", "CSP" maps to "content-security-policy") + +### Phase 2: CWE-Based Dedup + +After exact dedup, findings with the same CWE and endpoint are merged. This catches cases where different tools report the same underlying issue with different titles or vulnerability types (e.g., a missing HSTS header reported as both `security_header_missing` and `tls_misconfiguration`). + +The primary finding is selected by highest severity, then most evidence, then longest description. Evidence from merged findings is preserved. + +### When Dedup Applies + +- **At insertion time**: During a pentest session, before each finding is stored in MongoDB +- **At report export**: When generating a pentest report, all session findings are deduplicated before rendering + +## PR Review Comment Dedup + +PR review comments are deduplicated to prevent posting the same finding multiple times: + +- Each comment includes a fingerprint computed from the repository, PR number, file path, line, and finding title +- Within a single review run, duplicate findings are skipped +- The fingerprint is embedded as an HTML comment in the review body for future cross-run dedup + +## Issue Tracker Dedup + +Before creating an issue in GitHub, GitLab, Jira, or Gitea, the scanner: + +1. Searches for an existing issue matching the finding's fingerprint +2. Falls back to searching by issue title +3. Skips creation if a match is found + +## Code Review Dedup + +Multi-pass LLM code reviews (logic, security, convention, complexity) are deduplicated across passes using proximity-aware keys: + +- Findings within 3 lines of each other on the same file with similar normalized titles are considered duplicates +- The finding with the highest severity is kept +- CWE information is merged from duplicates diff --git a/docs/features/help-chat.md b/docs/features/help-chat.md new file mode 100644 index 0000000..e279515 --- /dev/null +++ b/docs/features/help-chat.md @@ -0,0 +1,60 @@ +# Help Chat Assistant + +The Help Chat is a floating assistant available on every page of the dashboard. It answers questions about the Compliance Scanner using the project documentation as its knowledge base. + +## How It Works + +1. Click the **?** button in the bottom-right corner of any page +2. Type your question and press Enter +3. The assistant responds with answers grounded in the project documentation + +The chat supports multi-turn conversations -- you can ask follow-up questions and the assistant will remember the context of your conversation. + +## What You Can Ask + +- **Getting started**: "How do I add a repository?" / "How do I trigger a scan?" +- **Features**: "What is SBOM?" / "How does the code knowledge graph work?" +- **Configuration**: "How do I set up webhooks?" / "What environment variables are needed?" +- **Scanning**: "What does the scan pipeline do?" / "How does LLM triage work?" +- **DAST & Pentesting**: "How do I run a pentest?" / "What DAST tools are available?" +- **Integrations**: "How do I connect to GitHub?" / "What is MCP?" + +## Technical Details + +The help chat loads all project documentation (README, guides, feature docs, reference) at startup and caches them in memory. When you ask a question, it sends your message along with the full documentation context to the LLM via LiteLLM, which generates a grounded response. + +### API Endpoint + +``` +POST /api/v1/help/chat +Content-Type: application/json + +{ + "message": "How do I add a repository?", + "history": [ + { "role": "user", "content": "previous question" }, + { "role": "assistant", "content": "previous answer" } + ] +} +``` + +### Configuration + +The help chat uses the same LiteLLM configuration as other LLM features: + +| Environment Variable | Description | Default | +|---------------------|-------------|---------| +| `LITELLM_URL` | LiteLLM API base URL | `http://localhost:4000` | +| `LITELLM_MODEL` | Model for chat responses | `gpt-4o` | +| `LITELLM_API_KEY` | API key (optional) | -- | + +### Documentation Sources + +The assistant indexes the following documentation at startup: + +- `README.md` -- Project overview and quick start +- `docs/guide/` -- Getting started, repositories, findings, SBOM, scanning, issues, webhooks +- `docs/features/` -- AI Chat, DAST, Code Graph, MCP Server, Pentesting, Help Chat +- `docs/reference/` -- Glossary, tools reference + +If documentation files are not found at startup (e.g., in a minimal Docker deployment), the assistant falls back to general knowledge about the project. diff --git a/docs/features/overview.md b/docs/features/overview.md index 8aef19e..37a9e4b 100644 --- a/docs/features/overview.md +++ b/docs/features/overview.md @@ -1,8 +1,6 @@ # Dashboard Overview -The Overview page is the landing page of Certifai. It gives you a high-level view of your security posture across all tracked repositories. - -![Dashboard overview with stats cards, severity distribution, AI chat, and MCP servers](/screenshots/dashboard-overview.png) +The Overview page is the landing page of the Compliance Scanner. It gives you a high-level view of your security posture across all tracked repositories. ## Stats Cards @@ -34,6 +32,10 @@ The overview includes quick-access cards for the AI Chat feature. Each card repr If you have MCP servers registered, they appear on the overview page with their status and connection details. This lets you quickly check that your MCP integrations are running. See [MCP Integration](/features/mcp-server) for details. +## Help Chat Assistant + +A floating help chat button is available in the bottom-right corner of every page. Click it to ask questions about the Compliance Scanner -- how to configure repositories, understand findings, set up webhooks, or use any feature. The assistant is grounded in the project documentation and uses LiteLLM for responses. + ## Recent Scan Runs The bottom section lists the most recent scan runs across all repositories, showing: