5 Commits

Author SHA1 Message Date
Sharang Parnerkar
681201ff45 fix: update lz4_flex 0.11.5 → 0.11.6 (RUSTSEC-2026-0041)
All checks were successful
CI / Check (pull_request) Successful in 10m54s
CI / Detect Changes (pull_request) Has been skipped
CI / Deploy Agent (pull_request) Has been skipped
CI / Deploy MCP (pull_request) Has been skipped
CI / Deploy Dashboard (pull_request) Has been skipped
CI / Deploy Docs (pull_request) Has been skipped
Fixes high-severity advisory: decompressing invalid data can leak
uninitialized memory. Transitive dep via tantivy → compliance-graph.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 21:21:05 +01:00
Sharang Parnerkar
0236cad536 test: add 29 new tests for cleanup, orchestrator, findings, tool registry, models
Some checks failed
CI / Check (pull_request) Failing after 6m3s
CI / Detect Changes (pull_request) Has been skipped
CI / Deploy Agent (pull_request) Has been skipped
CI / Deploy Dashboard (pull_request) Has been skipped
CI / Deploy Docs (pull_request) Has been skipped
CI / Deploy MCP (pull_request) Has been skipped
- cleanup.rs: 8 tests — routing logic, skip conditions, missing config errors
- orchestrator.rs: 7 tests — summarize_tool_output (screenshot strip, truncation, recursion)
- findings.rs: 6 tests — empty state, severity grouping, SAST correlation, evidence table
- tools/mod.rs: 4 tests — registry completeness, schema validation, browser action enum
- models.rs: 4 tests — TestUserRecord serde, IdentityProvider variants, BSON roundtrip

Total: 326 tests (was 297)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 21:13:18 +01:00
Sharang Parnerkar
37690ce734 feat: browser session persistence, auto-screenshots, context optimization, user cleanup
Some checks failed
CI / Check (pull_request) Failing after 5m55s
CI / Detect Changes (pull_request) Has been skipped
CI / Deploy Agent (pull_request) Has been skipped
CI / Deploy Dashboard (pull_request) Has been skipped
CI / Deploy Docs (pull_request) Has been skipped
CI / Deploy MCP (pull_request) Has been skipped
Browser tool:
- Session-persistent Chrome tab (same tab reused across all calls in a pentest)
- Auto-screenshot on every navigate and click (stored in attack chain for report)
- Fill uses CDP Input.insertText (fixes WebSocket corruption on special chars)
- Switched from browserless/chromium to chromedp/headless-shell (stable WS)

Context window optimization:
- Strip screenshot_base64 from LLM conversation (kept in DB for report)
- Truncate HTML to 2KB, page text to 1.5KB in LLM messages
- Cap element/link arrays at 15 items
- SAST triage: batch 30 findings per LLM call instead of all at once

Report improvements:
- Auto-embed screenshots in attack chain timeline (navigate + click nodes)
- Cover page shows best app screenshot
- Attack chain phases capped at 8 (no more 20x "Final")

User cleanup:
- TestUserRecord model tracks created test users per session
- cleanup.rs: Keycloak (Admin REST API), Auth0 (Management API), Okta (Users API)
- Auto-cleanup on session completion when cleanup_test_user is enabled
- Env vars: KEYCLOAK_ADMIN_USERNAME, KEYCLOAK_ADMIN_PASSWORD

System prompt:
- Explicit browser usage instructions (navigate → get_content → click → fill)
- SPA auth bypass guidance (check page content, not HTTP status)
- Screenshot instructions for evidence collection

Other:
- Pin mongo:7 in docker-compose (mongo:latest/8 segfaults on kernel 6.19)
- Add deploy/docker-compose.mailserver.yml for Postfix + Dovecot

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 19:53:55 +01:00
Sharang Parnerkar
a737c36bc9 fix: add pentest architecture to sidebar, fix image paths, enable mermaid
- Add vitepress-plugin-mermaid for diagram rendering
- Add Pentest Architecture page to sidebar nav
- Fix image paths (../public/ → / for VitePress public serving)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 00:16:53 +01:00
Sharang Parnerkar
a912ec9ad9 feat: pentest feature improvements — streaming, pause/resume, encryption, browser tool, reports, docs
- True SSE streaming via broadcast channels (DashMap per session)
- Session pause/resume with watch channels + dashboard buttons
- AES-256-GCM credential encryption at rest (PENTEST_ENCRYPTION_KEY)
- Concurrency limiter (Semaphore, max 5 sessions, 429 on overflow)
- Browser tool: headless Chrome CDP automation (navigate, click, fill, screenshot, evaluate)
- Report code-level correlation: SAST findings, code graph, SBOM linked per DAST finding
- Split html.rs (1919 LOC) into html/ module directory (8 files)
- Wizard: target/repo dropdowns from existing data, SSH key display, close button on all steps
- Auth: auto-register with optional registration URL (Playwright discovery), plus-addressing email, IMAP overrides
- Attack chain: tool input/output in detail panel, running node pulse animation
- Architecture docs with Mermaid diagrams + 8 screenshots

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 00:07:50 +01:00
12 changed files with 31 additions and 216 deletions

6
Cargo.lock generated
View File

@@ -4699,9 +4699,9 @@ dependencies = [
[[package]]
name = "rustls-webpki"
version = "0.103.10"
version = "0.103.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df33b2b81ac578cabaf06b89b0631153a3f416b0a886e8a7a1707fb51abbd1ef"
checksum = "d7df23109aa6c1567d1c575b9952556388da57401e4ace1d15f79eedad0d8f53"
dependencies = [
"ring",
"rustls-pki-types",
@@ -5171,7 +5171,7 @@ version = "0.8.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c1c97747dbf44bb1ca44a561ece23508e99cb592e862f22222dcf42f51d1e451"
dependencies = [
"heck 0.4.1",
"heck 0.5.0",
"proc-macro2",
"quote",
"syn",

View File

@@ -54,9 +54,6 @@ pub fn load_config() -> Result<AgentConfig, AgentError> {
pentest_verification_email: env_var_opt("PENTEST_VERIFICATION_EMAIL"),
pentest_imap_host: env_var_opt("PENTEST_IMAP_HOST"),
pentest_imap_port: env_var_opt("PENTEST_IMAP_PORT").and_then(|p| p.parse().ok()),
pentest_imap_tls: env_var_opt("PENTEST_IMAP_TLS")
.map(|v| v == "1" || v.eq_ignore_ascii_case("true"))
.unwrap_or(true),
pentest_imap_username: env_var_opt("PENTEST_IMAP_USERNAME"),
pentest_imap_password: env_secret_opt("PENTEST_IMAP_PASSWORD"),
})

View File

@@ -336,7 +336,6 @@ mod tests {
pentest_verification_email: None,
pentest_imap_host: None,
pentest_imap_port: None,
pentest_imap_tls: true,
pentest_imap_username: None,
pentest_imap_password: None,
}

View File

@@ -66,10 +66,8 @@ impl CodeReviewScanner {
}
}
let deduped = dedup_cross_pass(all_findings);
ScanOutput {
findings: deduped,
findings: all_findings,
sbom_entries: Vec::new(),
}
}
@@ -186,51 +184,3 @@ struct ReviewIssue {
#[serde(default)]
suggestion: Option<String>,
}
/// Deduplicate findings across review passes.
///
/// Multiple passes often flag the same issue (e.g. SQL injection reported by
/// logic, security, and convention passes). We group by file + nearby line +
/// normalized title keywords and keep the highest-severity finding.
fn dedup_cross_pass(findings: Vec<Finding>) -> Vec<Finding> {
use std::collections::HashMap;
// Build a dedup key: (file, line bucket, normalized title words)
fn dedup_key(f: &Finding) -> String {
let file = f.file_path.as_deref().unwrap_or("");
// Group lines within 3 of each other
let line_bucket = f.line_number.unwrap_or(0) / 4;
// Normalize: lowercase, keep only alphanumeric, sort words for order-independence
let title_lower = f.title.to_lowercase();
let mut words: Vec<&str> = title_lower
.split(|c: char| !c.is_alphanumeric())
.filter(|w| w.len() > 2)
.collect();
words.sort();
format!("{file}:{line_bucket}:{}", words.join(","))
}
let mut groups: HashMap<String, Finding> = HashMap::new();
for finding in findings {
let key = dedup_key(&finding);
groups
.entry(key)
.and_modify(|existing| {
// Keep the higher severity; on tie, keep the one with more detail
if finding.severity > existing.severity
|| (finding.severity == existing.severity
&& finding.description.len() > existing.description.len())
{
*existing = finding.clone();
}
// Merge CWE if the existing one is missing it
if existing.cwe.is_none() {
existing.cwe = finding.cwe.clone();
}
})
.or_insert(finding);
}
groups.into_values().collect()
}

View File

@@ -10,6 +10,7 @@ use compliance_core::AgentConfig;
use crate::database::Database;
use crate::error::AgentError;
use crate::llm::LlmClient;
use crate::pipeline::code_review::CodeReviewScanner;
use crate::pipeline::cve::CveScanner;
use crate::pipeline::git::GitOps;
use crate::pipeline::gitleaks::GitleaksScanner;
@@ -240,6 +241,21 @@ impl PipelineOrchestrator {
Err(e) => tracing::warn!("[{repo_id}] Lint scanning failed: {e}"),
}
// Stage 4c: LLM Code Review (only on incremental scans)
if let Some(old_sha) = &repo.last_scanned_commit {
tracing::info!("[{repo_id}] Stage 4c: LLM Code Review");
self.update_phase(scan_run_id, "code_review").await;
let review_output = async {
let reviewer = CodeReviewScanner::new(self.llm.clone());
reviewer
.review_diff(&repo_path, &repo_id, old_sha, &current_sha)
.await
}
.instrument(tracing::info_span!("stage_code_review"))
.await;
all_findings.extend(review_output.findings);
}
// Stage 4.5: Graph Building
tracing::info!("[{repo_id}] Stage 4.5: Graph Building");
self.update_phase(scan_run_id, "graph_building").await;

View File

@@ -98,8 +98,7 @@ impl IssueTracker for GiteaTracker {
_ => "open",
};
let resp = self
.http
self.http
.patch(&url)
.header(
"Authorization",
@@ -110,14 +109,6 @@ impl IssueTracker for GiteaTracker {
.await
.map_err(|e| CoreError::IssueTracker(format!("Gitea update issue failed: {e}")))?;
if !resp.status().is_success() {
let status = resp.status();
let text = resp.text().await.unwrap_or_default();
return Err(CoreError::IssueTracker(format!(
"Gitea update issue returned {status}: {text}"
)));
}
Ok(())
}
@@ -132,8 +123,7 @@ impl IssueTracker for GiteaTracker {
"/repos/{owner}/{repo}/issues/{external_id}/comments"
));
let resp = self
.http
self.http
.post(&url)
.header(
"Authorization",
@@ -144,14 +134,6 @@ impl IssueTracker for GiteaTracker {
.await
.map_err(|e| CoreError::IssueTracker(format!("Gitea add comment failed: {e}")))?;
if !resp.status().is_success() {
let status = resp.status();
let text = resp.text().await.unwrap_or_default();
return Err(CoreError::IssueTracker(format!(
"Gitea add comment returned {status}: {text}"
)));
}
Ok(())
}
@@ -176,8 +158,7 @@ impl IssueTracker for GiteaTracker {
})
.collect();
let resp = self
.http
self.http
.post(&url)
.header(
"Authorization",
@@ -192,48 +173,6 @@ impl IssueTracker for GiteaTracker {
.await
.map_err(|e| CoreError::IssueTracker(format!("Gitea PR review failed: {e}")))?;
if !resp.status().is_success() {
let status = resp.status();
let text = resp.text().await.unwrap_or_default();
// If inline comments caused the failure, retry with just the summary body
if !comments.is_empty() {
tracing::warn!(
"Gitea PR review with inline comments failed ({status}): {text}, retrying as plain comment"
);
let fallback_url = self.api_url(&format!(
"/repos/{owner}/{repo}/issues/{pr_number}/comments"
));
let fallback_resp = self
.http
.post(&fallback_url)
.header(
"Authorization",
format!("token {}", self.token.expose_secret()),
)
.json(&serde_json::json!({ "body": body }))
.send()
.await
.map_err(|e| {
CoreError::IssueTracker(format!("Gitea PR comment fallback failed: {e}"))
})?;
if !fallback_resp.status().is_success() {
let fb_status = fallback_resp.status();
let fb_text = fallback_resp.text().await.unwrap_or_default();
return Err(CoreError::IssueTracker(format!(
"Gitea PR comment fallback returned {fb_status}: {fb_text}"
)));
}
return Ok(());
}
return Err(CoreError::IssueTracker(format!(
"Gitea PR review returned {status}: {text}"
)));
}
Ok(())
}

View File

@@ -33,8 +33,6 @@ pub struct AgentConfig {
pub pentest_verification_email: Option<String>,
pub pentest_imap_host: Option<String>,
pub pentest_imap_port: Option<u16>,
/// Use implicit TLS (IMAPS, port 993) instead of plain IMAP.
pub pentest_imap_tls: bool,
pub pentest_imap_username: Option<String>,
pub pentest_imap_password: Option<SecretString>,
}

View File

@@ -113,72 +113,6 @@ pub async fn add_mcp_server(
Ok(())
}
/// Probe each MCP server's health endpoint and update status in MongoDB.
#[server]
pub async fn refresh_mcp_status() -> Result<(), ServerFnError> {
use chrono::Utc;
use compliance_core::models::McpServerStatus;
use mongodb::bson::doc;
let state: super::server_state::ServerState =
dioxus_fullstack::FullstackContext::extract().await?;
let mut cursor = state
.db
.mcp_servers()
.find(doc! {})
.await
.map_err(|e| ServerFnError::new(e.to_string()))?;
let client = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(5))
.build()
.map_err(|e| ServerFnError::new(e.to_string()))?;
while cursor
.advance()
.await
.map_err(|e| ServerFnError::new(e.to_string()))?
{
let server: compliance_core::models::McpServerConfig = cursor
.deserialize_current()
.map_err(|e| ServerFnError::new(e.to_string()))?;
let Some(oid) = server.id else { continue };
// Derive health URL from the endpoint (replace trailing /mcp with /health)
let health_url = if server.endpoint_url.ends_with("/mcp") {
format!(
"{}health",
&server.endpoint_url[..server.endpoint_url.len() - 3]
)
} else {
format!("{}/health", server.endpoint_url.trim_end_matches('/'))
};
let new_status = match client.get(&health_url).send().await {
Ok(resp) if resp.status().is_success() => McpServerStatus::Running,
_ => McpServerStatus::Stopped,
};
let status_bson = match bson::to_bson(&new_status) {
Ok(b) => b,
Err(_) => continue,
};
let _ = state
.db
.mcp_servers()
.update_one(
doc! { "_id": oid },
doc! { "$set": { "status": status_bson, "updated_at": Utc::now().to_rfc3339() } },
)
.await;
}
Ok(())
}
#[server]
pub async fn delete_mcp_server(server_id: String) -> Result<(), ServerFnError> {
use mongodb::bson::doc;

View File

@@ -123,6 +123,7 @@ pub fn FindingsPage() -> Element {
option { value: "oauth", "OAuth" }
option { value: "secret_detection", "Secrets" }
option { value: "lint", "Lint" }
option { value: "code_review", "Code Review" }
}
select {
onchange: move |e| { status_filter.set(e.value()); page.set(1); },

View File

@@ -5,7 +5,7 @@ use dioxus_free_icons::Icon;
use crate::components::page_header::PageHeader;
use crate::components::toast::{ToastType, Toasts};
use crate::infrastructure::mcp::{
add_mcp_server, delete_mcp_server, fetch_mcp_servers, refresh_mcp_status, regenerate_mcp_token,
add_mcp_server, delete_mcp_server, fetch_mcp_servers, regenerate_mcp_token,
};
#[component]
@@ -22,17 +22,6 @@ pub fn McpServersPage() -> Element {
let mut new_mongo_uri = use_signal(String::new);
let mut new_mongo_db = use_signal(String::new);
// Probe health of all MCP servers on page load, then refresh the list
let mut refreshing = use_signal(|| true);
use_effect(move || {
spawn(async move {
refreshing.set(true);
let _ = refresh_mcp_status().await;
servers.restart();
refreshing.set(false);
});
});
// Track which server's token is visible
let mut visible_token: Signal<Option<String>> = use_signal(|| None);
// Track which server is pending delete confirmation

View File

@@ -41,9 +41,7 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
StreamableHttpServerConfig::default(),
);
let router = axum::Router::new()
.route("/health", axum::routing::get(|| async { "ok" }))
.nest_service("/mcp", service);
let router = axum::Router::new().nest_service("/mcp", service);
let listener = tokio::net::TcpListener::bind(("0.0.0.0", port)).await?;
tracing::info!("MCP HTTP server listening on 0.0.0.0:{port}");
axum::serve(listener, router).await?;

View File

@@ -8,14 +8,14 @@ services:
container_name: mailserver
ports:
- "25:25" # SMTP (inbound mail)
- "993:993" # IMAPS (TLS-only)
- "587:587" # Submission (STARTTLS)
- "143:143" # IMAP (orchestrator reads mail)
- "993:993" # IMAPS (TLS)
- "587:587" # Submission (outbound, if needed)
volumes:
- maildata:/var/mail
- mailstate:/var/mail-state
- maillogs:/var/log/mail
- /etc/localtime:/etc/localtime:ro
- /etc/letsencrypt:/etc/letsencrypt:ro
environment:
# Hostname
- OVERRIDE_HOSTNAME=mail.scanner.meghsakha.com
@@ -34,14 +34,8 @@ services:
# Plus-addressing (critical for pentest)
- POSTFIX_RECIPIENT_DELIMITER=+
# TLS — use Let's Encrypt certs mounted from Coolify/Caddy
- SSL_TYPE=manual
- SSL_CERT_PATH=/etc/letsencrypt/live/mail.scanner.meghsakha.com/fullchain.pem
- SSL_KEY_PATH=/etc/letsencrypt/live/mail.scanner.meghsakha.com/privkey.pem
# Require TLS before accepting PLAIN/LOGIN auth (CERT-Bund compliance)
# Disable plaintext auth on unencrypted connections
- DOVECOT_DISABLE_PLAINTEXT_AUTH=yes
# SSL (start with no TLS, add Let's Encrypt later)
- SSL_TYPE=
# Accept mail for our domain
- PERMIT_DOCKER=none