Compare commits

..

9 Commits

Author SHA1 Message Date
Sharang Parnerkar 4424db5acb feat(dashboard): add light/dark theme with sidebar toggle
CI / Check (pull_request) Successful in 10m12s
CI / Detect Changes (pull_request) Has been skipped
CI / Deploy Agent (pull_request) Has been skipped
CI / Deploy Dashboard (pull_request) Has been skipped
CI / Deploy Docs (pull_request) Has been skipped
CI / Deploy MCP (pull_request) Has been skipped
Introduces a light theme alongside the existing dark Obsidian Control
look, plus a sun/moon toggle in the sidebar footer.

The dashboard's CSS already drove every surface through custom
properties on :root, so the light theme is added as a second token set
under `:root[data-theme="light"]` and, in parallel, inside a
`@media (prefers-color-scheme: light)` block guarded by
`:not([data-theme="dark"])`. Net effect:
- A user with no stored preference gets their OS theme via the media
  query (no flash, no JS required).
- A user who clicked the toggle gets `data-theme="light|dark"` set on
  `<html>`, which wins over the media query.

The toggle component (`theme_toggle.rs`) reads `localStorage` first
then `prefers-color-scheme` on mount, and writes both the DOM
attribute and `localStorage` on click. All `web_sys` calls are gated
behind `#[cfg(feature = "web")]` so the server build stays clean.

Three CSS rules that hardcoded near-black hex values (the page dot
grid, `.code-block`, and the graph stabilization overlay) get explicit
light-mode overrides so they don't render as dark patches on white.

web-sys feature list extended with Storage, MediaQueryList, and
Element so the toggle can read the media query and set the attribute.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-13 13:26:37 +02:00
sharang 927fbc8ecb fix: live progress + concurrency for embedding builds (#80)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 5s
CI / Deploy Agent (push) Successful in 7m59s
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-05-13 10:01:05 +00:00
sharang e67a13535a fix: add HTTP timeout to reqwest client and CVE stage timeout (#79)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 5s
CI / Deploy Agent (push) Successful in 8m26s
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-05-13 07:30:26 +00:00
sharang df0063abc0 fix: scanner timeouts, semgrep memory cap, syft remote lookups, Script error (#78)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 5s
CI / Deploy Agent (push) Successful in 9m41s
CI / Deploy Dashboard (push) Successful in 15m19s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Successful in 3m7s
## Summary

- **Scan produces no results in Orca** — semgrep (`--config=auto`, unbounded memory) and syft (remote license network calls) were getting OOM-killed or hanging in resource-constrained Orca containers. Scan would "complete" with 0 findings/SBOMs silently because each scanner failure is caught and logged as a warning.
- **Dashboard Script error spam** — `document::Script` in Dioxus 0.7 needs a single text node child for inline scripts; `dangerous_inner_html` was invalid and spammed the error log on every unauthenticated page load.

## Changes

| File | Change |
|------|--------|
| `semgrep.rs` | Add `--max-memory 500 --jobs 1`; 10-minute timeout |
| `syft.rs` | Remove remote license lookup env vars; 5-minute timeout |
| `gitleaks.rs` | 5-minute timeout |
| `app_shell.rs` | Fix `dangerous_inner_html` → text child in `document::Script` |

## Test plan

- [ ] Trigger a scan on a repo in Orca — findings and SBOM entries should now appear
- [ ] Agent logs should show timeout/error warnings rather than silent empty results when tools are killed
- [ ] Navigate to dashboard unauthenticated — Script error gone from logs
- [ ] Verify scans work end-to-end with `docker compose up`

---------

Co-authored-by: Sharang Parnerkar <30073382+mighty840@users.noreply.github.com>
Reviewed-on: #78
2026-05-12 11:27:24 +00:00
Sharang Parnerkar 5cafd13f44 ci: log orca webhook response so deploy steps arent silent
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 5s
CI / Deploy Agent (push) Has been skipped
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
Nightly E2E Tests / E2E Tests (push) Failing after 2m59s
2026-04-08 15:09:27 +02:00
Sharang Parnerkar 69209649a5 ci: trigger first orca build for all services
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 4s
CI / Deploy Agent (push) Successful in 7m5s
CI / Deploy Docs (push) Successful in 30s
CI / Deploy MCP (push) Successful in 1m31s
CI / Deploy Dashboard (push) Failing after 21m28s
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 10:10:07 +02:00
Sharang Parnerkar d5439adc0d ci: trigger build of dashboard, docs, mcp images for orca
CI / Check (push) Has been cancelled
CI / Detect Changes (push) Has been cancelled
CI / Deploy Agent (push) Has been cancelled
CI / Deploy Dashboard (push) Has been cancelled
CI / Deploy Docs (push) Has been cancelled
CI / Deploy MCP (push) Has been cancelled
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 10:09:49 +02:00
Sharang Parnerkar bc7cdd35e4 ci: replace coolify webhook with orca deploy
CI / Check (push) Has been cancelled
CI / Detect Changes (push) Has been cancelled
CI / Deploy Agent (push) Has been cancelled
CI / Deploy Dashboard (push) Has been cancelled
CI / Deploy Docs (push) Has been cancelled
CI / Deploy MCP (push) Has been cancelled
Each deploy job now builds the per-service image, pushes to the
private registry as :latest and :sha, then triggers an HMAC-signed
orca redeploy webhook. Coolify webhooks are no longer used.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 10:06:11 +02:00
Sharang Parnerkar c062d834a1 fix: downgrade dotenv missing file from FAILED to info message
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Successful in 2s
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
Nightly E2E Tests / E2E Tests (push) Failing after 2m16s
Non-fatal in Docker where env vars come from container config.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 15:33:24 +02:00
21 changed files with 502 additions and 101 deletions
+10
View File
@@ -0,0 +1,10 @@
[advisories]
ignore = [
# hickory-proto 0.25.x pulled in transitively via mongodb → hickory-resolver.
# MongoDB 3.x has not yet released with hickory-resolver 0.26.x, so we cannot
# upgrade past this without a mongodb release. Both are DNS-layer DoS vectors
# requiring a MITM/controlled DNS server against MongoDB's hostname resolution —
# not a realistic attack surface here. Revisit when mongodb bumps hickory.
"RUSTSEC-2026-0118", # NSEC3 loop, no fix available upstream
"RUSTSEC-2026-0119", # O(n²) name compression, fixed in hickory-proto >=0.26.1
]
+48 -20
View File
@@ -145,13 +145,20 @@ jobs:
needs: [detect-changes] needs: [detect-changes]
if: needs.detect-changes.outputs.agent == 'true' if: needs.detect-changes.outputs.agent == 'true'
container: container:
image: alpine:latest image: docker:27-cli
steps: steps:
- name: Trigger Coolify deploy - name: Build, push and trigger orca redeploy
run: | run: |
apk add --no-cache curl apk add --no-cache git curl openssl
curl -sf "${{ secrets.COOLIFY_WEBHOOK_AGENT }}" \ git init && git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
-H "Authorization: Bearer ${{ secrets.COOLIFY_TOKEN }}" git fetch --depth=1 origin "${GITHUB_SHA}" && git checkout FETCH_HEAD
IMAGE=registry.meghsakha.com/compliance-agent
echo "${{ secrets.REGISTRY_PASSWORD }}" | docker login registry.meghsakha.com -u "${{ secrets.REGISTRY_USERNAME }}" --password-stdin
docker build -f Dockerfile.agent -t "$IMAGE:latest" -t "$IMAGE:${GITHUB_SHA}" .
docker push "$IMAGE:latest" && docker push "$IMAGE:${GITHUB_SHA}"
PAYLOAD=$(printf '{"ref":"refs/heads/main","repository":{"full_name":"sharang/compliance-scanner-agent"},"head_commit":{"id":"%s","message":"deploy agent"}}' "${GITHUB_SHA}")
SIG=$(printf '%s' "$PAYLOAD" | openssl dgst -sha256 -hmac "${{ secrets.ORCA_WEBHOOK_SECRET }}" | awk '{print $2}')
RESP=$(curl -fsS -w "\nHTTP %{http_code}" -X POST "http://46.225.100.82:6880/api/v1/webhooks/github" -H "Content-Type: application/json" -H "X-Hub-Signature-256: sha256=$SIG" -d "$PAYLOAD"); echo "$RESP"
deploy-dashboard: deploy-dashboard:
name: Deploy Dashboard name: Deploy Dashboard
@@ -159,13 +166,20 @@ jobs:
needs: [detect-changes] needs: [detect-changes]
if: needs.detect-changes.outputs.dashboard == 'true' if: needs.detect-changes.outputs.dashboard == 'true'
container: container:
image: alpine:latest image: docker:27-cli
steps: steps:
- name: Trigger Coolify deploy - name: Build, push and trigger orca redeploy
run: | run: |
apk add --no-cache curl apk add --no-cache git curl openssl
curl -sf "${{ secrets.COOLIFY_WEBHOOK_DASHBOARD }}" \ git init && git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
-H "Authorization: Bearer ${{ secrets.COOLIFY_TOKEN }}" git fetch --depth=1 origin "${GITHUB_SHA}" && git checkout FETCH_HEAD
IMAGE=registry.meghsakha.com/compliance-dashboard
echo "${{ secrets.REGISTRY_PASSWORD }}" | docker login registry.meghsakha.com -u "${{ secrets.REGISTRY_USERNAME }}" --password-stdin
docker build -f Dockerfile.dashboard -t "$IMAGE:latest" -t "$IMAGE:${GITHUB_SHA}" .
docker push "$IMAGE:latest" && docker push "$IMAGE:${GITHUB_SHA}"
PAYLOAD=$(printf '{"ref":"refs/heads/main","repository":{"full_name":"sharang/compliance-scanner-agent"},"head_commit":{"id":"%s","message":"deploy dashboard"}}' "${GITHUB_SHA}")
SIG=$(printf '%s' "$PAYLOAD" | openssl dgst -sha256 -hmac "${{ secrets.ORCA_WEBHOOK_SECRET }}" | awk '{print $2}')
RESP=$(curl -fsS -w "\nHTTP %{http_code}" -X POST "http://46.225.100.82:6880/api/v1/webhooks/github" -H "Content-Type: application/json" -H "X-Hub-Signature-256: sha256=$SIG" -d "$PAYLOAD"); echo "$RESP"
deploy-docs: deploy-docs:
name: Deploy Docs name: Deploy Docs
@@ -173,13 +187,20 @@ jobs:
needs: [detect-changes] needs: [detect-changes]
if: needs.detect-changes.outputs.docs == 'true' if: needs.detect-changes.outputs.docs == 'true'
container: container:
image: alpine:latest image: docker:27-cli
steps: steps:
- name: Trigger Coolify deploy - name: Build, push and trigger orca redeploy
run: | run: |
apk add --no-cache curl apk add --no-cache git curl openssl
curl -sf "${{ secrets.COOLIFY_WEBHOOK_DOCS }}" \ git init && git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
-H "Authorization: Bearer ${{ secrets.COOLIFY_TOKEN }}" git fetch --depth=1 origin "${GITHUB_SHA}" && git checkout FETCH_HEAD
IMAGE=registry.meghsakha.com/compliance-docs
echo "${{ secrets.REGISTRY_PASSWORD }}" | docker login registry.meghsakha.com -u "${{ secrets.REGISTRY_USERNAME }}" --password-stdin
docker build -f Dockerfile.docs -t "$IMAGE:latest" -t "$IMAGE:${GITHUB_SHA}" .
docker push "$IMAGE:latest" && docker push "$IMAGE:${GITHUB_SHA}"
PAYLOAD=$(printf '{"ref":"refs/heads/main","repository":{"full_name":"sharang/compliance-scanner-agent"},"head_commit":{"id":"%s","message":"deploy docs"}}' "${GITHUB_SHA}")
SIG=$(printf '%s' "$PAYLOAD" | openssl dgst -sha256 -hmac "${{ secrets.ORCA_WEBHOOK_SECRET }}" | awk '{print $2}')
RESP=$(curl -fsS -w "\nHTTP %{http_code}" -X POST "http://46.225.100.82:6880/api/v1/webhooks/github" -H "Content-Type: application/json" -H "X-Hub-Signature-256: sha256=$SIG" -d "$PAYLOAD"); echo "$RESP"
deploy-mcp: deploy-mcp:
name: Deploy MCP name: Deploy MCP
@@ -187,10 +208,17 @@ jobs:
needs: [detect-changes] needs: [detect-changes]
if: needs.detect-changes.outputs.mcp == 'true' if: needs.detect-changes.outputs.mcp == 'true'
container: container:
image: alpine:latest image: docker:27-cli
steps: steps:
- name: Trigger Coolify deploy - name: Build, push and trigger orca redeploy
run: | run: |
apk add --no-cache curl apk add --no-cache git curl openssl
curl -sf "${{ secrets.COOLIFY_WEBHOOK_MCP }}" \ git init && git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
-H "Authorization: Bearer ${{ secrets.COOLIFY_TOKEN }}" git fetch --depth=1 origin "${GITHUB_SHA}" && git checkout FETCH_HEAD
IMAGE=registry.meghsakha.com/compliance-mcp
echo "${{ secrets.REGISTRY_PASSWORD }}" | docker login registry.meghsakha.com -u "${{ secrets.REGISTRY_USERNAME }}" --password-stdin
docker build -f Dockerfile.mcp -t "$IMAGE:latest" -t "$IMAGE:${GITHUB_SHA}" .
docker push "$IMAGE:latest" && docker push "$IMAGE:${GITHUB_SHA}"
PAYLOAD=$(printf '{"ref":"refs/heads/main","repository":{"full_name":"sharang/compliance-scanner-agent"},"head_commit":{"id":"%s","message":"deploy mcp"}}' "${GITHUB_SHA}")
SIG=$(printf '%s' "$PAYLOAD" | openssl dgst -sha256 -hmac "${{ secrets.ORCA_WEBHOOK_SECRET }}" | awk '{print $2}')
RESP=$(curl -fsS -w "\nHTTP %{http_code}" -X POST "http://46.225.100.82:6880/api/v1/webhooks/github" -H "Content-Type: application/json" -H "X-Hub-Signature-256: sha256=$SIG" -d "$PAYLOAD"); echo "$RESP"
Generated
+6 -6
View File
@@ -3524,9 +3524,9 @@ checksum = "224484c5d09285a7b8cb0a0c117e847ebd14cb6e4470ecf68cdb89c503b0edb9"
[[package]] [[package]]
name = "mongodb" name = "mongodb"
version = "3.5.1" version = "3.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "803dd859e8afa084c255a8effd8000ff86f7c8076a50cd6d8c99e8f3496f75c2" checksum = "1ef2c933617431ad0246fb5b43c425ebdae18c7f7259c87de0726d93b0e7e91b"
dependencies = [ dependencies = [
"base64", "base64",
"bitflags", "bitflags",
@@ -3570,9 +3570,9 @@ dependencies = [
[[package]] [[package]]
name = "mongodb-internal-macros" name = "mongodb-internal-macros"
version = "3.5.1" version = "3.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a973ef3dd3dbc6f6e65bbdecfd9ec5e781b9e7493b0f369a7c62e35d8e5ae2c8" checksum = "9e5758dc828eb2d02ec30563cba365609d56ddd833190b192beaee2b475a7bb3"
dependencies = [ dependencies = [
"macro_magic", "macro_magic",
"proc-macro2", "proc-macro2",
@@ -4699,9 +4699,9 @@ dependencies = [
[[package]] [[package]]
name = "rustls-webpki" name = "rustls-webpki"
version = "0.103.10" version = "0.103.13"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df33b2b81ac578cabaf06b89b0631153a3f416b0a886e8a7a1707fb51abbd1ef" checksum = "61c429a8649f110dddef65e2a5ad240f747e85f7758a6bccc7e5777bd33f756e"
dependencies = [ dependencies = [
"ring", "ring",
"rustls-pki-types", "rustls-pki-types",
+1
View File
@@ -44,3 +44,4 @@ RUN mkdir -p /data/compliance-scanner/ssh
EXPOSE 3001 3002 EXPOSE 3001 3002
ENTRYPOINT ["compliance-agent"] ENTRYPOINT ["compliance-agent"]
+1
View File
@@ -20,3 +20,4 @@ ENV IP=0.0.0.0
EXPOSE 8080 EXPOSE 8080
ENTRYPOINT ["./compliance-dashboard"] ENTRYPOINT ["./compliance-dashboard"]
+1
View File
@@ -12,3 +12,4 @@ RUN rm /etc/nginx/conf.d/default.conf
COPY docs/nginx.conf /etc/nginx/conf.d/default.conf COPY docs/nginx.conf /etc/nginx/conf.d/default.conf
COPY --from=builder /app/.vitepress/dist /usr/share/nginx/html COPY --from=builder /app/.vitepress/dist /usr/share/nginx/html
EXPOSE 80 EXPOSE 80
+1
View File
@@ -14,3 +14,4 @@ EXPOSE 8090
ENV MCP_PORT=8090 ENV MCP_PORT=8090
ENTRYPOINT ["compliance-mcp"] ENTRYPOINT ["compliance-mcp"]
+6 -1
View File
@@ -35,11 +35,16 @@ impl ComplianceAgent {
config.litellm_model.clone(), config.litellm_model.clone(),
config.litellm_embed_model.clone(), config.litellm_embed_model.clone(),
)); ));
let http = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(30))
.connect_timeout(std::time::Duration::from_secs(10))
.build()
.unwrap_or_default();
Self { Self {
config, config,
db, db,
llm, llm,
http: reqwest::Client::new(), http,
session_streams: Arc::new(DashMap::new()), session_streams: Arc::new(DashMap::new()),
session_pause: Arc::new(DashMap::new()), session_pause: Arc::new(DashMap::new()),
session_semaphore: Arc::new(Semaphore::new(DEFAULT_MAX_CONCURRENT_SESSIONS)), session_semaphore: Arc::new(Semaphore::new(DEFAULT_MAX_CONCURRENT_SESSIONS)),
+6 -1
View File
@@ -19,12 +19,17 @@ impl LlmClient {
model: String, model: String,
embed_model: String, embed_model: String,
) -> Self { ) -> Self {
let http = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(300))
.connect_timeout(std::time::Duration::from_secs(10))
.build()
.unwrap_or_default();
Self { Self {
base_url, base_url,
api_key, api_key,
model, model,
embed_model, embed_model,
http: reqwest::Client::new(), http,
} }
} }
+1 -1
View File
@@ -4,7 +4,7 @@ use compliance_agent::{agent, api, config, database, scheduler, ssh, webhooks};
async fn main() -> Result<(), Box<dyn std::error::Error>> { async fn main() -> Result<(), Box<dyn std::error::Error>> {
match dotenvy::dotenv() { match dotenvy::dotenv() {
Ok(path) => eprintln!("[dotenv] Loaded from: {}", path.display()), Ok(path) => eprintln!("[dotenv] Loaded from: {}", path.display()),
Err(e) => eprintln!("[dotenv] FAILED: {e}"), Err(_) => eprintln!("[dotenv] No .env file found, using environment variables"),
} }
let _telemetry_guard = compliance_core::telemetry::init_telemetry("compliance-agent"); let _telemetry_guard = compliance_core::telemetry::init_telemetry("compliance-agent");
+9 -2
View File
@@ -19,7 +19,9 @@ impl Scanner for GitleaksScanner {
#[tracing::instrument(skip_all)] #[tracing::instrument(skip_all)]
async fn scan(&self, repo_path: &Path, repo_id: &str) -> Result<ScanOutput, CoreError> { async fn scan(&self, repo_path: &Path, repo_id: &str) -> Result<ScanOutput, CoreError> {
let output = tokio::process::Command::new("gitleaks") let output = tokio::time::timeout(
std::time::Duration::from_secs(300),
tokio::process::Command::new("gitleaks")
.args([ .args([
"detect", "detect",
"--source", "--source",
@@ -33,8 +35,13 @@ impl Scanner for GitleaksScanner {
"0", "0",
]) ])
.current_dir(repo_path) .current_dir(repo_path)
.output() .output(),
)
.await .await
.map_err(|_| CoreError::Scanner {
scanner: "gitleaks".to_string(),
source: "timed out after 5 minutes".into(),
})?
.map_err(|e| CoreError::Scanner { .map_err(|e| CoreError::Scanner {
scanner: "gitleaks".to_string(), scanner: "gitleaks".to_string(),
source: Box::new(e), source: Box::new(e),
+11 -4
View File
@@ -174,19 +174,26 @@ impl PipelineOrchestrator {
k.expose_secret().to_string() k.expose_secret().to_string()
}), }),
); );
let cve_alerts = match async { let cve_alerts = match tokio::time::timeout(
std::time::Duration::from_secs(600),
async {
cve_scanner cve_scanner
.scan_dependencies(&repo_id, &mut sbom_entries) .scan_dependencies(&repo_id, &mut sbom_entries)
.await .await
} }
.instrument(tracing::info_span!("stage_cve_scanning")) .instrument(tracing::info_span!("stage_cve_scanning")),
)
.await .await
{ {
Ok(alerts) => alerts, Ok(Ok(alerts)) => alerts,
Err(e) => { Ok(Err(e)) => {
tracing::warn!("[{repo_id}] CVE scanning failed: {e}"); tracing::warn!("[{repo_id}] CVE scanning failed: {e}");
Vec::new() Vec::new()
} }
Err(_) => {
tracing::warn!("[{repo_id}] CVE scanning timed out after 10 minutes");
Vec::new()
}
}; };
// Stage 4: Pattern Scanning (GDPR + OAuth) // Stage 4: Pattern Scanning (GDPR + OAuth)
+9 -3
View File
@@ -5,16 +5,22 @@ use compliance_core::CoreError;
#[tracing::instrument(skip_all, fields(repo_id = %repo_id))] #[tracing::instrument(skip_all, fields(repo_id = %repo_id))]
pub(super) async fn run_syft(repo_path: &Path, repo_id: &str) -> Result<Vec<SbomEntry>, CoreError> { pub(super) async fn run_syft(repo_path: &Path, repo_id: &str) -> Result<Vec<SbomEntry>, CoreError> {
let output = tokio::process::Command::new("syft") let output = tokio::time::timeout(
std::time::Duration::from_secs(300),
tokio::process::Command::new("syft")
.arg(repo_path) .arg(repo_path)
.args(["-o", "cyclonedx-json"]) .args(["-o", "cyclonedx-json"])
// Enable remote license lookups for all ecosystems
.env("SYFT_GOLANG_SEARCH_REMOTE_LICENSES", "true") .env("SYFT_GOLANG_SEARCH_REMOTE_LICENSES", "true")
.env("SYFT_JAVASCRIPT_SEARCH_REMOTE_LICENSES", "true") .env("SYFT_JAVASCRIPT_SEARCH_REMOTE_LICENSES", "true")
.env("SYFT_PYTHON_SEARCH_REMOTE_LICENSES", "true") .env("SYFT_PYTHON_SEARCH_REMOTE_LICENSES", "true")
.env("SYFT_JAVA_USE_NETWORK", "true") .env("SYFT_JAVA_USE_NETWORK", "true")
.output() .output(),
)
.await .await
.map_err(|_| CoreError::Scanner {
scanner: "syft".to_string(),
source: "timed out after 5 minutes".into(),
})?
.map_err(|e| CoreError::Scanner { .map_err(|e| CoreError::Scanner {
scanner: "syft".to_string(), scanner: "syft".to_string(),
source: Box::new(e), source: Box::new(e),
+18 -3
View File
@@ -19,11 +19,26 @@ impl Scanner for SemgrepScanner {
#[tracing::instrument(skip_all)] #[tracing::instrument(skip_all)]
async fn scan(&self, repo_path: &Path, repo_id: &str) -> Result<ScanOutput, CoreError> { async fn scan(&self, repo_path: &Path, repo_id: &str) -> Result<ScanOutput, CoreError> {
let output = tokio::process::Command::new("semgrep") let output = tokio::time::timeout(
.args(["--config=auto", "--json", "--quiet"]) std::time::Duration::from_secs(600),
tokio::process::Command::new("semgrep")
.args([
"--config=auto",
"--json",
"--quiet",
"--max-memory",
"500",
"--jobs",
"1",
])
.arg(repo_path) .arg(repo_path)
.output() .output(),
)
.await .await
.map_err(|_| CoreError::Scanner {
scanner: "semgrep".to_string(),
source: "timed out after 10 minutes".into(),
})?
.map_err(|e| CoreError::Scanner { .map_err(|e| CoreError::Scanner {
scanner: "semgrep".to_string(), scanner: "semgrep".to_string(),
source: Box::new(e), source: Box::new(e),
+84 -16
View File
@@ -6,11 +6,16 @@ use compliance_core::models::embedding::{CodeEmbedding, EmbeddingBuildRun, Embed
use compliance_core::models::graph::CodeNode; use compliance_core::models::graph::CodeNode;
use compliance_graph::graph::chunking::extract_chunks; use compliance_graph::graph::chunking::extract_chunks;
use compliance_graph::graph::embedding_store::EmbeddingStore; use compliance_graph::graph::embedding_store::EmbeddingStore;
use futures_util::stream::{FuturesUnordered, StreamExt};
use tracing::{error, info}; use tracing::{error, info};
use crate::error::AgentError; use crate::error::AgentError;
use crate::llm::LlmClient; use crate::llm::LlmClient;
const EMBED_BATCH_SIZE: usize = 20;
const EMBED_CONCURRENCY: usize = 4;
const EMBED_FLUSH_EVERY: usize = 200;
/// RAG pipeline for building embeddings and performing retrieval /// RAG pipeline for building embeddings and performing retrieval
pub struct RagPipeline { pub struct RagPipeline {
llm: Arc<LlmClient>, llm: Arc<LlmClient>,
@@ -77,25 +82,33 @@ impl RagPipeline {
.await .await
.map_err(|e| AgentError::Other(format!("Failed to delete old embeddings: {e}")))?; .map_err(|e| AgentError::Other(format!("Failed to delete old embeddings: {e}")))?;
// Step 3: Batch embed (small batches to stay within model limits) // Step 3: Batch embed with bounded concurrency. Flush to Mongo and
let batch_size = 20; // update progress periodically so the dashboard can show live status.
let mut all_embeddings = Vec::new(); let mut pending = Vec::with_capacity(EMBED_FLUSH_EVERY);
let mut embedded_count = 0u32; let mut embedded_count = 0u32;
for batch_start in (0..chunks.len()).step_by(batch_size) { // Build the list of batch indices to process.
let batch_end = (batch_start + batch_size).min(chunks.len()); let batches: Vec<(usize, usize)> = (0..chunks.len())
let batch_chunks = &chunks[batch_start..batch_end]; .step_by(EMBED_BATCH_SIZE)
.map(|start| (start, (start + EMBED_BATCH_SIZE).min(chunks.len())))
// Prepare texts: context_header + content
let texts: Vec<String> = batch_chunks
.iter()
.map(|c| format!("{}\n{}", c.context_header, c.content))
.collect(); .collect();
match self.llm.embed(texts).await { let mut batch_iter = batches.into_iter();
Ok(vectors) => { let mut in_flight = FuturesUnordered::new();
// Prime up to EMBED_CONCURRENCY batches.
for _ in 0..EMBED_CONCURRENCY {
if let Some((start, end)) = batch_iter.next() {
in_flight.push(self.embed_batch(&chunks[start..end], start, end));
}
}
while let Some(result) = in_flight.next().await {
match result {
Ok((start, end, vectors)) => {
let batch_chunks = &chunks[start..end];
for (chunk, embedding) in batch_chunks.iter().zip(vectors) { for (chunk, embedding) in batch_chunks.iter().zip(vectors) {
all_embeddings.push(CodeEmbedding { pending.push(CodeEmbedding {
id: None, id: None,
repo_id: repo_id.to_string(), repo_id: repo_id.to_string(),
graph_build_id: graph_build_id.to_string(), graph_build_id: graph_build_id.to_string(),
@@ -113,9 +126,45 @@ impl RagPipeline {
}); });
} }
embedded_count += batch_chunks.len() as u32; embedded_count += batch_chunks.len() as u32;
// Flush pending embeddings to Mongo periodically and update progress.
if pending.len() >= EMBED_FLUSH_EVERY {
self.embedding_store
.store_embeddings(&pending)
.await
.map_err(|e| {
AgentError::Other(format!("Failed to store embeddings: {e}"))
})?;
pending.clear();
}
// Always update the progress counter on the build doc — even if
// we haven't flushed embeddings yet — so the UI shows movement.
if let Err(e) = self
.embedding_store
.update_build(
repo_id,
graph_build_id,
EmbeddingBuildStatus::Running,
embedded_count,
None,
)
.await
{
error!("[{repo_id}] Failed to update build progress: {e}");
}
// Queue the next batch to keep concurrency saturated.
if let Some((s, e)) = batch_iter.next() {
in_flight.push(self.embed_batch(&chunks[s..e], s, e));
}
} }
Err(e) => { Err(e) => {
error!("[{repo_id}] Embedding batch failed: {e}"); error!("[{repo_id}] Embedding batch failed: {e}");
// Flush whatever we have so partial progress isn't lost.
if !pending.is_empty() {
let _ = self.embedding_store.store_embeddings(&pending).await;
}
build.status = EmbeddingBuildStatus::Failed; build.status = EmbeddingBuildStatus::Failed;
build.error_message = Some(e.to_string()); build.error_message = Some(e.to_string());
build.completed_at = Some(Utc::now()); build.completed_at = Some(Utc::now());
@@ -134,11 +183,13 @@ impl RagPipeline {
} }
} }
// Step 4: Store all embeddings // Step 4: Flush any remaining embeddings
if !pending.is_empty() {
self.embedding_store self.embedding_store
.store_embeddings(&all_embeddings) .store_embeddings(&pending)
.await .await
.map_err(|e| AgentError::Other(format!("Failed to store embeddings: {e}")))?; .map_err(|e| AgentError::Other(format!("Failed to store embeddings: {e}")))?;
}
// Step 5: Update build status // Step 5: Update build status
build.status = EmbeddingBuildStatus::Completed; build.status = EmbeddingBuildStatus::Completed;
@@ -161,4 +212,21 @@ impl RagPipeline {
); );
Ok(build) Ok(build)
} }
/// Embed one batch of chunks. Returns the (start, end, vectors) tuple so
/// out-of-order completion from `FuturesUnordered` can still be reconciled
/// against the original chunk slice.
async fn embed_batch(
&self,
batch_chunks: &[compliance_graph::graph::chunking::CodeChunk],
start: usize,
end: usize,
) -> Result<(usize, usize, Vec<Vec<f64>>), AgentError> {
let texts: Vec<String> = batch_chunks
.iter()
.map(|c| format!("{}\n{}", c.context_header, c.content))
.collect();
let vectors = self.llm.embed(texts).await?;
Ok((start, end, vectors))
}
} }
+1 -1
View File
@@ -51,7 +51,7 @@ thiserror = { workspace = true }
# Web-only # Web-only
reqwest = { workspace = true, optional = true } reqwest = { workspace = true, optional = true }
web-sys = { version = "0.3", optional = true, features = ["Blob", "BlobPropertyBag", "HtmlAnchorElement", "Url", "Document", "Window"] } web-sys = { version = "0.3", optional = true, features = ["Blob", "BlobPropertyBag", "HtmlAnchorElement", "Url", "Document", "Element", "Window", "Storage", "MediaQueryList"] }
js-sys = { version = "0.3", optional = true } js-sys = { version = "0.3", optional = true }
wasm-bindgen = { version = "0.2", optional = true } wasm-bindgen = { version = "0.2", optional = true }
gloo-timers = { version = "0.3", features = ["futures"], optional = true } gloo-timers = { version = "0.3", features = ["futures"], optional = true }
+139
View File
@@ -61,6 +61,77 @@
--ease-spring: cubic-bezier(0.34, 1.56, 0.64, 1); --ease-spring: cubic-bezier(0.34, 1.56, 0.64, 1);
} }
/* ── Light theme tokens ──
Applied when the user has explicitly chosen light (`data-theme="light"`)
OR when their OS prefers light AND they have made no explicit choice. */
:root[data-theme="light"] {
--bg-primary: #f5f7fb;
--bg-secondary: #ffffff;
--bg-card: rgba(255, 255, 255, 0.85);
--bg-card-solid: #ffffff;
--bg-card-hover: #f1f5fb;
--bg-elevated: #f8fafc;
--text-primary: #0c1426;
--text-secondary: #475569;
--text-tertiary: #8a9bb4;
--accent: #0070d4;
--accent-hover: #0080f0;
--accent-muted: rgba(0, 112, 212, 0.10);
--accent-glow: 0 0 20px rgba(0, 112, 212, 0.10);
--border: #e2e8f0;
--border-bright: #cbd5e1;
--border-accent: rgba(0, 112, 212, 0.30);
--danger: #dc2626;
--danger-bg: rgba(220, 38, 38, 0.08);
--warning: #d97706;
--warning-bg: rgba(217, 119, 6, 0.08);
--success: #16a34a;
--success-bg: rgba(22, 163, 74, 0.08);
--info: #2563eb;
--info-bg: rgba(37, 99, 235, 0.08);
--orange: #ea580c;
--orange-bg: rgba(234, 88, 12, 0.08);
}
@media (prefers-color-scheme: light) {
:root:not([data-theme="dark"]) {
--bg-primary: #f5f7fb;
--bg-secondary: #ffffff;
--bg-card: rgba(255, 255, 255, 0.85);
--bg-card-solid: #ffffff;
--bg-card-hover: #f1f5fb;
--bg-elevated: #f8fafc;
--text-primary: #0c1426;
--text-secondary: #475569;
--text-tertiary: #8a9bb4;
--accent: #0070d4;
--accent-hover: #0080f0;
--accent-muted: rgba(0, 112, 212, 0.10);
--accent-glow: 0 0 20px rgba(0, 112, 212, 0.10);
--border: #e2e8f0;
--border-bright: #cbd5e1;
--border-accent: rgba(0, 112, 212, 0.30);
--danger: #dc2626;
--danger-bg: rgba(220, 38, 38, 0.08);
--warning: #d97706;
--warning-bg: rgba(217, 119, 6, 0.08);
--success: #16a34a;
--success-bg: rgba(22, 163, 74, 0.08);
--info: #2563eb;
--info-bg: rgba(37, 99, 235, 0.08);
--orange: #ea580c;
--orange-bg: rgba(234, 88, 12, 0.08);
}
}
/* ── Reset & Base ── */ /* ── Reset & Base ── */
@@ -396,6 +467,44 @@ code {
background: rgba(0, 200, 255, 0.06); background: rgba(0, 200, 255, 0.06);
} }
.theme-toggle {
background: none;
border: none;
border-top: 1px solid var(--border);
color: var(--text-secondary);
padding: 11px 18px;
cursor: pointer;
display: flex;
align-items: center;
gap: 11px;
font-family: var(--font-body);
font-size: 13.5px;
font-weight: 500;
transition: color 0.2s, background 0.2s;
width: 100%;
text-align: left;
}
.theme-toggle:hover {
color: var(--accent);
background: var(--accent-muted);
}
.theme-toggle svg {
flex-shrink: 0;
opacity: 0.75;
transition: opacity 0.2s;
}
.theme-toggle:hover svg {
opacity: 1;
}
.sidebar.collapsed .theme-toggle {
justify-content: center;
padding: 11px 0;
}
.sidebar.collapsed .sidebar-header { .sidebar.collapsed .sidebar-header {
padding: 22px 0; padding: 22px 0;
justify-content: center; justify-content: center;
@@ -3889,3 +3998,33 @@ tbody tr:last-child td {
.copyable code, .copyable .mono { flex: 1; min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; } .copyable code, .copyable .mono { flex: 1; min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.code-snippet-wrapper { position: relative; } .code-snippet-wrapper { position: relative; }
.code-snippet-header { display: flex; align-items: center; justify-content: space-between; margin-bottom: 4px; gap: 8px; } .code-snippet-header { display: flex; align-items: center; justify-content: space-between; margin-bottom: 4px; gap: 8px; }
/* ═══════════════════════════════════════════════════════════════
LIGHT THEME — surface overrides for the few hardcoded dark
colors that don't go through CSS custom properties.
═══════════════════════════════════════════════════════════════ */
:root[data-theme="light"] .main-content {
background-image: radial-gradient(circle at 1px 1px, rgba(100, 116, 139, 0.18) 1px, transparent 0);
}
:root[data-theme="light"] .code-block {
background: #f8fafc;
color: #0c1426;
}
:root[data-theme="light"] .graph-stab-overlay {
background: radial-gradient(ellipse at center, rgba(245, 247, 251, 0.92) 0%, rgba(245, 247, 251, 0.98) 100%);
}
@media (prefers-color-scheme: light) {
:root:not([data-theme="dark"]) .main-content {
background-image: radial-gradient(circle at 1px 1px, rgba(100, 116, 139, 0.18) 1px, transparent 0);
}
:root:not([data-theme="dark"]) .code-block {
background: #f8fafc;
color: #0c1426;
}
:root:not([data-theme="dark"]) .graph-stab-overlay {
background: radial-gradient(ellipse at center, rgba(245, 247, 251, 0.92) 0%, rgba(245, 247, 251, 0.98) 100%);
}
}
@@ -32,7 +32,7 @@ pub fn AppShell() -> Element {
// Not authenticated — redirect to Keycloak login // Not authenticated — redirect to Keycloak login
rsx! { rsx! {
document::Script { document::Script {
dangerous_inner_html: "window.location.href = '/auth';" "window.location.href = '/auth';"
} }
} }
} }
@@ -12,4 +12,5 @@ pub mod pentest_wizard;
pub mod severity_badge; pub mod severity_badge;
pub mod sidebar; pub mod sidebar;
pub mod stat_card; pub mod stat_card;
pub mod theme_toggle;
pub mod toast; pub mod toast;
@@ -4,6 +4,7 @@ use dioxus_free_icons::icons::bs_icons::*;
use dioxus_free_icons::Icon; use dioxus_free_icons::Icon;
use crate::app::Route; use crate::app::Route;
use crate::components::theme_toggle::ThemeToggle;
struct NavItem { struct NavItem {
label: &'static str, label: &'static str,
@@ -106,6 +107,7 @@ pub fn Sidebar() -> Element {
} }
// Spacer pushes footer to the bottom // Spacer pushes footer to the bottom
div { class: "sidebar-spacer" } div { class: "sidebar-spacer" }
ThemeToggle { collapsed: collapsed() }
button { button {
class: "sidebar-toggle", class: "sidebar-toggle",
onclick: move |_| collapsed.set(!collapsed()), onclick: move |_| collapsed.set(!collapsed()),
@@ -0,0 +1,104 @@
use dioxus::prelude::*;
use dioxus_free_icons::icons::bs_icons::{BsMoonStars, BsSun};
use dioxus_free_icons::Icon;
#[cfg(feature = "web")]
const STORAGE_KEY: &str = "compliance-scanner.theme";
/// Sidebar-footer theme toggle. Reads the initial state on mount from
/// localStorage (explicit user choice) or `prefers-color-scheme` (OS default),
/// then writes back to both the `<html data-theme="...">` attribute and
/// localStorage on every click.
#[component]
pub fn ThemeToggle(collapsed: bool) -> Element {
// `None` until the on-mount effect resolves the real value, so SSR doesn't
// render the wrong icon for the user's actual theme.
let mut is_dark = use_signal(|| None::<bool>);
use_effect(move || {
let (dark, from_storage) = initial_theme();
is_dark.set(Some(dark));
// If the user already made an explicit choice (in localStorage), assert it
// on the DOM so an OS-vs-stored mismatch can't briefly show the wrong theme.
if from_storage {
apply_theme(dark);
}
});
let label = if collapsed {
""
} else if is_dark().unwrap_or(true) {
"Light mode"
} else {
"Dark mode"
};
let title = if is_dark().unwrap_or(true) {
"Switch to light mode"
} else {
"Switch to dark mode"
};
rsx! {
button {
class: "theme-toggle",
r#type: "button",
title: "{title}",
"aria-label": "{title}",
onclick: move |_| {
let next_dark = !is_dark().unwrap_or(true);
is_dark.set(Some(next_dark));
apply_theme(next_dark);
},
if is_dark().unwrap_or(true) {
Icon { icon: BsSun, width: 16, height: 16 }
} else {
Icon { icon: BsMoonStars, width: 16, height: 16 }
}
if !collapsed {
span { class: "theme-toggle-label", "{label}" }
}
}
}
}
/// Returns `(is_dark, from_storage)`. `from_storage` is true when an explicit
/// user choice is in localStorage; false when we fell back to OS preference
/// (or to the dark default).
#[cfg(feature = "web")]
fn initial_theme() -> (bool, bool) {
if let Some(window) = web_sys::window() {
if let Ok(Some(storage)) = window.local_storage() {
if let Ok(Some(value)) = storage.get_item(STORAGE_KEY) {
return (value == "dark", true);
}
}
if let Ok(Some(mql)) = window.match_media("(prefers-color-scheme: dark)") {
return (mql.matches(), false);
}
}
(true, false)
}
#[cfg(not(feature = "web"))]
fn initial_theme() -> (bool, bool) {
(true, false)
}
#[cfg(feature = "web")]
fn apply_theme(dark: bool) {
let theme = if dark { "dark" } else { "light" };
if let Some(window) = web_sys::window() {
if let Some(document) = window.document() {
if let Some(root) = document.document_element() {
let _ = root.set_attribute("data-theme", theme);
}
}
if let Ok(Some(storage)) = window.local_storage() {
let _ = storage.set_item(STORAGE_KEY, theme);
}
}
}
#[cfg(not(feature = "web"))]
fn apply_theme(_dark: bool) {}