Initial commit: Compliance Scanner Agent
Autonomous security and compliance scanning agent for git repositories. Features: SAST (Semgrep), SBOM (Syft), CVE monitoring (OSV.dev/NVD), GDPR/OAuth pattern detection, LLM triage, issue creation (GitHub/GitLab/Jira), PR reviews, and Dioxus fullstack dashboard. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Sharang Parnerkar
205
README.md
Normal file
205
README.md
Normal file
@@ -0,0 +1,205 @@
|
||||
<p align="center">
|
||||
<img src="assets/favicon.svg" width="96" height="96" alt="Compliance Scanner Logo" />
|
||||
</p>
|
||||
|
||||
<h1 align="center">Compliance Scanner</h1>
|
||||
|
||||
<p align="center">
|
||||
<strong>Autonomous security and compliance scanning agent for git repositories</strong>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="https://www.rust-lang.org/"><img src="https://img.shields.io/badge/Rust-1.89-orange?logo=rust&logoColor=white" alt="Rust" /></a>
|
||||
<a href="https://dioxuslabs.com/"><img src="https://img.shields.io/badge/Dioxus-0.7-blue?logo=webassembly&logoColor=white" alt="Dioxus" /></a>
|
||||
<a href="https://www.mongodb.com/"><img src="https://img.shields.io/badge/MongoDB-8.0-47A248?logo=mongodb&logoColor=white" alt="MongoDB" /></a>
|
||||
<a href="https://axum.rs/"><img src="https://img.shields.io/badge/Axum-0.8-4A4A55?logo=rust&logoColor=white" alt="Axum" /></a>
|
||||
<a href="https://tailwindcss.com/"><img src="https://img.shields.io/badge/Tailwind_CSS-4-06B6D4?logo=tailwindcss&logoColor=white" alt="Tailwind CSS" /></a>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<img src="https://img.shields.io/badge/GDPR-Scanning-green" alt="GDPR" />
|
||||
<img src="https://img.shields.io/badge/OAuth-Scanning-green" alt="OAuth" />
|
||||
<img src="https://img.shields.io/badge/SAST-Semgrep-blue" alt="SAST" />
|
||||
<img src="https://img.shields.io/badge/CVE-OSV.dev%20%2B%20NVD-orange" alt="CVE" />
|
||||
<img src="https://img.shields.io/badge/Platform-Linux%20%7C%20Docker-lightgrey?logo=linux&logoColor=white" alt="Platform" />
|
||||
</p>
|
||||
|
||||
---
|
||||
|
||||
## About
|
||||
|
||||
Compliance Scanner is an autonomous agent that continuously monitors git repositories for security vulnerabilities, GDPR/OAuth compliance patterns, and dependency risks. It creates issues in external trackers (GitHub/GitLab/Jira) with evidence and remediation suggestions, reviews pull requests, and exposes a Dioxus-based dashboard for visualization.
|
||||
|
||||
> **How it works:** The agent runs as a lazy daemon -- it only scans when new commits are detected, triggered by cron schedules or webhooks. LLM-powered triage filters out false positives and generates actionable remediation.
|
||||
|
||||
## Features
|
||||
|
||||
| Area | Capabilities |
|
||||
|------|-------------|
|
||||
| **SAST Scanning** | Semgrep-based static analysis with auto-config rules |
|
||||
| **SBOM Generation** | Syft + cargo-audit for complete dependency inventory |
|
||||
| **CVE Monitoring** | OSV.dev batch queries, NVD CVSS enrichment, SearXNG context |
|
||||
| **GDPR Patterns** | Detect PII logging, missing consent, hardcoded retention, missing deletion |
|
||||
| **OAuth Patterns** | Detect implicit grant, missing PKCE, token in localStorage, token in URLs |
|
||||
| **LLM Triage** | Confidence scoring via LiteLLM to filter false positives |
|
||||
| **Issue Creation** | Auto-create issues in GitHub, GitLab, or Jira with code evidence |
|
||||
| **PR Reviews** | Post security review comments on pull requests |
|
||||
| **Dashboard** | Fullstack Dioxus UI with findings, SBOM, issues, and statistics |
|
||||
| **Webhooks** | GitHub (HMAC-SHA256) and GitLab webhook receivers for push/PR events |
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Cargo Workspace │
|
||||
├──────────────┬──────────────────┬───────────────────────────┤
|
||||
│ compliance- │ compliance- │ compliance- │
|
||||
│ core │ agent │ dashboard │
|
||||
│ (lib) │ (bin) │ (bin, Dioxus 0.7.3) │
|
||||
│ │ │ │
|
||||
│ Models │ Scan Pipeline │ Fullstack Web UI │
|
||||
│ Traits │ LLM Client │ Server Functions │
|
||||
│ Config │ Issue Trackers │ Charts + Tables │
|
||||
│ Errors │ Scheduler │ Settings Page │
|
||||
│ │ REST API │ │
|
||||
│ │ Webhooks │ │
|
||||
└──────────────┴──────────────────┴───────────────────────────┘
|
||||
│
|
||||
MongoDB (shared)
|
||||
```
|
||||
|
||||
## Scan Pipeline (7 Stages)
|
||||
|
||||
1. **Change Detection** -- `git2` fetch, compare HEAD SHA with last scanned commit
|
||||
2. **Semgrep SAST** -- CLI wrapper with JSON output parsing
|
||||
3. **SBOM Generation** -- Syft (CycloneDX) + cargo-audit vulnerability merge
|
||||
4. **CVE Scanning** -- OSV.dev batch + NVD CVSS enrichment + SearXNG context
|
||||
5. **Pattern Scanning** -- Regex-based GDPR and OAuth compliance checks
|
||||
6. **LLM Triage** -- LiteLLM confidence scoring, filter findings < 3/10
|
||||
7. **Issue Creation** -- Dedup via SHA-256 fingerprint, create tracker issues
|
||||
|
||||
## Tech Stack
|
||||
|
||||
| Layer | Technology |
|
||||
|-------|-----------|
|
||||
| Shared Library | `compliance-core` -- models, traits, config |
|
||||
| Agent | Axum REST API, git2, tokio-cron-scheduler, Semgrep, Syft |
|
||||
| Dashboard | Dioxus 0.7.3 fullstack, Tailwind CSS |
|
||||
| Database | MongoDB with typed collections |
|
||||
| LLM | LiteLLM (OpenAI-compatible API) |
|
||||
| Issue Trackers | GitHub (octocrab), GitLab (REST v4), Jira (REST v3) |
|
||||
| CVE Sources | OSV.dev, NVD, SearXNG |
|
||||
|
||||
## Getting Started
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Rust 1.89+
|
||||
- [Dioxus CLI](https://dioxuslabs.com/learn/0.7/getting_started) (`dx`)
|
||||
- MongoDB
|
||||
- Docker & Docker Compose (optional)
|
||||
|
||||
### Optional External Tools
|
||||
|
||||
- [Semgrep](https://semgrep.dev/) -- for SAST scanning
|
||||
- [Syft](https://github.com/anchore/syft) -- for SBOM generation
|
||||
- [cargo-audit](https://github.com/rustsec/rustsec) -- for Rust dependency auditing
|
||||
|
||||
### Setup
|
||||
|
||||
```bash
|
||||
# Clone the repository
|
||||
git clone <repo-url>
|
||||
cd compliance-scanner
|
||||
|
||||
# Start MongoDB + SearXNG
|
||||
docker compose up -d mongo searxng
|
||||
|
||||
# Configure environment
|
||||
cp .env.example .env
|
||||
# Edit .env with your LiteLLM, tracker tokens, and MongoDB settings
|
||||
|
||||
# Run the agent
|
||||
cargo run -p compliance-agent
|
||||
|
||||
# Run the dashboard (separate terminal)
|
||||
dx serve --features server --platform web
|
||||
```
|
||||
|
||||
### Docker Compose (Full Stack)
|
||||
|
||||
```bash
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
This starts MongoDB, SearXNG, the agent (port 3001), and the dashboard (port 8080).
|
||||
|
||||
## REST API
|
||||
|
||||
The agent exposes a REST API on port 3001:
|
||||
|
||||
| Method | Endpoint | Description |
|
||||
|--------|----------|-------------|
|
||||
| `GET` | `/api/v1/health` | Health check |
|
||||
| `GET` | `/api/v1/stats/overview` | Summary statistics and trends |
|
||||
| `GET` | `/api/v1/repositories` | List tracked repositories |
|
||||
| `POST` | `/api/v1/repositories` | Add a repository to track |
|
||||
| `POST` | `/api/v1/repositories/:id/scan` | Trigger a manual scan |
|
||||
| `GET` | `/api/v1/findings` | List findings (filterable) |
|
||||
| `GET` | `/api/v1/findings/:id` | Get finding with code evidence |
|
||||
| `PATCH` | `/api/v1/findings/:id/status` | Update finding status |
|
||||
| `GET` | `/api/v1/sbom` | List dependencies |
|
||||
| `GET` | `/api/v1/issues` | List cross-tracker issues |
|
||||
| `GET` | `/api/v1/scan-runs` | Scan execution history |
|
||||
| `POST` | `/webhook/github` | GitHub webhook (HMAC-SHA256) |
|
||||
| `POST` | `/webhook/gitlab` | GitLab webhook (token verify) |
|
||||
|
||||
## Dashboard Pages
|
||||
|
||||
| Page | Description |
|
||||
|------|-------------|
|
||||
| **Overview** | Stat cards, severity distribution chart |
|
||||
| **Repositories** | Add/manage tracked repos, trigger scans |
|
||||
| **Findings** | Filterable table by severity, type, status |
|
||||
| **Finding Detail** | Code evidence, remediation, suggested fix, linked issue |
|
||||
| **SBOM** | Dependency inventory with vulnerability badges |
|
||||
| **Issues** | Cross-tracker view (GitHub + GitLab + Jira) |
|
||||
| **Settings** | Configure LiteLLM, tracker tokens, SearXNG URL |
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
compliance-scanner/
|
||||
├── compliance-core/ Shared library (models, traits, config, errors)
|
||||
├── compliance-agent/ Agent daemon (pipeline, LLM, trackers, API, webhooks)
|
||||
│ └── src/
|
||||
│ ├── pipeline/ 7-stage scan pipeline
|
||||
│ ├── llm/ LiteLLM client, triage, descriptions, fixes, PR review
|
||||
│ ├── trackers/ GitHub, GitLab, Jira integrations
|
||||
│ ├── api/ REST API (Axum)
|
||||
│ └── webhooks/ GitHub + GitLab webhook receivers
|
||||
├── compliance-dashboard/ Dioxus fullstack dashboard
|
||||
│ └── src/
|
||||
│ ├── components/ Reusable UI components
|
||||
│ ├── infrastructure/ Server functions, DB, config
|
||||
│ └── pages/ Full page views
|
||||
├── assets/ Static assets (CSS, icons)
|
||||
├── styles/ Tailwind input stylesheet
|
||||
└── bin/ Dashboard binary entrypoint
|
||||
```
|
||||
|
||||
## External Services
|
||||
|
||||
| Service | Purpose | Default URL |
|
||||
|---------|---------|-------------|
|
||||
| MongoDB | Persistence | `mongodb://localhost:27017` |
|
||||
| LiteLLM | LLM proxy for triage and generation | `http://localhost:4000` |
|
||||
| SearXNG | CVE context search | `http://localhost:8888` |
|
||||
| Semgrep | SAST scanning | CLI tool |
|
||||
| Syft | SBOM generation | CLI tool |
|
||||
|
||||
---
|
||||
|
||||
<p align="center">
|
||||
<sub>Built with Rust, Dioxus, and a commitment to automated security compliance.</sub>
|
||||
</p>
|
||||
Reference in New Issue
Block a user