feat(pipeline): add adversarial tests (30 cases) + regression harness

Block C implementation:
- adversarial_cases.yaml: 30 tricky cases in 5 categories
  (wrong legal basis, dark patterns, incomplete docs, similar-but-different, homonyms)
- test_adversarial.py: 63 tests validating adversarial cases
- test_regression.py: ontology stability, dependency engine, quality metrics
- conftest.py: shared fixtures (DB session, sample controls)

Total: 371 tests passing (221 existing + 150 new).
Real-world benchmarks (C1) need manual ground truth creation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Benjamin Admin
2026-05-01 13:02:29 +02:00
parent 6f58fdbaa5
commit d9c16fb914
4 changed files with 740 additions and 0 deletions
+36
View File
@@ -0,0 +1,36 @@
"""Shared test fixtures for the control pipeline test suite."""
import os
import sys
import pytest
# Ensure control-pipeline is in path
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
@pytest.fixture(scope="session")
def db_session():
"""DB session for integration tests — skip if no DATABASE_URL."""
url = os.getenv("DATABASE_URL")
if not url:
pytest.skip("DATABASE_URL not set — skipping DB tests")
from db.session import SessionLocal
db = SessionLocal()
yield db
db.close()
@pytest.fixture
def sample_controls(db_session):
"""Load 100 random draft controls for regression testing."""
from sqlalchemy import text
rows = db_session.execute(text("""
SELECT control_id, title, category, severity,
generation_metadata->>'assertion' as assertion,
generation_metadata->>'check_type' as check_type,
generation_metadata->>'merge_group_hint' as merge_key
FROM compliance.canonical_controls
WHERE release_state = 'draft' AND decomposition_method = 'pass0b'
ORDER BY random() LIMIT 100
""")).fetchall()
return [dict(r._mapping) for r in rows]