Files
Benjamin Admin f2819b99af
Some checks failed
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Failing after 36s
CI/CD / test-python-backend-compliance (push) Successful in 36s
CI/CD / test-python-document-crawler (push) Successful in 27s
CI/CD / test-python-dsms-gateway (push) Successful in 18s
CI/CD / validate-canonical-controls (push) Successful in 11s
CI/CD / Deploy (push) Has been skipped
feat(pipeline): v3 — scoped control applicability + source_type classification
Phase 4: source_type (law/guideline/standard/restricted) on source_citation
- NIST/OWASP/ENISA correctly shown as "Standard" instead of "Gesetzliche Grundlage"
- Dynamic frontend labels based on source_type
- Backfill endpoint POST /v1/canonical/generate/backfill-source-type

Phase v3: Scoped Control Applicability
- 3 new fields: applicable_industries, applicable_company_size, scope_conditions
- LLM prompt extended with 39 industries, 5 company sizes, 10 scope signals
- All 5 generation paths (Rule 1/2/3, batch structure, batch reform) updated
- _build_control_from_json: parsing + validation (string→list, size validation)
- _store_control: writes 3 new JSONB columns
- API: response models, create/update requests, SELECT queries extended
- Migration 063: 3 new JSONB columns with GIN indexes
- 110 generator tests + 28 route tests = 138 total, all passing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 16:28:05 +01:00

258 lines
7.2 KiB
Markdown

# Test-Regeln
## Automatische Test-Erweiterung
**WICHTIG:** Bei JEDER Code-Aenderung muessen entsprechende Tests erstellt oder aktualisiert werden!
## Wann Tests schreiben?
### IMMER wenn du:
1. **Neue Funktionen** erstellst → Unit Test
2. **Neue API-Endpoints** hinzufuegst → Handler Test
3. **Bugs fixst** → Regression Test (der Bug sollte nie wieder auftreten)
4. **Bestehenden Code aenderst** → Bestehende Tests anpassen
## Test-Struktur
### Go Tests (Consent Service)
**Speicherort:** Im gleichen Verzeichnis wie der Code
```
internal/
├── services/
│ ├── auth_service.go
│ └── auth_service_test.go ← Test hier
├── handlers/
│ ├── handlers.go
│ └── handlers_test.go ← Test hier
└── middleware/
├── auth.go
└── middleware_test.go ← Test hier
```
**Test-Namenskonvention:**
```go
func TestFunctionName_Scenario_ExpectedResult(t *testing.T)
// Beispiele:
func TestHashPassword_ValidPassword_ReturnsHash(t *testing.T)
func TestLogin_InvalidCredentials_Returns401(t *testing.T)
func TestCreateDocument_MissingTitle_ReturnsError(t *testing.T)
```
**Test-Template:**
```go
func TestFunctionName(t *testing.T) {
// Arrange
service := &MyService{}
input := "test-input"
// Act
result, err := service.DoSomething(input)
// Assert
if err != nil {
t.Fatalf("Expected no error, got %v", err)
}
if result != expected {
t.Errorf("Expected %v, got %v", expected, result)
}
}
```
**Table-Driven Tests bevorzugen:**
```go
func TestValidateEmail(t *testing.T) {
tests := []struct {
name string
email string
expected bool
}{
{"valid email", "test@example.com", true},
{"missing @", "testexample.com", false},
{"empty", "", false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result := ValidateEmail(tt.email)
if result != tt.expected {
t.Errorf("Expected %v, got %v", tt.expected, result)
}
})
}
}
```
### Python Tests (Backend)
**Speicherort:** `/backend/tests/`
```
backend/
├── consent_client.py
├── gdpr_api.py
└── tests/
├── __init__.py
├── test_consent_client.py ← Tests fuer consent_client.py
└── test_gdpr_api.py ← Tests fuer gdpr_api.py
```
**Test-Namenskonvention:**
```python
class TestClassName:
def test_method_scenario_expected_result(self):
pass
# Beispiele:
class TestConsentClient:
def test_check_consent_valid_token_returns_status(self):
pass
def test_check_consent_expired_token_raises_error(self):
pass
```
**Test-Template:**
```python
import pytest
from unittest.mock import AsyncMock, patch, MagicMock
class TestMyFeature:
def test_sync_function(self):
# Arrange
input_data = "test"
# Act
result = my_function(input_data)
# Assert
assert result == expected
@pytest.mark.asyncio
async def test_async_function(self):
# Arrange
client = MyClient()
# Act
with patch("httpx.AsyncClient") as mock:
mock_instance = AsyncMock()
mock.return_value = mock_instance
result = await client.fetch_data()
# Assert
assert result is not None
```
## Test-Kategorien
### 1. Unit Tests (Hoechste Prioritaet)
- Testen einzelne Funktionen/Methoden
- Keine externen Abhaengigkeiten (Mocks verwenden)
- Schnell ausfuehrbar
### 2. Integration Tests
- Testen Zusammenspiel mehrerer Komponenten
- Koennen echte DB verwenden (Test-DB)
### 3. Security Tests
- Auth/JWT Validierung
- Passwort-Hashing
- Berechtigungspruefung
## Checkliste vor Abschluss
Vor dem Abschluss einer Aufgabe:
- [ ] Gibt es Tests fuer alle neuen Funktionen?
- [ ] Gibt es Tests fuer alle Edge Cases?
- [ ] Gibt es Tests fuer Fehlerfaelle?
- [ ] Laufen alle bestehenden Tests noch? (`go test ./...` / `pytest`)
- [ ] Ist die Test-Coverage angemessen?
## Tests ausfuehren
```bash
# Go - Alle Tests
cd consent-service && go test -v ./...
# Go - Mit Coverage
cd consent-service && go test -cover ./...
# Python - Alle Tests
cd backend && source venv/bin/activate && pytest -v
# Python - Mit Coverage
cd backend && pytest --cov=. --cov-report=html
```
## Beispiel: Vollstaendiger Test-Workflow
Wenn du z.B. eine neue `GetUserStats()` Funktion im Go Service hinzufuegst:
1. **Funktion schreiben** in `internal/services/stats_service.go`
2. **Test erstellen** in `internal/services/stats_service_test.go`:
```go
func TestGetUserStats_ValidUser_ReturnsStats(t *testing.T) {...}
func TestGetUserStats_InvalidUser_ReturnsError(t *testing.T) {...}
func TestGetUserStats_NoConsents_ReturnsEmptyStats(t *testing.T) {...}
```
3. **Tests ausfuehren**: `go test -v ./internal/services/...`
4. **Dokumentation aktualisieren** (siehe [Dokumentation](./documentation.md))
---
## Modul-spezifische Tests
### Canonical Control Generator (98+ Tests)
Die Control Library hat eine umfangreiche Test-Suite ueber 6 Dateien.
Siehe [Canonical Control Library — Tests](../services/sdk-modules/canonical-control-library.md#tests) und [Control Generator Pipeline](../services/sdk-modules/control-generator-pipeline.md) fuer Details.
```bash
# Alle Generator-Tests (98 Tests in 13 Klassen)
cd backend-compliance && pytest -v tests/test_control_generator.py
# Similarity Detector Tests
cd backend-compliance && pytest -v compliance/tests/test_similarity_detector.py
# API Route Tests
cd backend-compliance && pytest -v tests/test_canonical_control_routes.py
# License Gate Tests
cd backend-compliance && pytest -v tests/test_license_gate.py
# CI/CD Validator Tests
cd backend-compliance && pytest -v tests/test_validate_controls.py
```
**Wichtig:** Die Generator-Tests nutzen Mocks fuer Anthropic-API und Qdrant — sie laufen ohne externe Abhaengigkeiten.
**Testklassen in `test_control_generator.py`:**
| Klasse | Tests | Prueft |
|--------|-------|--------|
| `TestLicenseMapping` | 13 | Lizenz-Klassifikation (Rule 1/2/3), Case-Insensitivitaet, source_type |
| `TestDomainDetection` | 5 | Keyword-basierte Domain-Erkennung (AUTH, CRYP, NET, DATA) |
| `TestJsonParsing` | 4 | JSON-Parser fuer LLM-Responses (Markdown-Fencing, Preamble) |
| `TestGeneratedControlRules` | 3 | Rule-spezifische Felder (original_text, citation, source_info) |
| `TestAnchorFinder` | 2 | RAG-Suche + Web-Framework-Erkennung |
| `TestPipelineMocked` | 5 | End-to-End Pipeline mit Mocks (Lizenz, Hash-Dedup, Config) |
| `TestParseJsonArray` | 15 | JSON-Array-Parser (Wrapper-Objekte, Bracket-Extraction, Fallbacks) |
| `TestBatchSizeConfig` | 5 | Batch-Groesse-Konfiguration + Defaults |
| `TestBatchProcessingLoop` | 10 | Batch-Verarbeitung (Rule-Split, Mixed-Rules, Too-Close, Null-Handling) |
| `TestRegulationFilter` | 5 | regulation_filter Prefix-Matching, leere regulation_codes |
| `TestPipelineVersion` | 5 | pipeline_version=2 in DB-Writes, null-Handling in Structure/Reform |
| `TestRecitalDetection` | 10 | Erwaegungsgrund-Erkennung in Quelltexten (Regex, Phrasen, Kombiniert) |
| `TestSourceTypeClassification` | 16 | law/guideline/standard/restricted Klassifizierung aller Quellentypen |