feat(ai-sdk): authority-aware re-ranking for legal RAG (Phase 1) #31

Merged

Benjamin_Boenisch merged 1 commits from feat/authority-rerank into main 2026-06-23 09:30:53 +00:00

Conversation 0 Commits 1 Files Changed 9

+641 -2

Benjamin_Boenisch commented 2026-06-23 09:01:38 +00:00

Owner

Copy Link

Copy Source

Summary

Authority-aware re-ranking in the Go ai-sdk legal RAG retrieval path (Phase 1, no contract change). Binding law from the matching jurisdiction and domain now ranks above guidance, foreign and off-domain law, without dropping anything (guidance stays as interpretation context). Internal-only: the response schema is unchanged, so every consumer benefits without a contract change.

• authority.go: classify (authority/domain/scope/topic). Tagged payload (authority_weight/source_class/jurisdiction) wins; deterministic fallback via category and name markers for the not-yet-tagged corpus.
• authority_rerank.go: rerankByAuthority writes the authority score back into Score so the multi-collection advisor merge preserves the order.
• legal_rag_client: stratified retrieval — the binding-law pool AUGMENTS the semantic pool (mergeDedupHits), then re-rank.
• legal_rag_http: searchBinding (source_class filter) plus a shared doPointsSearch.
• legal_rag_types: three internal non-serialized fields so the response schema is unchanged.

Test plan

• go test -race ./internal/ucca — acceptance criteria as table-driven tests (guidance does not overtake competitive binding; foreign demoted and kept; off-domain demoted and kept; same-regime binding wins; BDSG Teil 3 below DSGVO; nothing dropped)
• go vet and go build green
• Live golden harness vs the tagged dev Qdrant: guidance-Top-1 = 0, foreign-Top-1 = 0, Top-1 6/9, Top-3 8/9, 438 ms per question (3 collections)
• dev smoke test after deploy

Phase 2 (separate PR): confidence/abstention safety fields (contract change) and interpretation-intent (guidance may be Top-1 for questions that explicitly ask for interpretation).

## Summary Authority-aware re-ranking in the Go ai-sdk legal RAG retrieval path (Phase 1, no contract change). Binding law from the matching jurisdiction and domain now ranks above guidance, foreign and off-domain law, without dropping anything (guidance stays as interpretation context). Internal-only: the response schema is unchanged, so every consumer benefits without a contract change. - authority.go: classify (authority/domain/scope/topic). Tagged payload (authority_weight/source_class/jurisdiction) wins; deterministic fallback via category and name markers for the not-yet-tagged corpus. - authority_rerank.go: rerankByAuthority writes the authority score back into Score so the multi-collection advisor merge preserves the order. - legal_rag_client: stratified retrieval — the binding-law pool AUGMENTS the semantic pool (mergeDedupHits), then re-rank. - legal_rag_http: searchBinding (source_class filter) plus a shared doPointsSearch. - legal_rag_types: three internal non-serialized fields so the response schema is unchanged. ## Test plan - [x] go test -race ./internal/ucca — acceptance criteria as table-driven tests (guidance does not overtake competitive binding; foreign demoted and kept; off-domain demoted and kept; same-regime binding wins; BDSG Teil 3 below DSGVO; nothing dropped) - [x] go vet and go build green - [x] Live golden harness vs the tagged dev Qdrant: guidance-Top-1 = 0, foreign-Top-1 = 0, Top-1 6/9, Top-3 8/9, 438 ms per question (3 collections) - [ ] dev smoke test after deploy Phase 2 (separate PR): confidence/abstention safety fields (contract change) and interpretation-intent (guidance may be Top-1 for questions that explicitly ask for interpretation).

Benjamin_Boenisch added 1 commit 2026-06-23 09:01:41 +00:00

feat(ai-sdk): authority-aware re-ranking for legal RAG retrieval (Phase 1) ... 49147d9497

Re-orders /sdk/v1/rag/search results so binding law from the matching
jurisdiction and domain ranks above guidance, foreign and off-domain law —
without dropping anything (guidance stays as interpretation context).
Internal-only: response schema is unchanged (json:"-" fields), so every
consumer benefits without a contract change.

- authority.go: classifyAuthority / queryDomain / chunkDomain / scopeClass /
  topic ontology. Tagged payload (authority_weight/source_class/jurisdiction)
  wins; deterministic fallback via category + name markers for the untagged corpus.
- authority_rerank.go: rerankByAuthority. final = semantic + authority +
  jurisdiction + domain + scope + topic; the authority score is written back to
  Score so the multi-collection advisor merge preserves the order.
- legal_rag_client: stratified retrieval — the binding-law pool AUGMENTS the
  semantic pool (mergeDedupHits), then re-rank.
- legal_rag_http: searchBinding (source_class filter) + shared doPointsSearch.
- table-driven tests for authority/domain/scope/topic + rerank acceptance +
  a stratified-binding integration test. go test -race green.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Benjamin_Boenisch merged commit a1f425d43a into main 2026-06-23 09:30:53 +00:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

Clear labels

config

Env vars, secrets management, startup validation

data-integrity

Transactions, indexes, pagination

frontend

Next.js, client-side concerns

observability

Logging, audit trails, health checks

reliability

Error handling, retries, circuit breakers

security

Auth, secrets, injection, tenant isolation

severity: critical

System-level risk, data breach or full outage

severity: high

Significant risk, must fix before go-live

severity: medium

Important but not blocking go-live

testing

Test coverage, integration tests

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

sharang (Sharang Parnerkar) Benjamin_Boenisch

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: Benjamin_Boenisch/breakpilot-compliance#31