Cap unbounded list endpoints with mandatory pagination #13

Open
opened 2026-04-20 09:35:51 +00:00 by sharang · 0 comments
Owner

Problem

List endpoints return all matching records without a limit cap:

  • canonical_control_routes.py:276-293limit parameter is optional with no enforced maximum
  • Similar pattern across controls, obligations, VVT entries, and audit trail routes

A single query against a large tenant could return 100,000+ rows, causing memory exhaustion and timeouts.

Required Actions

  1. Add a shared PaginatedResponse base schema to compliance/schemas/pagination.py:
class PaginatedResponse(GenericModel, Generic[T]):
    items: List[T]
    total: int
    offset: int
    limit: int
  1. Update all list endpoints to use limit: int = Query(100, ge=1, le=1000) — default 100, hard cap 1000
  2. Return PaginatedResponse from every list endpoint
  3. Update frontend SDK client to handle paginated responses

Acceptance Criteria

  • No list endpoint returns more than 1000 records in a single response
  • All list responses include total, offset, limit metadata
  • Frontend handles total > limit case (shows load-more or page controls)
## Problem List endpoints return all matching records without a limit cap: - `canonical_control_routes.py:276-293` — `limit` parameter is optional with no enforced maximum - Similar pattern across controls, obligations, VVT entries, and audit trail routes A single query against a large tenant could return 100,000+ rows, causing memory exhaustion and timeouts. ## Required Actions 1. Add a shared `PaginatedResponse` base schema to `compliance/schemas/pagination.py`: ```python class PaginatedResponse(GenericModel, Generic[T]): items: List[T] total: int offset: int limit: int ``` 2. Update all list endpoints to use `limit: int = Query(100, ge=1, le=1000)` — default 100, hard cap 1000 3. Return `PaginatedResponse` from every list endpoint 4. Update frontend SDK client to handle paginated responses ## Acceptance Criteria - No list endpoint returns more than 1000 records in a single response - All list responses include `total`, `offset`, `limit` metadata - Frontend handles `total > limit` case (shows load-more or page controls)
sharang added this to the M2: Data Integrity & Reliability milestone 2026-04-20 09:35:51 +00:00
sharang added the severity: mediumdata-integrity labels 2026-04-20 09:35:51 +00:00
Sign in to join this conversation.