Architecture

Subsystem map and the end-to-end lifecycle of a chat completion.

Scope: quill-router (this repo) at commit 778120494 (2026-10-19). All paths are relative to the repo root /Users/elevenarms/Code/trustedrouter/quill-router. The attested gateway is in a separate repo, quill-cloud-proxy, which is not checked out here. Anything this doc says about the gateway's internals comes from this repo's contract code, docs and comments, not from gateway source.

1. Overview in one page

TrustedRouter is an OpenAI-, Anthropic- and OpenRouter-compatible LLM router. Its main selling point is that the prompt path is verifiable: prompts go only through an attested enclave, and the control plane never sees prompt or output bytes. Three planes share the work:

The README states the trust boundary directly (README.md:127-131): "api.trustedrouter.com is the attested prompt path and must terminate TLS inside Confidential Space. trustedrouter.com is the control plane and must never serve a production inference fallback." The code enforces it: /chat/completions, /messages, /responses and /embeddings are mounted on the control plane only when environment in {local,test} (src/trusted_router/main.py:880-883, main.py:912-919).

What the control plane does for each production request. The enclave makes two or three server-to-server calls to the control plane's /internal/gateway/* routes (src/trusted_router/routes/internal/gateway.py:2453-2600):

1. authorize: validate the key hash, resolve the model and routing candidates (applying privacy filters), estimate cost, and reserve credits and key caps atomically in Spanner. It returns authorization_id, route_candidates, any BYOK ciphertext, and the regions.
2. Optional heartbeat (Stage D): usage heartbeats during long streams.
3. settle (success) or refund (failure or disconnect): book actual tokens to the hold, write the generation record, and enqueue analytics, activity, broadcast and auto-refill work.

Prompt and output bytes never cross this boundary. Only hashes, token counts and metadata do.

Storage in production is TR_STORAGE_BACKEND=spanner-clickhouse (scripts/deploy/rollout.sh:274). Spanner is the system of record for money and identity. ClickHouse holds request analytics, fed through a durable Spanner outbox. Postgres-wire backends (Azure/AWS standalone clouds) and an in-memory store (tests/local) implement the same Store protocol.

2. System context (C4 level 1)

flowchart LR
    dev["Developer app / SDK<br/>OpenAI, Anthropic or OpenRouter client"]
    browser["Browser user<br/>console, chat playground"]
    subgraph TR["TrustedRouter"]
        gw["Attested gateway<br/>api.trustedrouter.com<br/>quill-cloud-proxy in Confidential Space"]
        cp["Control plane<br/>trustedrouter.com<br/>FastAPI trusted_router.main:app on Cloud Run"]
        fp["Fast path pilot<br/>fastpath/ Go service, VPC only"]
        sp[("Cloud Spanner<br/>money, identity, outboxes")]
        ch[("ClickHouse<br/>request analytics")]
    end
    prov["Upstream LLM providers<br/>OpenAI, Anthropic, Vertex, Bedrock, Tinfoil, ..."]
    pay["Payments<br/>Stripe, Adyen, PayPal, Lightning, x402"]
    obs["Observability<br/>Sentry and Axiom, control plane only"]
    sinks["Customer broadcast sinks<br/>PostHog, OTLP webhooks"]

    dev -- "HTTPS, TLS ends in enclave" --> gw
    browser -- "HTTPS" --> cp
    gw -- "/internal/gateway authorize, settle, refund<br/>metadata only" --> cp
    gw -- "prompt and completion" --> prov
    cp -- "lease admit and settle, pilot only" --> fp
    cp --> sp
    fp --> sp
    cp -- "outbox drain" --> ch
    cp -- "checkout sessions" --> pay
    pay -- "signed webhooks" --> cp
    cp --> obs
    cp -- "metadata broadcast outbox" --> sinks
    cp -- "/chat-proxy byte pipe, browser only" --> gw

flowchart LR
    dev["Developer app / SDK<br/>OpenAI, Anthropic or OpenRouter client"]
    browser["Browser user<br/>console, chat playground"]
    subgraph TR["TrustedRouter"]
        gw["Attested gateway<br/>api.trustedrouter.com<br/>quill-cloud-proxy in Confidential Space"]
        cp["Control plane<br/>trustedrouter.com<br/>FastAPI trusted_router.main:app on Cloud Run"]
        fp["Fast path pilot<br/>fastpath/ Go service, VPC only"]
        sp[("Cloud Spanner<br/>money, identity, outboxes")]
        ch[("ClickHouse<br/>request analytics")]
    end
    prov["Upstream LLM providers<br/>OpenAI, Anthropic, Vertex, Bedrock, Tinfoil, ..."]
    pay["Payments<br/>Stripe, Adyen, PayPal, Lightning, x402"]
    obs["Observability<br/>Sentry and Axiom, control plane only"]
    sinks["Customer broadcast sinks<br/>PostHog, OTLP webhooks"]

    dev -- "HTTPS, TLS ends in enclave" --> gw
    browser -- "HTTPS" --> cp
    gw -- "/internal/gateway authorize, settle, refund<br/>metadata only" --> cp
    gw -- "prompt and completion" --> prov
    cp -- "lease admit and settle, pilot only" --> fp
    cp --> sp
    fp --> sp
    cp -- "outbox drain" --> ch
    cp -- "checkout sessions" --> pay
    pay -- "signed webhooks" --> cp
    cp --> obs
    cp -- "metadata broadcast outbox" --> sinks
    cp -- "/chat-proxy byte pipe, browser only" --> gw

Multi-cloud: each cloud (GCP, AWS, Azure) is a standalone TrustedRouter. Only identity federates; credits, keys and analytics stay in their own cloud (docs/storage-portability/multi-cloud-separation.md). Peer planes resolve keys lazily from the home plane (src/trusted_router/services/federation.py) and forward deferred debt home (src/trusted_router/services/home_settlement.py).

3. One image, six process "surfaces"

One container image serves several process roles, chosen by TR_SERVICE_SURFACE (src/trusted_router/config.py:438-445). create_app() registers only the routes that belong to the chosen surface (main.py:709-909).

The six-service split is in progress (issue #712 per config.py:446-449). Until it completes, the legacy combined service still runs in production next to the split public and internal companions.

4. Subsystem map

src/trusted_router/ has 371 top-level modules and 533 .py files once subpackages are included (routes/ 77, services/ 62, synthetic/ 22, content/, evals/). Grouped by prefix and responsibility:

Component diagram (control plane internals)

flowchart TB
    subgraph MW["middleware.py register_http_middleware"]
        m1["UnreadRequestBodyClose"] --> m2["security headers, CSP report-only"] --> m3["rate_limit, process-local"] --> m4["read_only guard"] --> m5["request_id"] --> m6["internal auth before body"] --> m7["GZip, MarkdownNegotiation"]
    end
    subgraph R["Routers, selected by TR_SERVICE_SURFACE"]
        pub["routes/public.py, catalog"]
        con["routes/console/*, keys, billing, byok, oauth"]
        gwr["routes/internal/gateway.py<br/>authorize, heartbeat, settle, refund"]
        inf["routes/inference.py<br/>local and test only"]
        hooks["routes/internal/webhook.py<br/>Stripe, Adyen, PayPal"]
    end
    auth["auth.py principal_from_request"]
    rt["routing.py and routing_candidates.py"]
    cat["catalog.py, catalog_registry.py, catalog_privacy.py<br/>MODELS, MODEL_ENDPOINTS built at import"]
    rs["routing_state.py ROUTING_STATE ranking hints"]
    fpc["services/fastpath_frontdoor.py"]
    store["storage.py STORE proxy implementing store_protocol.Store"]
    gcp["storage_gcp.py SpannerStore<br/>storage_gcp_authorize.py one-txn authorize"]
    pg["storage_postgres.py PostgresStore"]
    mem["storage.py InMemoryStore"]
    sob["storage_gcp_settle_outbox.py"]
    oa["operational analytics outbox"]
    bc["broadcast outbox"]
    sink["analytics_sink.py best-effort ClickHouse sink"]
    provs["providers.py and provider_adapters.py<br/>local inference only"]

    MW --> R
    con --> auth
    inf --> auth
    gwr --> rt
    inf --> rt
    rt --> cat
    gwr --> rs
    gwr --> fpc
    gwr --> store
    inf --> provs
    inf --> store
    con --> store
    hooks --> store
    store --> gcp
    store --> pg
    store --> mem
    gwr --> sob
    gcp --> oa
    gwr --> bc
    gwr --> sink

flowchart TB
    subgraph MW["middleware.py register_http_middleware"]
        m1["UnreadRequestBodyClose"] --> m2["security headers, CSP report-only"] --> m3["rate_limit, process-local"] --> m4["read_only guard"] --> m5["request_id"] --> m6["internal auth before body"] --> m7["GZip, MarkdownNegotiation"]
    end
    subgraph R["Routers, selected by TR_SERVICE_SURFACE"]
        pub["routes/public.py, catalog"]
        con["routes/console/*, keys, billing, byok, oauth"]
        gwr["routes/internal/gateway.py<br/>authorize, heartbeat, settle, refund"]
        inf["routes/inference.py<br/>local and test only"]
        hooks["routes/internal/webhook.py<br/>Stripe, Adyen, PayPal"]
    end
    auth["auth.py principal_from_request"]
    rt["routing.py and routing_candidates.py"]
    cat["catalog.py, catalog_registry.py, catalog_privacy.py<br/>MODELS, MODEL_ENDPOINTS built at import"]
    rs["routing_state.py ROUTING_STATE ranking hints"]
    fpc["services/fastpath_frontdoor.py"]
    store["storage.py STORE proxy implementing store_protocol.Store"]
    gcp["storage_gcp.py SpannerStore<br/>storage_gcp_authorize.py one-txn authorize"]
    pg["storage_postgres.py PostgresStore"]
    mem["storage.py InMemoryStore"]
    sob["storage_gcp_settle_outbox.py"]
    oa["operational analytics outbox"]
    bc["broadcast outbox"]
    sink["analytics_sink.py best-effort ClickHouse sink"]
    provs["providers.py and provider_adapters.py<br/>local inference only"]

    MW --> R
    con --> auth
    inf --> auth
    gwr --> rt
    inf --> rt
    rt --> cat
    gwr --> rs
    gwr --> fpc
    gwr --> store
    inf --> provs
    inf --> store
    con --> store
    hooks --> store
    store --> gcp
    store --> pg
    store --> mem
    gwr --> sob
    gcp --> oa
    gwr --> bc
    gwr --> sink

5. Request lifecycle: production /v1/chat/completions

5.1 Step by step

0. Client → enclave. The client calls https://api.trustedrouter.com/v1/chat/completions with Authorization: Bearer sk-tr-v1-…. TLS terminates inside the Confidential Space enclave. The enclave parses the body, derives estimated_input_tokens, input_modalities and requested_parameters, and hashes the API key. The raw key and the prompt stay inside the enclave.

1. Enclave → control plane POST /internal/gateway/authorize (routes/internal/gateway.py:2492). Body is GatewayAuthorizeRequest (src/trusted_router/schemas.py:366-431): api_key_lookup_hash, model, models, provider, token estimates, stream, region, tags, metadata, idempotency_key, route_type, and so on. - Middleware: internal_auth_before_body_middleware checks the internal token before the body is read (middleware.py:153-161, _internal_auth_before_body at middleware.py:473). Then request id and the process-local rate limit. - Per-key admission: _AUTHORIZE_ADMISSION.try_acquire(subject) returns 429 when one key has too many authorizations in flight. The handler then moves to a worker thread (gateway.py:534-587). All gateway storage IO is synchronous and runs through run_in_threadpool. - Internal auth: require_internal_gateway compares the bearer or x-trustedrouter-internal-token with TR_INTERNAL_GATEWAY_TOKEN in constant time, and is fail-closed outside local/test (routes/internal/_shared.py:53-65). - Key + workspace snapshot (gateway.py:856-940): one folded read loads the key, workspace, BYOK rows and boot record. It rejects disabled or expired keys (401) and wrong-scope keys, and verifies X-TR-Boot-Auth if present (gateway_boot.py). The workspace must be billing-active (assert_workspace_billing_active). Tags merge from the key and the request. - Idempotency replay: a matching idempotency_key plus request_fingerprint returns the original authorization, which prevents double reservation (gateway.py:~1200-1320). - Model resolution and routing: normalize_routing_inputs(body_dict, settings, resolved_region) (gateway.py:1321, routing.py:926) gives NormalizedRoutingInputs(model_ids, preferences).

   • provider_route_preferences (routing.py:633-746) validates every provider.* field. An unknown field returns 400 with the exact name. min_privacy maps through PRIVACY_TIER_ALIASES (catalog_data.py:61-89): any/standard=0, no_store=1, zdr=2, confidential/e2e/e2ee=3. A typo is a 400 so the request can't silently weaken. provider.zdr=true adds the ZDR requirement.
   • Meta-model aliases (trustedrouter/auto, zdr, e2e, eu, tiers, …) expand to ordered pools (routing_candidates.py:138-420). trustedrouter/zdr and trustedrouter/e2e carry implicit privacy floors (ROUTING_MODEL_MIN_PRIVACY_TIERS).
   • Dispatch is by route type: chat → chat_route_endpoint_candidates (routing.py:253-289), or embeddings, decide (fusion/selector), image, speech, transcription, video, fetch, user model (gateway.py:~1410-1470).
   • Privacy filter: _apply_endpoint_provider_filters keeps only endpoints where endpoint_meets_privacy_requirement(endpoint, req) holds for every requirement (catalog_privacy.py:231-251). Confidential needs verified confidential compute and provider E2EE and explicit ZDR, and never the model's own vendor. Unknown values fail closed. If nothing passes, the request gets a 400 ("No route candidates match the requested provider filters"). It never falls back to a weaker route.
   • data_collection=deny is a soft filter. It is relaxed when it would empty the set (_filter_candidates_soft_data_collection, routing.py:~1080-1106). min_privacy is never relaxed.
   • BYOK eligibility per candidate (_byok_configs_for_candidates, _eligible_gateway_endpoint_candidates), service-tier filtering, then ROUTING_STATE.rank(...) reorders using process-local latency/throughput hints (routing_state.py:71-179; hints only, never an authority). allow_fallbacks=false trims to one candidate (gateway.py:~1580-1597).
   • Estimate: _endpoint_cost_microdollars over the candidates (the max across candidates when auto fallback is possible), plus receipt fee, custom-model markup, app markup and additional hosted-tool cost (gateway.py:~1595-1690).
   • Fast path pilot: for pilot workspaces with credit-only candidates, _fastpath_admission calls the Go front door. If admitted, the typed store reserves nothing and the lease holds the money (gateway.py:~1820-1835, 5087).
   • Reserve (the money step): _typed_store.authorize_gateway_typed(...) (gateway.py:~1866, storage_gcp.py:5479, implemented in storage_gcp_authorize.py). One Spanner read-write transaction runs the idempotency read, then conditional credit DML on a credit shard, then pause/strict-budget checks, then conditional key-cap DML, then the tr_reservation INSERT, then the gateway_authorization INSERT (storage_gcp_authorize.py:1-14). Insufficient balance returns 402. If a peer plane has deferred settlement enabled, a local refusal may fall back to the home plane.
   • Response: GatewayAuthorizeData (schemas.py:325-348) with authorization_id, prospective generation_id, route_candidates (provider, endpoint, upstream model, inference location, BYOK ciphertext and cache key), regions, broadcast_destinations, usage_type, timing phases (key_lookup_ms, routing_ms, store_ms, post_commit_ms).

2. Enclave → provider. The enclave tries route_candidates in order. A failure before any output token moves on to the next candidate, and it records route_fallbacks and route_failures. It streams SSE back to the client and counts tokens.

3. Optional POST /internal/gateway/heartbeat (Stage D, gateway.py:611-680). Boot-signed usage heartbeats during long streams. If the final settle is lost, the reaper can book a reaped_snapshot charge from the frozen Stage D pricing snapshot (stage_d.py) instead of releasing the hold for free. Eligibility depends on a signed accepted-image policy (stage_d_policy.py).

4. Enclave → POST /internal/gateway/settle (gateway.py:2552) with GatewaySettleRequest (schemas.py:566+): authorization_id, actual input/output/cache/reasoning tokens, selected_model/selected_endpoint, finish_reason, streamed, elapsed_seconds, first_token_seconds, route_fallbacks, and client context. - AsyncSettlementRoute (routes/settlements.py) intercepts only when async-settle protection is on and the gateway sent X-TR-Settlement-Mode: async-v1|sync. Otherwise it is the legacy handler. - _settle_gateway_with_admission_sync (gateway.py:3721) applies per-key settle admission (503 + Retry-After when overloaded). Then _settle_gateway_authorization (gateway.py:3776):

   • One-shot: a settle on an already-settled authorization returns already_settled and never charges twice (_already_settled_gateway_data, gateway.py:5271).
   • Prompt-cache normalization (normalized_prompt_accounting, services/settle_outbox_apply.py:112). Anthropic input_tokens exclude cached tokens, while OpenAI and Gemini prompt counts include them.
   • Actual cost in microdollars from the authorized endpoint's frozen terms, then a Generation.from_settle_body(...) metadata record. No content.
   • Durable settle outbox (TR_SETTLE_OUTBOX_ENABLED=true in prod): enqueue the settle intent first (storage_gcp_settle_outbox.py), then typed_finalize_gateway_authorization atomically books usage, releases the unused hold, writes the bounded tr_generation record and the activity/analytics outbox rows, and marks the outbox row done. If finalize fails, the outbox drain (services/settle_outbox_drain.py) replays it. Deterministic generation IDs make replays idempotent.
   • Post-commit side effects: auto-refill scheduling (Stripe, owned by the control surface), creator/app/custom-model payouts, metadata broadcast enqueue, and benchmark samples to the best-effort analytics_sink (analytics_sink.py, a bounded queue that drops rows rather than block).
   • Fast path authorizations are settled through _settle_leased_authorization (gateway.py:5181).

5. Refund (POST /internal/gateway/refund, gateway.py:2561) uses the same _settle_gateway_authorization(success=False). It releases the reservation and key-cap hold after a provider failure or client disconnect before output. Expired holds are reaped (storage_gcp_authorize.py:1224), and that is why the settle outbox and Stage D exist.

6. Analytics. Spanner outbox rows are drained into ClickHouse by the clickhouse/ingest_* jobs, using one-way workspace/key surrogates (storage_gcp_operational_analytics_outbox.py:1-12). ClickHouse failures never affect inference or settle (docs/clickhouse-reliability.md).

5.2 Sequence diagram (streaming chat with settle)

sequenceDiagram
    autonumber
    participant C as Client SDK
    participant G as Attested gateway quill-cloud-proxy
    participant CP as Control plane internal surface
    participant RT as routing.py and catalog
    participant FP as fastpath front door pilot
    participant SP as Spanner
    participant P as Upstream provider
    participant CH as ClickHouse

    C->>G: POST /v1/chat/completions stream true, Bearer sk-tr
    Note over G: TLS ends in enclave, key hashed, tokens estimated
    G->>CP: POST /internal/gateway/authorize key hash, model, provider prefs, estimates
    CP->>CP: internal token check, per-key admission, run_in_threadpool
    CP->>SP: folded read of key, workspace, BYOK, boot
    CP->>RT: normalize_routing_inputs, min_privacy filter, sort, rank
    RT-->>CP: ordered route candidates or 400 fail closed
    opt pilot workspace
        CP->>FP: admit under lease
        FP-->>CP: admitted or off, busy, invalid
    end
    CP->>SP: one RW txn: credit shard DML, key cap DML, tr_reservation, authorization
    SP-->>CP: committed or 402 insufficient credits
    CP-->>G: authorization_id, generation_id, route_candidates, BYOK ciphertext
    loop candidates until first token
        G->>P: provider request with operator key or decrypted BYOK
        P-->>G: error before output, so try next candidate
    end
    P-->>G: SSE chunks
    G-->>C: SSE chunks then DONE
    opt Stage D long stream
        G->>CP: POST /internal/gateway/heartbeat boot signed usage
    end
    alt success
        G->>CP: POST /internal/gateway/settle actual tokens, selected endpoint, timings
        CP->>SP: enqueue settle outbox intent
        CP->>SP: typed finalize: book usage, release remainder, generation record, analytics outbox
        CP-->>G: settled, or already_settled on retry
    else provider failure or disconnect
        G->>CP: POST /internal/gateway/refund
        CP->>SP: release reservation and key cap hold
    end
    CP-)CH: outbox drain jobs and best-effort benchmark sink

sequenceDiagram
    autonumber
    participant C as Client SDK
    participant G as Attested gateway quill-cloud-proxy
    participant CP as Control plane internal surface
    participant RT as routing.py and catalog
    participant FP as fastpath front door pilot
    participant SP as Spanner
    participant P as Upstream provider
    participant CH as ClickHouse

    C->>G: POST /v1/chat/completions stream true, Bearer sk-tr
    Note over G: TLS ends in enclave, key hashed, tokens estimated
    G->>CP: POST /internal/gateway/authorize key hash, model, provider prefs, estimates
    CP->>CP: internal token check, per-key admission, run_in_threadpool
    CP->>SP: folded read of key, workspace, BYOK, boot
    CP->>RT: normalize_routing_inputs, min_privacy filter, sort, rank
    RT-->>CP: ordered route candidates or 400 fail closed
    opt pilot workspace
        CP->>FP: admit under lease
        FP-->>CP: admitted or off, busy, invalid
    end
    CP->>SP: one RW txn: credit shard DML, key cap DML, tr_reservation, authorization
    SP-->>CP: committed or 402 insufficient credits
    CP-->>G: authorization_id, generation_id, route_candidates, BYOK ciphertext
    loop candidates until first token
        G->>P: provider request with operator key or decrypted BYOK
        P-->>G: error before output, so try next candidate
    end
    P-->>G: SSE chunks
    G-->>C: SSE chunks then DONE
    opt Stage D long stream
        G->>CP: POST /internal/gateway/heartbeat boot signed usage
    end
    alt success
        G->>CP: POST /internal/gateway/settle actual tokens, selected endpoint, timings
        CP->>SP: enqueue settle outbox intent
        CP->>SP: typed finalize: book usage, release remainder, generation record, analytics outbox
        CP-->>G: settled, or already_settled on retry
    else provider failure or disconnect
        G->>CP: POST /internal/gateway/refund
        CP->>SP: release reservation and key cap hold
    end
    CP-)CH: outbox drain jobs and best-effort benchmark sink

5.3 The local/test path (same semantics, no enclave)

In local/test, routes/inference.py mounts /chat/completions and the other inference routes (routes/inference.py:150-420):

• InferencePrincipal → require_inference_key → principal_from_request (auth.py:90-177, auth.py:361).
• provider_route_preferences + chat_route_candidates or chat_route_endpoint_candidates, then _local_routes_meeting_privacy_floor (routes/inference.py:106).
• More than one candidate, or trustedrouter/auto, means meta-route rollover (run_chat_candidates or _candidate_stream_bytes). Otherwise a single candidate (run_chat or run_chat_stream).
• services/inference.py:_run_stream (services/inference.py:162-218) opens reserved_quota(...) (services/inference_quota.py:84), which reserves the key limit and then credits. It streams from ProviderClient.stream_chat, computes the actual cost, calls ticket.settle(actual_cost), and runs STORE.add_generation(...). An exception refunds (QuotaTicket, inference_quota.py:42-80).
• Live upstream calls happen only with TR_ENABLE_LIVE_PROVIDERS=true (default false, config.py:486) and keys from the local keys file.

5.4 Anthropic /v1/messages

• Client config: ANTHROPIC_BASE_URL=https://api.trustedrouter.com (no /v1, the SDK adds it), ANTHROPIC_API_KEY=sk-tr-… (README.md:62-64).
• Production: the enclave serves /v1/messages. The control-plane contract is the same authorize/settle/refund. Model and provider filters go through the same chat_route_endpoint_candidates and privacy logic. The settle path normalizes Anthropic cache accounting, where input_tokens excludes cache_read_input_tokens and cache_creation_input_tokens (schemas.py:574-578). Coverage registry: openrouter_coverage.py:99.
• Local/test (routes/inference.py:307-351): _require_messages_model (:929), then the privacy floor, then messages_to_chat_body (adapter.py:83-99), which turns Anthropic system into a leading system chat message and copies allowed params. The request then runs on the same run_chat / run_messages_stream, and the result is wrapped by _anthropic_messages_envelope (routes/inference.py:506). On the outbound side, chat_to_anthropic (adapter.py:102+) rebuilds Anthropic payloads for the Anthropic provider adapter (provider_adapters.py:338-465).
• /v1/responses works the same way through responses_to_chat_body and _responses_api_envelope.

6. Control plane vs attested gateway vs fastpath/

flowchart LR
    subgraph ENC["Attested gateway, quill-cloud-proxy"]
        e1["TLS termination and attestation endpoint /attestation"]
        e2["Parse request, estimate tokens, hash key"]
        e3["Provider calls, fallback loop, SSE streaming"]
        e4["Decrypt BYOK in memory, short TTL cache"]
        e5["Inference receipts signed with boot key"]
    end
    subgraph CPL["Control plane, this repo"]
        c1["Key and workspace validation"]
        c2["Model, alias and privacy routing decision"]
        c3["Reserve, settle, refund in Spanner"]
        c4["Generation records, activity, analytics outbox"]
        c5["Console, billing, catalog, status, trust page"]
    end
    subgraph FPL["fastpath, Go, pilot"]
        f1["Front door admits under regional lease"]
        f2["Owner and auditor batch settlement"]
        f3["TLA plus shadows in proofs/"]
    end
    e2 -- "authorize" --> c1
    c1 --> c2 --> c3
    e3 -- "settle or refund" --> c3
    c3 --> c4
    c3 -- "pilot workspaces only" --> f1
    f1 --> f2

flowchart LR
    subgraph ENC["Attested gateway, quill-cloud-proxy"]
        e1["TLS termination and attestation endpoint /attestation"]
        e2["Parse request, estimate tokens, hash key"]
        e3["Provider calls, fallback loop, SSE streaming"]
        e4["Decrypt BYOK in memory, short TTL cache"]
        e5["Inference receipts signed with boot key"]
    end
    subgraph CPL["Control plane, this repo"]
        c1["Key and workspace validation"]
        c2["Model, alias and privacy routing decision"]
        c3["Reserve, settle, refund in Spanner"]
        c4["Generation records, activity, analytics outbox"]
        c5["Console, billing, catalog, status, trust page"]
    end
    subgraph FPL["fastpath, Go, pilot"]
        f1["Front door admits under regional lease"]
        f2["Owner and auditor batch settlement"]
        f3["TLA plus shadows in proofs/"]
    end
    e2 -- "authorize" --> c1
    c1 --> c2 --> c3
    e3 -- "settle or refund" --> c3
    c3 --> c4
    c3 -- "pilot workspaces only" --> f1
    f1 --> f2

The long-term design (docs/design/fast-admission-and-batched-settlement.md) has the enclave call the fast path directly, removing the control plane from request latency. That has not been built.

7. Must-read files (in reading order)

8. Glossary

9. Gotchas

1. Prod control plane does not serve inference. _control_plane_inference_enabled returns true only for local/test (main.py:912-919). If you debug a prod chat bug in routes/inference.py, you are reading the wrong code. Prod goes through the enclave → routes/internal/gateway.py.
2. /chat-proxy/v1/chat/completions terminates TLS on the control plane. It forwards bytes unparsed to the attested API for the browser playground only (routes/chat_proxy.py:1-30). SDK traffic must never use it.
3. The STORE proxy. monkeypatch.setattr(STORE, …) poisons later tests (once 136 failures). Patch the backend class (AGENTS.md:42-45). typed_billing_store() exists because isinstance on the proxy lies (storage.py:3954-3968).
4. The catalog is built at import time. MODELS and MODEL_ENDPOINTS come from data/*.json when the module loads. An hourly price-refresh bot commits to main, so rebase before trusting a failure (AGENTS.md:47-49).
5. Money code needs differential/conformance tests. Run the proof oracle when touching settle/billing: uv run pytest -q -n 4 -m proof_oracle tests/test_async_settle_proof_oracle.py (AGENTS.md:39-40, 81-87).
6. Async settle is huge and dormant. 98 modules (async_settle_shadow_v2_* alone is 86) are shadow/evidence machinery behind TR_ASYNC_SETTLE_ENABLED=false. Do not mistake it for the live settle path, which is _settle_gateway_authorization.
7. Config is code. Cloud Run env vars come from the ENV_VARS arrays in scripts/deploy/rollout.sh and the companion surface scripts. Setting them by hand is overwritten on the next rollout. Pushing src/** to main auto-deploys (AGENTS.md:60-66).
8. Combined surface is fail-closed in deploys. A missing TR_SERVICE_SURFACE defaults to combined, and that raises outside local/test unless the migration bridge flag is set (main.py:235-244, rollout.sh:12-15).
9. Gateway storage is synchronous. Every gateway handler runs in run_in_threadpool to avoid head-of-line blocking. AnyIO's 40-token pool is shared, which is safe only because Cloud Run concurrency is ~8 (gateway.py:2454-2467).
10. Rate limiting is process-local by design (middleware.py:144-146). Fleet-wide limits belong at the front door. (The README's "rate limits use the configured store" line is out of date.)
11. min_privacy vs data_collection. The first is a hard filter and the second is soft. ZDR and confidential compute are separate dimensions: a TEE without a retention promise does not satisfy ZDR (routing.py:1131-1135).
12. Anthropic cache-token semantics differ. Anthropic's input_tokens excludes cached tokens, while OpenAI and Gemini include them. Settle normalizes this (schemas.py:574-578, settle_outbox_apply.py:112).
13. Never scan tr_entities.body in prod Spanner, even read-only with LIMIT. Use scripts.inspect_payment_owner and ClickHouse (AGENTS.md:70-79). A body scan caused the September 6 CPU incident.
14. The README mentions Bigtable for activity. The current write path is the Spanner tr_generation bounded record plus the Spanner analytics outbox → ClickHouse (storage_gcp_generation_records.py:1-6, docs/storage-portability/analytics-ingestion.md). Treat Bigtable references as historical unless the code says otherwise.
15. Unknown routing fields return 400 and unimplemented OpenRouter fields return 501 not_supported_in_alpha. Nothing is silently ignored (README.md:334-336).
16. Each cloud is its own TrustedRouter. An AWS-issued key does not work on GCP, and credits don't move without an explicit transfer (multi-cloud-separation.md).
17. Analytics must never affect money. The benchmark sink drops rows when its queue is full, and the outbox enqueue rides the settle commit only on the one-commit happy path (AGENTS.md:83-87, analytics_sink.py:1-25).