Contributing & tests

The three CI gates, test taxonomy, PR conventions, and your first PR.

Scope: how code gets from an idea to production in quill-router. This covers the three CI gates, the test suite's structure, branch and commit habits, how review works (mostly coding agents), and a first-PR walkthrough with a checklist.

All paths are relative to quill-router/. Repo state: main @ 778120494, 2026-10-19.

1. The non-negotiables (from AGENTS.md)

AGENTS.md is the entry point for every coding agent and human (CLAUDE.md:3 only points at it). These are its hard rules:

The STORE pollution trap (issue #333)

STORE is a module-level _StoreProxy (src/trusted_router/storage.py:3880, :3947). It forwards calls to a backend that can be swapped. monkeypatch.setattr(STORE, "x", fake) creates an instance attribute that shadows __getattr__. On teardown, monkeypatch "restores" a bound method of the old backend, which silently breaks later tests. It once broke 136 unrelated tests (issue #333, "Inject the store instead of importing the STORE module global"). Patch the backend class (for example InMemoryStore.create_workspace), never the proxy. The conftest does this itself and says so in a comment (tests/conftest.py:163-215, especially :179). An autouse reset_store fixture resets the in-memory store before every test (tests/conftest.py:105-109).

2. Tool configuration (pyproject.toml)

xdist and --dist loadgroup. CI runs -n 4 --dist loadgroup. The loadgroup part matters: each server-backed conformance backend (postgres, spanner-pg) is pinned to one xdist_group, so its DDL runs serially (tests/conformance/conftest.py:225-241, ci.yml:227-232).

.test_durations sharding. pytest-split splits the suite using recorded per-test durations (38,116 entries, about 49k test-seconds). Splitting by test count produced 7 m vs 21 m shards (ci.yml:244-248). The file must be recorded on CI runners. Local runs skip the conformance suites, record them as ~0 s, and unbalance the shards (.github/workflows/refresh-test-durations.yml:1-12). To refresh it, dispatch that workflow, or merge shard artifacts with scripts/ci/merge_shard_durations.py (ci.yml:178). Example: PR #1816 "ci: refresh .test_durations from main run 38858291020's shard artifacts".

3. CI anatomy (.github/workflows/ci.yml)

Triggers: push to main, PRs to main, and workflow_dispatch. The price bot uses dispatch because GITHUB_TOKEN pushes don't trigger workflows (ci.yml:3-11). The jobs run in parallel, so wall time is the slowest job (ci.yml:13-16).

Other workflows that matter to contributors: - deploy.yml. Push to main touching src/**, scripts/deploy/**, scripts/entrypoint.sh, frontend/src/**, Dockerfile, .gcloudignore, pyproject.toml, uv.lock or deploy.yml auto-deploys to Cloud Run (deploy.yml:28-37). Its gate-on-ci job polls the CI run for the SHA and refuses to deploy unless it is green (:57-80). A hotfix input can skip the gate. A supersede check skips stale deploys (:424-445). - proofs-guards.yml. Runs only when proofs/** changes. 16 shards re-run every guard-table row (guard_sweep.py --verify): each guard is removed and must break exactly what its row claims (proofs-guards.yml:1-55). - typed-audit.yml. Daily at 11:43 UTC. Not a PR gate. Runs scripts/audit_typed_counters.py against prod via WIF, plus a Spanner schema-drift audit against the emulator (typed-audit.yml:1-70). - refresh-prices.yml. Hourly at :37. Scrapes provider prices. A sandboxed, LLM-rewritable parser tier is AST-whitelisted. The bot commits the snapshot to main (refresh-prices.yml:1-30). That accounts for the [bot] hourly price refresh: … commits by trustedrouter-pricing-bot (163 commits in 60 days). - refresh-test-durations.yml (manual), dependency-scan.yml (OSV), and Dependabot (.github/dependabot.yml).

4. Dev loop

flowchart TD
    A[git fetch and rebase on origin/main] --> B[branch: codex/..., fastpath/..., fix/...-YYYYMMDD, jasmine/...]
    B --> C[Code and tests. Patch backend classes, not STORE]
    C --> D{"3 local gates:<br/>uv run ruff check .<br/>uv run mypy<br/>uv run pytest -q (FULL suite)"}
    D -- red --> C
    D -- "touch settle/billing?" --> D2["uv run pytest -q -n 4 -m proof_oracle<br/>tests/test_async_settle_proof_oracle.py"]
    D2 --> E
    D -- green --> E[Agent review rounds: Codex or Claude review. Fix findings, add commits]
    E --> F[gh pr create: summary, verification, evidence]
    F --> G[CI: lint, frontend, production-import, spanner-emulator,<br/>test x12, coverage>=70, post-cutover x8, proof-oracle x12,<br/>browser, byok, fastpath Go, proofs TLC]
    G -- red --> R{Failing on main too?}
    R -- "no, branch is stale" --> A
    R -- "yes, real failure" --> C
    G -- green --> H[Self-merge: squash, usually by author]
    H --> I{Changed paths match deploy.yml filter?<br/>src/**, scripts/deploy/**, frontend/src/**, Dockerfile, pyproject, uv.lock}
    I -- no --> J[Done. Docs-only changes don't deploy]
    I -- yes --> K[deploy.yml: gate-on-ci waits for green CI on SHA,<br/>build image, staged regional rollout, watchdog]
    K --> L[Verify behavior and prod health.<br/>Never call a partial rollout 'deployed']

flowchart TD
    A[git fetch and rebase on origin/main] --> B[branch: codex/..., fastpath/..., fix/...-YYYYMMDD, jasmine/...]
    B --> C[Code and tests. Patch backend classes, not STORE]
    C --> D{"3 local gates:<br/>uv run ruff check .<br/>uv run mypy<br/>uv run pytest -q (FULL suite)"}
    D -- red --> C
    D -- "touch settle/billing?" --> D2["uv run pytest -q -n 4 -m proof_oracle<br/>tests/test_async_settle_proof_oracle.py"]
    D2 --> E
    D -- green --> E[Agent review rounds: Codex or Claude review. Fix findings, add commits]
    E --> F[gh pr create: summary, verification, evidence]
    F --> G[CI: lint, frontend, production-import, spanner-emulator,<br/>test x12, coverage>=70, post-cutover x8, proof-oracle x12,<br/>browser, byok, fastpath Go, proofs TLC]
    G -- red --> R{Failing on main too?}
    R -- "no, branch is stale" --> A
    R -- "yes, real failure" --> C
    G -- green --> H[Self-merge: squash, usually by author]
    H --> I{Changed paths match deploy.yml filter?<br/>src/**, scripts/deploy/**, frontend/src/**, Dockerfile, pyproject, uv.lock}
    I -- no --> J[Done. Docs-only changes don't deploy]
    I -- yes --> K[deploy.yml: gate-on-ci waits for green CI on SHA,<br/>build image, staged regional rollout, watchdog]
    K --> L[Verify behavior and prod health.<br/>Never call a partial rollout 'deployed']

5. Test taxonomy

tests/ has 922 entries: 855 test_*.py files at the top level, ~60 support modules (fakes, mutation tables, fixture generators, oracles), and 6 subdirectories. About 36.7k tests pass locally (PR #1860 body: "36,751 passed").

By area (top-level test_*.py, keyword buckets, counts approximate)

Subdirectories and adjacent suites

What "proofs" means here

A spec covers a protocol whose failures are interleavings, cases a unit test can't reach. The spec is written before the code (proofs/README.md:5-7). Every spec needs: - a .cfg naming each invariant - small bounds, stated - no CONSTRAINT - TypeOK first - liveness where the design promises progress - assumptions listed, each with a widening mutant (proofs/README.md:39-55)

manifest.toml lists mutants: one text replacement that must violate a named invariant. If nothing can break a claim, the claim isn't shown to say anything (proofs/README.md:60-77). *.guards.toml tables say which guard protects which claim. proofs-guards.yml proves each table is true. The same idea of mutation tables shows up in the Python tests (tests/async_settle_mutations.py, tests/gateway_edge_mutations.py, …).

6. Branch, commit and PR conventions

Who commits (last 60 days)

Joseph Perla 1,181 · Jasmine Lee 190 (+18 as jasminelee) · trustedrouter-pricing-bot 163 · dependabot[bot] 13.

Branch names (remote branch prefixes)

codex/ 359 · fastpath/ 66 · fix/ 39 · async-settle/ 36 · fable/ 20 · test/ 15 · infra/ 11 · feat/ 11 · ci/ 11 · dependabot/ 10. Also jasmine/<topic> for Jasmine's work.

• codex/<topic>-YYYYMMDD is agent-driven work, e.g. codex/bedrock-provider-20261018, codex/transcription-20261015.
• fix/<topic>-YYYYMMDD is a dated bug fix, e.g. fix/settlelog-cancel-release-20261018.
• <subsystem>/<topic> covers longer programs, e.g. fastpath/edge-e1-role, async-settle/shadow-v2-opt-in.
• Generated chores: deprecations-202610190004, featherless-stepfun-202610180100, deps/security-bumps.

Commit subjects

Two styles coexist. Most commits (~410 of the last 800 non-merge commits) are plain imperative sentences. The rest use a lowercase subsystem prefix (fastpath: 59, infra: 25, docs: 19, catalog: 17, test(s): 31, deploy: 12, async settle shadow v2: 12, fix: 12, pricing: 11, ci: 8, proofs: 8). Squash merges add (#NNNN). Design-program commits cite the design section and step: (design 15.8 step 5) points to docs/design/async-settle-shadow-v2-activation.md §15.8. (E1, second increment) refers to staged increments.

Real examples:

deploy: shadow v2 opt-in pins for the probe workspace (design 15.8 step 5) (#1853)
tests: fake Spanner reads hold the commit lock (#1860)
service: the edge as a part of the node (E1, second increment) (#1859)
catalog: add frontier, advanced, professional, efficient routing tiers (#1855)
store: refuse a terminal retried after its lease closed (#1843)
infra: pin the fast path release d99cdc049 (a retried terminal after close answers from the disposition) (#1851)
fix(bedrock): require explicit upgraded gateway origin
Schedule provider model retirements from notices received 2026-10-18 (#1852)
Serve the landscape homepage unconditionally and remove the old one
ci: refresh .test_durations from main run 38858291020's shard artifacts (#1816)
[bot] hourly price refresh: 2026-10-18T17:35Z

Commit bodies are long, plain prose: the mechanism, then the evidence. Agent-authored commits end with Co-Authored-By: Claude …. Follow-up commits on a PR often record review rounds, for example (from 28e447a0b):

Review round one (Codex, 2026-10-19), five findings, all closed: the edge
listens and checks its upstream before admission takes the ring row, ...

PR bodies

There's no PR template. The common shape, drawn from #1853, #1858, #1860 and #1857: 1. What changed and why. Root cause for fixes ("Fix (the class, not the line)", #1860). 2. Tests and verification with numbers. "On the old fake, both tests fail with the exact CI error. With the fix, both pass", "The full suite passes locally: 36,751 passed", "293 focused control-plane tests passed". 3. Explicit integrity statements. "No test was skipped, retried or loosened." 4. Rollout notes and opt-out for deploy-affecting changes ("Opt-out: empty the four lines", #1853). 5. Hold markers and companion PRs, e.g. "DO NOT MERGE UNTIL GATEWAY VERIFIED. Companion gateway: Lore-Hex/quill-cloud-proxy#535" (#1858). The author then posts release-evidence comments with CI run URLs before merging. 6. UI PRs embed QA screenshots committed under docs/homepage/qa/ (#1857). 7. Agent-generated PRs end with 🤖 Generated with [Claude Code](https://claude.com/claude-code).

Review culture: who reviews

• .github/CODEOWNERS assigns * @jperla and says so directly: the company "currently has one person in it… this file becomes load-bearing at first hire".
• No branch protection and no rulesets on main (gh api …/branches/main/protection returns 404, rulesets []). Every one of the last 40 merged PRs had 0 GitHub reviews and was merged by its author (jperla or jasminelee).
• The substitutes are agent review rounds and CI gates. Claude (often "Claude Fable" per the co-author trailers) writes code. Codex reviews it in numbered rounds, and each round's findings are closed in follow-up commits ("Review round ten (Codex, 2026-10-19)…"). Some PR bodies log P1/P2 findings per round: "round 1 … P1 none, one P2; round 2: APPROVE" (#1853). The controls that don't depend on headcount are gated CI and measured or attested deployments (CODEOWNERS header, SECURITY.md).
• Bots: trustedrouter-pricing-bot (hourly prices; CI is dispatched on its branch before main advances), Dependabot, and generated deprecation PRs (deprecations-YYYYMMDDhhmm).

Agent tooling in the repo

• AGENTS.md is the shared instructions for codex-cli and Claude Code. CLAUDE.md defers to it and repeats the Spanner query safety rule.
• .claude/launch.json holds one launch config: uv run uvicorn trusted_router.main:app --port 18080.
• skills/trustedrouter-model-advisor/SKILL.md is a pointer skill. The canonical copy lives in Lore-Hex/LLM-advisor and is also served through the TrustedRouter MCP server's Skills Extension. It's a product skill for end-user agents, not a dev-workflow skill.
• Some deep areas have their own handoff docs that agents must read first: docs/storage-portability/HANDOFF.md (AGENTS.md:10-23) and sites/token-exchange/ADDING-MARKETS.md (AGENTS.md:5-8).

7. Your first PR: walkthrough

Worked example: a small, non-money change in src/trusted_router.

1. Setup. Install uv and Node. Run uv sync and npm ci. If you'll touch browser tests, also run npx playwright install chromium. Start the app with uv run uvicorn trusted_router.main:app --reload (README.md:134-139).
2. Read AGENTS.md. If the area has a handoff or design doc (docs/design/…, docs/storage-portability/HANDOFF.md), read that too.
3. Branch from a fresh main. Run git fetch origin && git switch -c fix/<topic>-$(date +%Y%m%d) origin/main.
4. Write the failing test first. Put it in the matching tests/test_<area>_*.py (look at the bucket table above). Use the client, user_headers, inference_key and inference_headers fixtures (tests/conftest.py:217-260). If you must fake storage behavior, patch InMemoryStore.<method>, not STORE.<method>. For flaky or concurrency bugs, force the interleaving deterministically instead of relying on timing (the PR #1860 pattern).
5. Implement. Fully type every new function: mypy's disallow_untyped_defs applies to src/. Keep lines reasonable, though E501 is off. Don't reformat files you didn't change.
6. Run the three gates. Use uv run ruff check . && uv run mypy && uv run pytest -q. Add -n auto to speed up the full run. If ruff check complains, fix it or run uv run ruff check --fix <your files>, never ruff format ..
7. Extra gates by area: - Settle or billing: uv run pytest -q -n 4 -m proof_oracle tests/test_async_settle_proof_oracle.py. - Store or backends: start the Postgres/PGAdapter containers and set TR_CONFORMANCE_*_DSN (AGENTS.md:95-103). - Frontend TS/CSS: npx tsc, npx eslint frontend/src, npx stylelint "src/trusted_router/static/*.css", npm run test:browser. - Catalog data: uv run python scripts/update_model_changes.py --check. - proofs/: ./proofs/check.sh and python3 proofs/guard_sweep.py --verify <Spec>.
8. Get a review round. Ask Codex or Claude to review the diff. Address findings in follow-up commits that name the round.
9. Open the PR with gh pr create. Body: what and why, root cause, tests with real numbers, "nothing skipped or loosened", and rollout or opt-out notes.
10. CI red? Rebase onto origin/main and re-run before you debug. A failure that also happens on main is not yours, but tell someone, because it blocks every deploy.
11. Merge (squash). If your paths match the deploy.yml filter, the merge deploys to production. Watch the deploy run and verify the behavior and health in prod (AGENTS.md:53-58). Runtime env vars only change through scripts/deploy/rollout.sh ENV_VARS (AGENTS.md:64-66).

8. PR checklist

• [ ] Rebased on current origin/main
• [ ] uv run ruff check . is clean (and I did not run ruff format .)
• [ ] uv run mypy is clean, with all new src/ functions typed
• [ ] uv run pytest -q passes on the full suite, not only my file
• [ ] No monkeypatch.setattr(STORE, …). Backend classes are patched instead (#333)
• [ ] Coverage won't drop below 70%: new code has tests
• [ ] Settle or billing touched: -m proof_oracle run, plus a differential or conformance test added
• [ ] Store touched: conformance passes on memory/postgres/spanner-pg, and no conformance test weakened
• [ ] Frontend touched: .ts compiled and the .js committed, eslint, stylelint and Playwright pass
• [ ] Catalog or model data touched: scripts/update_model_changes.py --check passes
• [ ] proofs/ touched: check.sh and the guard sweep pass, with each new claim backed by a mutant
• [ ] No ad hoc production Spanner scans were used in the investigation (AGENTS.md:68-79)
• [ ] Runtime config changed through scripts/deploy/rollout.sh ENV_VARS, not by hand
• [ ] PR body covers what and why, root cause, verification numbers, "nothing skipped or loosened", and rollout or opt-out notes
• [ ] Companion PRs (e.g. quill-cloud-proxy) linked, and hold noted if ordering matters
• [ ] At least one agent review round done, with findings closed
• [ ] I know whether this merge auto-deploys (path filter), and I'll verify prod afterwards