Skip to main content

​
CLARITY COMMISSION OS

​
ENTERPRISE ENGINEERING & GOVERNANCE CHARTER

​
I. SYSTEM CLASSIFICATION

Clarity is classified as:
  • Financial processing system
  • Multi-tenant SaaS
  • Commission calculation engine
  • Audit-bound data pipeline
Therefore:
  • Data integrity > velocity
  • Determinism > convenience
  • Explicit failure > silent fallback
  • Traceability > abstraction elegance

​
II. CHANGE CONTROL FRAMEWORK

Every change must be classified before implementation.

​
Change Risk Levels

​
Level 0 — Cosmetic / UI only

  • Styling
  • Copy
  • Non-functional UX changes
Deploy Surface: Frontend only Risk: Low

​
Level 1 — UX Logic / Wizard Semantics

  • Gating logic
  • Readiness display
  • Draft store behavior
Deploy Surface: Frontend Risk: Medium Requires:
  • Regression tests
  • Network proof validation

​
Level 2 — Backend Orchestration

  • Readiness logic
  • Scope derivation
  • Materialization
  • Preflight classification
  • /process orchestration
Deploy Surface: Backend (Cloud Run) Risk: High Requires:
  • Test updates
  • Invariant validation
  • Structured error consistency
  • BigQuery verification proof

​
Level 3 — Money Engine / Stage Writes

  • Commission calculation logic
  • Stage1/Stage2/Stage3 write behavior
  • Idempotency logic
Deploy Surface: Backend + Data Risk: Critical Requires:
  • Deterministic proof
  • Replay test
  • Double-run test (must produce identical outputs)
  • Decimal verification
  • Snapshot diff check

​
Level 4 — Schema / Contract Change

  • BigQuery schema modifications
  • API contract changes
  • Batch ID semantics
  • Scope semantics
Deploy Surface: Backend + Data + possibly Frontend Risk: Severe Requires:
  • ADR document
  • Migration plan
  • Backfill plan
  • Rollback strategy
  • Explicit approval

​
III. ARCHITECTURAL DECISION RECORD (ADR) REQUIREMENT

An ADR must be created when:
  • Changing scope semantics
  • Changing readiness behavior
  • Changing /process behavior
  • Modifying batch_id lifecycle
  • Introducing new money math logic
  • Changing idempotency guarantees
ADR format must include:
  1. Problem statement
  2. Invariants affected
  3. Alternative solutions considered
  4. Risk analysis
  5. Rollback plan
  6. Test strategy

​
IV. DEPLOYMENT GATES (ENFORCED)

​
Backend Gate

Must pass:
  • All readiness tests
  • Scope mismatch tests
  • Mixed scope tests
  • Materialization failure tests
  • Idempotency tests (if /process touched)
Must verify:
  • Cloud Run revision SHA
  • Debug proof (if readiness involved)
  • BigQuery proof when writes involved

​
Frontend Gate

Must pass:
  • Smoke suite
  • Gating regression tests
  • No removed endpoints still being called
  • Production build hides dev panels

​
V. MONEY ENGINE CONTROL REQUIREMENTS

The commission engine must:
  1. Use Decimal exclusively.
  2. Produce deterministic outputs across re-runs.
  3. Produce identical outputs across environment replay.
  4. Be able to reprocess historical periods without drift.
  5. Never depend on real-time system clock.
  6. Never compute commissions without effective-dated config resolution.

​
VI. READINESS GOVERNANCE

Readiness exists in exactly two domains:
  1. Batch readiness (wizard gate)
  2. Tenant readiness (admin informational)
These domains must never be conflated. Violation of this principle is classified as:
  • Severity: High UX + Medium data risk

​
VII. CONFIGURATION GOVERNANCE

Configuration must:
  • Be effective-dated where appropriate.
  • Never require reconfiguration for future periods unless intentionally superseded.
  • Be traceable with created_by + timestamp.

​
VIII. IDENTITY & SCOPE ENFORCEMENT

All queries must include:
  • tenant_id filter
  • org_id filter when ORG_SCOPED
No silent broadening to SHARED unless explicitly documented.

​
IX. INCIDENT RESPONSE PROTOCOL

If production issue occurs involving:
  • Scope mismatch
  • Materialization failure
  • Duplicate stage rows
  • Commission miscalculation
Procedure:
  1. Freeze deployment.
  2. Capture failing batch_id + period_label.
  3. Capture readiness debug output.
  4. Capture stage snapshot row count.
  5. Replay locally.
  6. Diff outputs.
  7. Create ADR before fix.

​
X. QUALITY BAR FOR “DONE”

A feature is not “done” unless:
  • Deterministic behavior is proven.
  • Tests exist covering the invariant.
  • Deploy surface is identified.
  • Network + DB + UI proof captured.
  • Documentation updated if semantics changed.

​
XI. ROADMAP CONTROL (LOCKED ORDER)

Phase 8 — Wizard Completion Phase 9 — Deterministic Math Engine Phase 10 — Audit Outputs Phase 11 — Performance Hardening + Multi-Vertical Scaling No skipping ahead without closing invariants.

​
XII. CURSOR EXECUTION PROTOCOL

Before any change, Cursor must:
  1. Identify deploy surface (Frontend / Backend / Data / Pipeline).
  2. Identify risk level (0–4).
  3. Identify invariants affected.
  4. Perform red-flag scan with file references.
  5. Implement minimum diff.
  6. Add tests.
  7. Provide deploy + validation proof checklist.
If any of these are skipped, work stops.

​
INVARIANT ENFORCEMENT LAYER

​
What we’re adding

​
A) Static Invariant Checks (fast, deterministic)

Runs on every PR/push:
  1. No float math in backend money paths (Python)
  2. Every BigQuery query must filter by tenant_id (Python SQL strings)
  3. Readiness endpoints must require period_label (FastAPI route signatures)
  4. Wizard UI must not call tenant-wide readiness (Frontend network call guard)
  5. No dev-only panels in prod build (frontend build guard)

​
B) Deterministic Replay Tests (critical, slower)

Runs nightly or on-demand (or blocking once stable):
  • Re-run /process twice for the same (tenant, org, period, batch) → identical stage outputs
  • Snapshot diffs / row-count invariants / checksum invariants

​
Repo Additions (Concrete Files)

​
1) Create tools/invariants/ folder

Add:

​
tools/invariants/run_all.py

Orchestrator that runs all checks and exits non-zero if any blocking invariant fails.

​
tools/invariants/python_no_float.py

AST-based scan (real enforcement, not grep):
  • Blocks float(...), math.* operations in money modules, /process, stage writers, commission calculators
  • Blocks literal floats like 1.23 in selected directories
Target paths (tighten over time):
  • api/**
  • especially api/math/**, api/commissions/**, api/bigquery/**, api/routes/** (or wherever money logic lives)

​
tools/invariants/bq_tenant_filter.py

Heuristic enforcement:
  • Find BigQuery query strings (f-strings / triple-quoted strings passed to client.query)
  • Require presence of either:
    • tenant_id = @tenant_id
    • OR WHERE tenant_id + parameter usage
  • Allow explicit exceptions via inline comment tag:
    • # INVARIANT_ALLOW: tenant_filter (must include justification text)
This prevents the #1 multi-tenant foot-gun.

​
tools/invariants/readiness_requires_period.py

AST scan for FastAPI routes:
  • Ensure readiness endpoints require period_label in signature
  • Specifically:
    • /api/v1/intake/ingestion-wizard/readiness
    • /api/v1/admin/onboarding/businesses/readiness (if exists)
  • Reject defaults like period_label: str = Query("...") (must be required)

​
tools/invariants/frontend_no_tenant_readiness_in_wizard.py

Scan for forbidden endpoint usage in wizard pages/components:
  • Forbid strings:
    • /api/v1/admin/onboarding/businesses/readiness
    • admin/onboarding/businesses/readiness inside:
    • dashboard/src/pages/ingestion/**
    • dashboard/src/components/intake/**
Allow exceptions only outside wizard (admin onboarding pages).

​
tools/invariants/frontend_no_dev_panels_prod.py

Simple check ensuring debug panels are guarded:
  • Search for DebugPanel components and assert they’re wrapped in:
    • process.env.NODE_ENV !== "production"
  • Or enforce a single DEV_ONLY helper component

​
CI Wiring (GitHub Actions)

​
2) Add workflow: .github/workflows/invariants.yml

Runs:
  • python invariant suite (fast)
  • frontend invariant suite (fast)
  • optionally: backend unit tests subset
Trigger:
  • pull_request
  • push to main
Blocking:
  • Invariants are blocking (fail workflow)
Reporting-only (optional):
  • Deterministic replay can be non-blocking initially.

​
Deterministic Replay (Stage Output Proof)

​
3) Add a replay test harness

Goal: if we run /process twice, outputs are identical.

​
Option A (best): pure BigQuery replay test

Create api/tests/test_deterministic_replay_stage1.py that:
  1. Seeds a small synthetic batch input (or uses a fixture table)
  2. Calls the internal stage writer function twice
  3. Compares:
    • row count
    • stable checksum of ordered rows (e.g., hash of concatenated stable columns)
This avoids needing HTTP and auth.

​
Option B: API integration replay test (nightly)

api/tests/integration/test_process_replay_deterministic.py:
  • Requires env vars & credentials → run nightly or manual
Validation criteria:
  • stage1_bridge_rows for batch_id unchanged after second run
  • ingestion_batches.status stable
  • Any derived snapshots match checksum

​
What “Enterprise Grade” Means Here (Enforcement Levels)

We implement in phases so you don’t stall delivery:

​
Phase 1 (now) — Block merges on catastrophic invariants

Blocking:
  • tenant filter presence
  • period_label required in readiness routes
  • forbidden tenant-wide readiness call in wizard
Warn only:
  • no float scan (until tuned)
  • dev panel guard (until tuned)

​
Phase 2 — Block merges on money integrity

Blocking:
  • no float usage in money paths
  • explicit rounding policy tests

​
Phase 3 — Block merges on deterministic replay

Blocking:
  • deterministic replay test for stage outputs

​
Cursor Implementation Order

​
Objective

Add an automated invariant gate so we stop relying on human memory. This must be deterministic and auditable.

​
Step 0 — Red-flag scan (required)

Confirm:
  • where money math lives (api/** paths)
  • where BigQuery queries are built
  • exact readiness route files
  • wizard file locations (already known from PR2)
Paste back the paths you’ll target before implementing.

​
Step 1 — Add tools

Create folder tools/invariants/ with:
  • run_all.py
  • python_no_float.py (AST)
  • bq_tenant_filter.py
  • readiness_requires_period.py
  • frontend_no_tenant_readiness_in_wizard.py
  • frontend_no_dev_panels_prod.py
Each script must:
  • print violations with file:line
  • exit 1 if violations found (for blocking checks)
  • support --paths and --allow-tag (optional but preferred)

​
Step 2 — Add GitHub Actions workflow

Add .github/workflows/invariants.yml that runs:
  • python tools/invariants/run_all.py
  • optionally npm run test:smoke for dashboard (already exists)
Make invariants blocking.

​
Step 3 — Add initial deterministic replay test (Stage1)

Add one unit-level deterministic replay test for Stage1 writer logic (no HTTP). Pass criteria:
  • running the same computation twice yields identical row checksums.

​
Step 4 — Docs

Add docs/INVARIANT_ENFORCEMENT.md describing:
  • invariants
  • allow-tags + justification policy
  • how to run locally
  • escalation process

​
Step 5 — Ship

Merge to main and confirm the invariants workflow runs on PR. STOP if:
  • the invariant checks produce too many false positives; in that case, narrow scopes and mark non-blocking until tuned.