Skip to main content

Dashboard Data Contract Map - Code Validation

Summary (3–6 bullets)

  • Validates dashboard contract assumptions against actual query functions and SQL view definitions.
  • Extracts concrete FROM objects for endpoint-related backend query paths.
  • Resolves naming and contract contradictions (table names, ranking behavior, source objects).
  • Produces a dependency matrix from API endpoints to base BigQuery objects.

When to use this (3–6 bullets)

  • When reconciling docs with backend/query reality.
  • Before refactoring dashboard query functions or analytics views.
  • During incident response for contract drift or data source confusion.
  • When auditing whether endpoint behavior still matches documented contracts.

What you’ll walk away with (2–5 bullets)

  • A code-backed contract map for dashboard endpoint dependencies.
  • Clear resolutions for common source-of-truth ambiguities.
  • A validation checklist you can reuse after query/view changes.
Date: 2025-01-XX
Purpose: Validate dashboard contract map against actual code and SQL

1. Query Functions - FROM Clauses

get_ceo_snapshot_from_view

Location: api/bigquery/queries.py:556
FROM Object:
FROM `payroll-bi-gauntlet.payroll_analytics.ceo_snapshot`

get_ceo_metrics_range

Location: api/bigquery/queries.py:8179
FROM Objects (3 queries):
  1. Financial metrics:
    FROM `payroll-bi-gauntlet.payroll_analytics.ceo_snapshot`
    
  2. Distinct counts:
    FROM `payroll-bi-gauntlet.payroll_raw.stage1_snapshots`
    
  3. Active snapshot:
    FROM `payroll-bi-gauntlet.payroll_raw.stage1_snapshots`
    
  4. MoM fields (for single period):
    FROM `payroll-bi-gauntlet.payroll_analytics.ceo_snapshot`
    

get_agent_commissions_range

Location: api/bigquery/queries.py:8850
FROM Object:
FROM `payroll-bi-gauntlet.payroll_processed.stage3_snapshots`
Filter: tpa_applied = TRUE

get_top_businesses_from_view

Location: api/bigquery/queries.py:5233
FROM Object:
FROM `payroll-bi-gauntlet.payroll_analytics.top_businesses`

get_top_businesses_ytd

Location: api/bigquery/queries.py:9223
FROM Object:
FROM `payroll-bi-gauntlet.payroll_raw.stage1_snapshots`

get_top_businesses_qtd

Location: api/bigquery/queries.py:9295
FROM Object:
FROM `payroll-bi-gauntlet.payroll_raw.stage1_snapshots`

get_top_agents_from_view

Location: api/bigquery/queries.py:5300
FROM Object:
FROM `payroll-bi-gauntlet.payroll_analytics.top_agents`

get_top_agents_ytd

Location: api/bigquery/queries.py:9367
FROM Object:
FROM `payroll-bi-gauntlet.payroll_processed.stage3_snapshots`
Filter: tpa_applied = TRUE

get_top_agents_qtd

Location: api/bigquery/queries.py:9438
FROM Object:
FROM `payroll-bi-gauntlet.payroll_processed.stage3_snapshots`
Filter: tpa_applied = TRUE

get_growth_loss_summary

Location: api/bigquery/queries.py:677
Delegates to:
  • get_growth_loss_summary_mom() → calls get_growth_loss_details_from_view() → queries stage1_snapshots (FULL OUTER JOIN)
  • get_growth_loss_summary_snapshot() → queries stage1_snapshots directly
FROM Objects:
FROM `payroll-bi-gauntlet.payroll_raw.stage1_snapshots`  -- Via get_growth_loss_details_from_view

get_growth_loss_details

Location: api/bigquery/queries.py:5755
FROM Object:
FROM `payroll-bi-gauntlet.payroll_analytics.business_growth_loss`

get_new_lost_businesses

Location: api/bigquery/queries.py:5511
FROM Objects (FULL OUTER JOIN):
-- Current period CTE:
FROM `payroll-bi-gauntlet.payroll_raw.stage1_snapshots`

-- Previous period CTE:
FROM `payroll-bi-gauntlet.payroll_raw.stage1_snapshots`

get_business_health_distribution

Location: api/bigquery/queries.py:5837
FROM Object:
FROM `payroll-bi-gauntlet.payroll_analytics.business_growth_loss`

2. View SQL - SELECT Clauses

analytics_ceo_snapshot.sql

Location: integration/bigquery/sql/views/analytics_ceo_snapshot.sql
Base Table: payroll-bi-gauntlet.payroll_raw.stage1_snapshots
SELECT Columns:
SELECT
  s.tenant_id,
  s.period_label,
  s.total_businesses,
  s.total_employees,
  s.gross_payout,
  s.chargebacks,
  s.net_payout,
  -- MoM changes
  s.total_businesses - COALESCE(p.prev_total_businesses, 0) as business_change_mom,
  s.total_employees - COALESCE(p.prev_total_employees, 0) as employee_change_mom,
  s.gross_payout - COALESCE(p.prev_gross_payout, 0) as gross_payout_change_mom,
  s.net_payout - COALESCE(p.prev_net_payout, 0) as net_payout_change_mom,
  -- MoM percentages
  CASE WHEN COALESCE(p.prev_total_businesses, 0) > 0 
    THEN ROUND((s.total_businesses - p.prev_total_businesses) / p.prev_total_businesses * 100, 2)
    ELSE 0 END as business_change_pct_mom,
  CASE WHEN COALESCE(p.prev_total_employees, 0) > 0
    THEN ROUND((s.total_employees - p.prev_total_employees) / p.prev_total_employees * 100, 2)
    ELSE 0 END as employee_change_pct_mom,
  CASE WHEN COALESCE(p.prev_gross_payout, 0) > 0
    THEN ROUND((s.gross_payout - p.prev_gross_payout) / p.prev_gross_payout * 100, 2)
    ELSE 0 END as gross_payout_change_pct_mom,
  CASE WHEN COALESCE(p.prev_net_payout, 0) > 0
    THEN ROUND((s.net_payout - p.prev_net_payout) / p.prev_net_payout * 100, 2)
    ELSE 0 END as net_payout_change_pct_mom

analytics_top_businesses.sql

Location: integration/bigquery/sql/views/analytics_top_businesses.sql
Base Table: payroll-bi-gauntlet.payroll_raw.stage1_snapshots
SELECT Columns:
SELECT
  tenant_id,
  period_label,
  business_label AS business_name,
  SUM(total) AS gross_payout,
  COUNT(DISTINCT member_id) AS employees,
  SAFE_DIVIDE(SUM(total), NULLIF(COUNT(DISTINCT member_id), 0)) AS per_employee,
  CASE
    WHEN COUNT(DISTINCT member_id) >= 250 THEN 'Large'
    WHEN COUNT(DISTINCT member_id) >= 100 THEN 'Medium'
    WHEN COUNT(DISTINCT member_id) >= 25 THEN 'Small'
    ELSE 'Micro'
  END AS status
Rank Computation: View uses QUALIFY ROW_NUMBER() OVER (PARTITION BY tenant_id, period_label ORDER BY SUM(total) DESC) <= 10 but does NOT include rank column in SELECT. Rank is computed in backend query function.

analytics_top_agents.sql

Location: integration/bigquery/sql/views/analytics_top_agents.sql
Base Tables:
  • payroll-bi-gauntlet.payroll_processed.stage3_snapshots (for commissions)
  • payroll-bi-gauntlet.payroll_raw.stage1_snapshots (for employee counts, via JOIN)
SELECT Columns:
SELECT
  c.tenant_id,
  c.period_label,
  c.agent_name,
  c.total_commission,
  c.businesses,
  COALESCE(e.employees, 0) AS employees,
  SAFE_DIVIDE(c.total_commission, NULLIF(e.employees, 0)) AS avg_pepm
Rank Computation: View uses QUALIFY ROW_NUMBER() OVER (PARTITION BY c.tenant_id, c.period_label ORDER BY c.total_commission DESC) <= 10 but does NOT include rank column in SELECT. Rank is computed in backend query function: ROW_NUMBER() OVER (ORDER BY total_commission DESC) as rank.

analytics_business_growth_loss.sql

Location: integration/bigquery/sql/views/analytics_business_growth_loss.sql
Base Table: payroll-bi-gauntlet.payroll_raw.stage1_snapshots
SELECT Columns:
SELECT
  tenant_id,
  business_label AS business_name,
  period_label,
  employees AS current_employees,
  LAG(employees) OVER (PARTITION BY tenant_id, business_label ORDER BY period_label) AS previous_employees,
  employees - LAG(employees) OVER (PARTITION BY tenant_id, business_label ORDER BY period_label) AS net_change,
  SAFE_DIVIDE(
    employees - LAG(employees) OVER (PARTITION BY tenant_id, business_label ORDER BY period_label),
    LAG(employees) OVER (PARTITION BY tenant_id, business_label ORDER BY period_label)
  ) * 100 AS pct_change

3. Object Dependency Table

EndpointQuery FunctionBase Objects (Tables/Views)Notes
/api/v1/ceo-metrics (MTD)get_ceo_snapshot_from_view()payroll_analytics.ceo_snapshot (view)View aggregates from stage1_snapshots
/api/v1/ceo-metrics (YTD/QTD)get_ceo_metrics_range()payroll_analytics.ceo_snapshot (view)
payroll_raw.stage1_snapshots (table)
Hybrid: financial from view, counts from table
/api/v1/ceo-metrics (agent commissions)get_agent_commissions_range()payroll_processed.stage3_snapshots (table)Filter: tpa_applied = TRUE
/api/v1/top-businesses (MTD)get_top_businesses_from_view()payroll_analytics.top_businesses (view)View aggregates from stage1_snapshots
/api/v1/top-businesses (YTD)get_top_businesses_ytd()payroll_raw.stage1_snapshots (table)Direct query, aggregates across date range
/api/v1/top-businesses (QTD)get_top_businesses_qtd()payroll_raw.stage1_snapshots (table)Direct query, aggregates across date range
/api/v1/top-agents (MTD)get_top_agents_from_view()payroll_analytics.top_agents (view)View joins stage3_snapshots + stage1_snapshots
/api/v1/top-agents (YTD)get_top_agents_ytd()payroll_processed.stage3_snapshots (table)Filter: tpa_applied = TRUE
/api/v1/top-agents (QTD)get_top_agents_qtd()payroll_processed.stage3_snapshots (table)Filter: tpa_applied = TRUE
/api/v1/growth-loss (MTD)get_growth_loss_summary()get_growth_loss_summary_mom()get_growth_loss_details_from_view()payroll_raw.stage1_snapshots (table)FULL OUTER JOIN between current/previous month
/api/v1/growth-loss (YTD/QTD)get_growth_loss_summary()get_growth_loss_summary_snapshot()payroll_raw.stage1_snapshots (table)Direct query with date range comparison
/api/v1/growth-loss-detailsget_growth_loss_details()payroll_analytics.business_growth_loss (view)View uses LAG() window function
/api/v1/new-lost-businessesget_new_lost_businesses()payroll_raw.stage1_snapshots (table)FULL OUTER JOIN between current/previous period
/api/v1/business-healthget_business_health_distribution()payroll_analytics.business_growth_loss (view)View uses LAG() window function

4. Contradiction Resolution

4.1 stage1_snapshot vs stage1_snapshots

RESOLUTION:Correct name is stage1_snapshots (plural) Evidence:
  • All query functions use: payroll-bi-gauntlet.payroll_raw.stage1_snapshots
  • View SQL files use: payroll-bi-gauntlet.payroll_raw.stage1_snapshots
  • No references to stage1_snapshot (singular) found in codebase
Conclusion: The table name is stage1_snapshots (plural). Any documentation referring to stage1_snapshot (singular) is incorrect.

4.2 Rank Computed in View vs Rank Added in Backend

RESOLUTION:Rank is computed in BOTH places, but backend adds it to SELECT Evidence: Top Businesses View (analytics_top_businesses.sql):
  • View uses QUALIFY ROW_NUMBER() OVER (PARTITION BY tenant_id, period_label ORDER BY SUM(total) DESC) <= 10 to LIMIT results
  • View does NOT include rank column in SELECT clause
  • Backend function (get_top_businesses_from_view) adds: ROW_NUMBER() OVER (ORDER BY gross_payout DESC) as payout_rank
  • Backend also adds: 0 as employee_change (placeholder)
Top Agents View (analytics_top_agents.sql):
  • View uses QUALIFY ROW_NUMBER() OVER (PARTITION BY c.tenant_id, c.period_label ORDER BY c.total_commission DESC) <= 10 to LIMIT results
  • View does NOT include rank column in SELECT clause
  • Backend function (get_top_agents_from_view) adds: ROW_NUMBER() OVER (ORDER BY total_commission DESC) as rank
YTD/QTD Functions:
  • get_top_businesses_ytd(): Adds ROW_NUMBER() OVER (ORDER BY total_payout DESC) AS rank in SELECT
  • get_top_businesses_qtd(): Adds ROW_NUMBER() OVER (ORDER BY total_payout DESC) AS rank in SELECT
  • get_top_agents_ytd(): Adds ROW_NUMBER() OVER (ORDER BY total_commission DESC) AS rank in SELECT
  • get_top_agents_qtd(): Adds ROW_NUMBER() OVER (ORDER BY total_commission DESC) AS rank in SELECT
Conclusion:
  • Views use QUALIFY ROW_NUMBER() ... <= 10 to LIMIT results to top 10, but do NOT expose rank column
  • Backend query functions add rank column using ROW_NUMBER() in their SELECT clauses
  • This is intentional: views filter to top 10, backend adds rank for display purposes

4.3 stage3_snapshots vs agent_report_v5

RESOLUTION:Dashboard uses stage3_snapshots, NOT agent_report_v5 Evidence:
  • All dashboard query functions query payroll_processed.stage3_snapshots
  • get_top_agents_from_view() queries payroll_analytics.top_agents view, which joins stage3_snapshots
  • get_agent_commissions_range() queries payroll_processed.stage3_snapshots directly
  • No references to agent_report_v5 found in dashboard-related query functions
Note: agent_report_v5 may exist as a separate view for other purposes (agent dashboard, reports), but it is NOT used by the CEO dashboard endpoints. Conclusion: Dashboard endpoints use stage3_snapshots table and top_agents view. agent_report_v5 is not part of the CEO dashboard data contract.

5. Summary of Base Objects

Tables (Source of Truth)

  1. payroll_raw.stage1_snapshots
    • Used by: CEO metrics (distinct counts), Top Businesses (YTD/QTD), Growth/Loss, New/Lost Businesses
    • Key fields: tenant_id, period_label, business_label, member_id, credit, debit, total, payee_agent_id
  2. payroll_processed.stage3_snapshots
    • Used by: Agent Commissions, Top Agents (YTD/QTD), Top Agents view (via JOIN)
    • Key fields: tenant_id, period_label, business_name, downline_agent_name, agent_total, tpa_applied

Views (Pre-aggregated)

  1. payroll_analytics.ceo_snapshot
    • Base: stage1_snapshots
    • Used by: CEO Metrics (MTD), CEO Metrics Range (financial metrics)
  2. payroll_analytics.top_businesses
    • Base: stage1_snapshots
    • Used by: Top Businesses (MTD)
  3. payroll_analytics.top_agents
    • Base: stage3_snapshots (commissions) + stage1_snapshots (employee counts, via JOIN)
    • Used by: Top Agents (MTD)
  4. payroll_analytics.business_growth_loss
    • Base: stage1_snapshots
    • Used by: Growth/Loss Details, Business Health Distribution

6. Validation Checklist

  • All query functions mapped to FROM objects
  • All view SQL SELECT clauses extracted
  • Object dependency table created
  • stage1_snapshots vs stage1_snapshot resolved (plural is correct)
  • Rank computation resolved (views filter, backend adds rank column)
  • stage3_snapshots vs agent_report_v5 resolved (dashboard uses stage3_snapshots)

Status: ✅ Validation Complete
Next Step: Update main dashboard contract map document with validated information