Skip to main content

Metrics API Host Configuration Fix

Date: 2026-01-18
Issue: Dashboards broken - NEXT_PUBLIC_API_URL incorrectly points to payroll-backend-prod, causing metrics endpoints to fail with OPTIONS 404.

Root Cause Analysis

Current (BROKEN) Configuration

  • NEXT_PUBLIC_API_URL = https://payroll-backend-prod-238826317621.us-central1.run.app
  • Problem: Metrics endpoints (/api/v1/ceo-metrics, /api/v1/commissions, /api/v1/top-agents, etc.) are NOT implemented on payroll-backend-prod
  • Result: OPTIONS preflight requests return 404, breaking all dashboard functionality

Historical Service Architecture

Two Production Cloud Run Services (by design):
  1. payroll-pipeline-cbs (Config Plane + Metrics)
    • Service URL: https://payroll-pipeline-cbs-238826317621.us-central1.run.app
    • Purpose: Rate Plans, admin config, AND metrics/CEO dashboards
    • Endpoints:
      • /api/v1/ceo-metrics
      • /api/v1/commissions
      • /api/v1/top-agents
      • /api/v1/top-businesses
      • /api/v1/growth-loss*
      • /api/v1/admin/rate-plans/*
      • /api/v1/admin/* (other admin endpoints)
  2. payroll-backend-prod (System of Record)
    • Service URL: https://payroll-backend-prod-238826317621.us-central1.run.app
    • Purpose: Agent identities, assignments, hierarchy, audit-grade logic
    • Endpoints:
      • /api/v1/admin/hierarchy/*
      • /api/v1/admin/agents/resolver
      • /api/v1/admin/agent-profiles/*
      • /api/v1/ai/*

Evidence from Codebase

Metrics routers are included in api/main.py:
  • Line 2240: app.include_router(ceo_metrics_router, prefix="/api/v1", tags=["CEO Metrics"])
  • Line 2276: app.include_router(dashboard_router, prefix="/api/v1", tags=["Dashboard"])
  • Line 2181: app.include_router(agents_router, prefix="/api/v1", tags=["Agents"])
Router files exist:
  • api/routes/ceo_metrics.py - /api/v1/ceo-metrics
  • api/routes/commissions.py - /api/v1/commissions
  • api/routes/dashboard.py - /api/v1/top-businesses, /api/v1/top-agents, /api/v1/growth-loss*
  • api/routes/agents.py - /api/v1/leaderboard
Historical validation confirms payroll-pipeline-cbs serves metrics:
  • BACKEND_VALIDATION_SUMMARY.md: “payroll-pipeline-cbs (Main Backend)” serves CEO metrics
  • DEPLOYMENT_GUIDE.md: Test commands use payroll-pipeline-cbs-238826317621.us-central1.run.app for CEO metrics
  • PHASE2D_CHECKPOINT.md: Service payroll-pipeline-cbs-00093-lwl serves /api/v1/commissions, /api/v1/leaderboard, /api/v1/growth-loss

Corrected Vercel PROD Configuration

Required Environment Variables

# Metrics & CEO Dashboard endpoints (primary API URL)
NEXT_PUBLIC_API_URL=https://payroll-pipeline-cbs-238826317621.us-central1.run.app

# Admin endpoints (Rate Plans, other admin config)
NEXT_PUBLIC_PIPELINE_API_URL=https://payroll-pipeline-cbs-238826317621.us-central1.run.app

# Hierarchy & Resolver endpoints (System of Record)
NEXT_PUBLIC_HIERARCHY_API_URL=https://payroll-backend-prod-238826317621.us-central1.run.app

Routing Logic (from dashboard/src/lib/apiClient.ts)

  1. Hierarchy endpoints (/api/v1/admin/hierarchy/*, /api/v1/admin/agents/resolver): → Uses NEXT_PUBLIC_HIERARCHY_API_URLpayroll-backend-prod
  2. Other admin endpoints (/api/v1/admin/*): → Uses NEXT_PUBLIC_PIPELINE_API_URLpayroll-pipeline-cbs
  3. Metrics/CEO endpoints (/api/v1/ceo-metrics, /api/v1/commissions, /api/v1/top-*, etc.): → Uses NEXT_PUBLIC_API_URLpayroll-pipeline-cbs

Smoke Test Commands

1. Metrics Endpoint (CEO Dashboard)

# Test OPTIONS preflight
curl -X OPTIONS "https://payroll-pipeline-cbs-238826317621.us-central1.run.app/api/v1/ceo-metrics?period_label=2025-12-01" \
  -H "Origin: https://payroll-pipeline-cbs.vercel.app" \
  -H "Access-Control-Request-Method: GET" \
  -H "Access-Control-Request-Headers: Authorization" \
  -v

# Expected: 200 OK with CORS headers
# Access-Control-Allow-Origin: https://payroll-pipeline-cbs.vercel.app
# Access-Control-Allow-Methods: GET, POST, OPTIONS, ...

# Test GET (requires auth token)
curl "https://payroll-pipeline-cbs-238826317621.us-central1.run.app/api/v1/ceo-metrics?period_label=2025-12-01" \
  -H "Authorization: Bearer $TOKEN" \
  -H "X-Org-Id: creative_benefit_strategies"

# Expected: 200 OK with JSON response containing CEO metrics

2. Admin Rate Plans Endpoint

# Test OPTIONS preflight
curl -X OPTIONS "https://payroll-pipeline-cbs-238826317621.us-central1.run.app/api/v1/admin/rate-plans" \
  -H "Origin: https://payroll-pipeline-cbs.vercel.app" \
  -H "Access-Control-Request-Method: GET" \
  -H "Access-Control-Request-Headers: Authorization,X-Org-Id" \
  -v

# Expected: 200 OK with CORS headers

# Test GET (requires auth token)
curl "https://payroll-pipeline-cbs-238826317621.us-central1.run.app/api/v1/admin/rate-plans" \
  -H "Authorization: Bearer $TOKEN" \
  -H "X-Org-Id: creative_benefit_strategies"

# Expected: 200 OK with rate plans list

3. Hierarchy Resolver Endpoint

# Test OPTIONS preflight
curl -X OPTIONS "https://payroll-backend-prod-238826317621.us-central1.run.app/api/v1/admin/agents/resolver?include_inactive=true" \
  -H "Origin: https://payroll-pipeline-cbs.vercel.app" \
  -H "Access-Control-Request-Method: GET" \
  -H "Access-Control-Request-Headers: Authorization,X-Org-Id" \
  -v

# Expected: 200 OK with CORS headers

# Test GET (requires auth token)
curl "https://payroll-backend-prod-238826317621.us-central1.run.app/api/v1/admin/agents/resolver?include_inactive=true" \
  -H "Authorization: Bearer $TOKEN" \
  -H "X-Org-Id: creative_benefit_strategies"

# Expected: 200 OK with agent resolver JSON array

Browser DevTools Verification

After updating Vercel env vars and redeploying:
  1. Open Dashboard: https://payroll-pipeline-cbs.vercel.app/ceo
  2. Open DevTools → Network tab
  3. Verify requests: CEO Metrics call:
    • URL: https://payroll-pipeline-cbs-238826317621.us-central1.run.app/api/v1/ceo-metrics?period_label=2025-12-01
    • Status: 200 OK
    • CORS headers present: Access-Control-Allow-Origin: https://payroll-pipeline-cbs.vercel.app
    Rate Plans call (if navigating to admin):
    • URL: https://payroll-pipeline-cbs-238826317621.us-central1.run.app/api/v1/admin/rate-plans
    • Status: 200 OK
    Hierarchy Resolver call (if navigating to hierarchy tab):
    • URL: https://payroll-backend-prod-238826317621.us-central1.run.app/api/v1/admin/agents/resolver
    • Status: 200 OK

Action Items

  1. Update Vercel Production Environment Variables:
    • Set NEXT_PUBLIC_API_URL = https://payroll-pipeline-cbs-238826317621.us-central1.run.app
    • Keep NEXT_PUBLIC_PIPELINE_API_URL = https://payroll-pipeline-cbs-238826317621.us-central1.run.app
    • Keep NEXT_PUBLIC_HIERARCHY_API_URL = https://payroll-backend-prod-238826317621.us-central1.run.app
  2. Trigger Vercel Redeploy:
    • Push a no-op commit OR manually trigger redeploy in Vercel dashboard
  3. Verify Dashboard Functionality:
    • CEO Dashboard loads metrics
    • No CORS errors in console
    • All API calls return 200 OK

Summary

Correct Metrics API Host: https://payroll-pipeline-cbs-238826317621.us-central1.run.app Why: This service implements all metrics endpoints (ceo-metrics, commissions, top-agents, top-businesses, growth-loss) as confirmed by:
  • Router inclusion in api/main.py
  • Historical deployment validation documents
  • Test commands in deployment guides
Fix: Update NEXT_PUBLIC_API_URL in Vercel to point to payroll-pipeline-cbs, not payroll-backend-prod.