Skip to main content

P0 Discovery Deployment & Validation Report

Part 1: Deployment Status

Deployment Method

Cloud Build (via cloudbuild.yaml)

Commit Information

  • Short SHA: 65c90d3
  • Full SHA: 65c90d3351bc3981491bf8a047828300683f3da2
  • Commit Message: fix(p0): discover businesses from intake batch into onboarding registry

Deployment Command

gcloud builds submit \
  --config=cloudbuild.yaml \
  --substitutions=SHORT_SHA=65c90d3,COMMIT_SHA=65c90d3351bc3981491bf8a047828300683f3da2 \
  --project=payroll-bi-gauntlet

Build Status

Status: ⏳ IN PROGRESS (running in background) To Check Build Status:
gcloud builds list --ongoing --limit=1 --project=payroll-bi-gauntlet
To Check Latest Revision After Build Completes:
gcloud run revisions list \
  --service=payroll-pipeline-cbs-api \
  --region=us-central1 \
  --project=payroll-bi-gauntlet \
  --limit=1 \
  --format="table(name,metadata.labels.'git-sha',metadata.creationTimestamp)" \
  --sort-by=~metadata.creationTimestamp
Expected Result: Latest revision should show git-sha=65c90d3

Verification Commands

1. Verify Revision SHA:
gcloud run revisions describe {LATEST_REVISION_NAME} \
  --region=us-central1 \
  --project=payroll-bi-gauntlet \
  --format="value(metadata.labels.'git-sha')"
2. Verify Environment Variable:
gcloud run services describe payroll-pipeline-cbs-api \
  --region=us-central1 \
  --project=payroll-bi-gauntlet \
  --format="value(spec.template.spec.containers[0].env)"
Expected: GIT_COMMIT_SHA=65c90d3351bc3981491bf8a047828300683f3da2 (or GIT_COMMIT_SHA=65c90d3)

Part 2: Runtime Validation Steps

⚠️ MANUAL STEPS REQUIRED

The following steps require manual execution with real data and authentication:

Step 2.1: Upload December Payroll

  1. Navigate to intake UI: https://payroll-pipeline-cbs.vercel.app
  2. Upload December payroll file
  3. Capture batch_id from:
    • API response
    • Browser network tab
    • Backend logs
Example batch_id: UUID format (e.g., a1b2c3d4-e5f6-7890-abcd-ef1234567890)

Step 2.2: Debug Discovery Endpoint (Dry-Run)

Endpoint:
GET https://payroll-pipeline-cbs-api-evndxpcirq-uc.a.run.app/api/v1/admin/onboarding/debug/discover/{batch_id}
Headers:
Authorization: Bearer {admin_jwt_token}
cURL Example:
curl -X GET \
  "https://payroll-pipeline-cbs-api-evndxpcirq-uc.a.run.app/api/v1/admin/onboarding/debug/discover/{batch_id}" \
  -H "Authorization: Bearer {token}"
Expected Response:
{
  "success": true,
  "tenant_id": "creative_benefit_strategies",
  "batch_id": "{batch_id}",
  "discovered_count": 3,
  "dim_upserted_count": 0,
  "onboarding_upserted_count": 0,
  "invalid_period_count": 0,
  "dry_run": true,
  "businesses": [
    {
      "raw_name": "AIC",
      "normalized_name": "AIC",
      "business_id": "a1b2c3d4e5f6g7h8",
      "first_seen_date": "2026-12-01",
      "last_seen_date": "2026-12-01"
    },
    {
      "raw_name": "Motor Medics",
      "normalized_name": "MOTOR MEDICS",
      "business_id": "i9j0k1l2m3n4o5p6",
      "first_seen_date": "2026-12-01",
      "last_seen_date": "2026-12-01"
    },
    {
      "raw_name": "Auto Intensive Care of Savannah",
      "normalized_name": "AUTO INTENSIVE CARE OF SAVANNAH",
      "business_id": "q7r8s9t0u1v2w3x4",
      "first_seen_date": "2026-12-01",
      "last_seen_date": "2026-12-01"
    }
  ]
}
Validation Checklist:
  • discovered_count > 0
  • ✅ At least one of: AIC, Motor Medics, Auto Intensive Care of Savannah present
  • invalid_period_count present (should be 0 ideally)
  • dry_run: true present
  • dim_upserted_count: 0 (dry-run, no writes)
  • onboarding_upserted_count: 0 (dry-run, no writes)
Capture: JSON response snippet (redact sensitive IDs)

Step 2.3: Verify Discovery Auto-Runs on Intake

Discovery should run automatically after intake processing completes. Check Backend Logs:
gcloud logging read \
  "resource.type=cloud_run_revision AND resource.labels.service_name=payroll-pipeline-cbs-api AND textPayload=~'Discover'" \
  --limit=20 \
  --format=json \
  --project=payroll-bi-gauntlet
Expected Log Entries:
[ONBOARDING] Discovering businesses from upload: tenant=creative_benefit_strategies, batch_id={batch_id}
[ONBOARDING] Discovered 3 distinct businesses from upload
[ONBOARDING] Discovery completed: tenant=creative_benefit_strategies, discovered=3, dim_upserted=3, onboarding_upserted=3, invalid_period_count=0
If invalid_period_count > 0:
[ONBOARDING] WARNING: invalid_period_count=5 detected during discovery for batch_id={batch_id}

Step 2.4: Preflight / Readiness Check

  1. Navigate to Preflight / Readiness UI
  2. Refresh readiness check
  3. Expected: Newly discovered businesses appear under “Not ready for ingest”
Expected Behavior:
  • Businesses show as “Not ready”
  • Reasons include: missing agent assignment, missing PEPM configuration, etc.
Capture: Screenshot or log confirming at least one newly discovered business appears in “Not ready” list

Step 2.5: Idempotency Check (No Duplicates)

Option A: Re-upload Same December Payroll
  1. Upload same December payroll file again
  2. Refresh Preflight
  3. Expected: No duplicate entries in UI
Option B: Re-trigger Discovery for Same batch_id
  1. Call debug endpoint again with same batch_id
  2. Expected: Same discovered_count, no new rows in BigQuery
BigQuery Verification (Run in BigQuery Console):
-- Check for duplicates in config_business_onboarding
SELECT tenant_id, business_id, COUNT(*) as count
FROM `payroll-bi-gauntlet.payroll_analytics.config_business_onboarding`
WHERE tenant_id = 'creative_benefit_strategies'
GROUP BY tenant_id, business_id
HAVING COUNT(*) > 1;

-- Check for duplicates in dim_business_mapping
SELECT tenant_id, normalized_name, COUNT(*) as count
FROM `payroll-bi-gauntlet.payroll_analytics.dim_business_mapping`
WHERE tenant_id = 'creative_benefit_strategies'
GROUP BY tenant_id, normalized_name
HAVING COUNT(*) > 1;
Expected Result: 0 rows returned (no duplicates)

Part 3: Success Criteria

P0 Fully Closed if ALL are true:
  1. Deployment: Latest revision shows git-sha=65c90d3
  2. Debug Endpoint: Returns discovered businesses (dry-run verified, no writes)
  3. Auto-Discovery: Runs automatically on intake completion
  4. Preflight: Shows missing businesses as “Not ready for ingest”
  5. Idempotency: No duplicates on re-run (verified via BigQuery)
  6. Tenant Isolation: No cross-tenant data leakage

Part 4: Rollback (If Needed)

If validation fails or issues are discovered:
# Get previous revision
PREV_REV=$(gcloud run revisions list \
  --service=payroll-pipeline-cbs-api \
  --region=us-central1 \
  --project=payroll-bi-gauntlet \
  --limit=2 \
  --format="value(name)" | Select-Object -Last 1)

# Rollback to previous revision
gcloud run services update-traffic payroll-pipeline-cbs-api \
  --to-revisions=${PREV_REV}=100 \
  --region=us-central1 \
  --project=payroll-bi-gauntlet

Report Template

When validation is complete, report:
  1. Cloud Run Revision ID: {revision_name} with git-sha=65c90d3
  2. Debug Endpoint JSON: {snippet showing discovered business}
  3. Preflight Screenshot/Log: {evidence of business in "Not ready"}
  4. Duplicate Check Result: 0 rows returned
If anything deviates, STOP and report findings.