Skip to main content

​
Dashboard Smoke Test Checklist

​
Purpose

Manual verification checklist for QA after any backend or dashboard deploy.

​
When to run this

  • After backend deploys
  • After dashboard/frontend deploys
  • Before release sign-off

​
Prerequisites

  • Access to Agent, Admin, and CEO/Executive dashboards
  • Valid login credentials for each role
  • Browser DevTools (Network tab, Console tab)

​
Inputs

  • Environment URL
  • Role test users (Agent/Admin/CEO)
  • Expected active period (example: 2025-11-01)
  • Release identifier (commit SHA/build id)

​
Procedure

​
1. Login flows

​
1.1 Agent Login

  • Navigate to login page
  • Enter agent credentials
  • Login succeeds without errors
  • Redirected to /agent dashboard
  • Token present in localStorage (check DevTools → Application → Local Storage)
  • API calls include Authorization: Bearer <token> header (check Network tab)

​
1.2 Admin Login

  • Navigate to login page
  • Enter admin credentials
  • Login succeeds without errors
  • Redirected to /admin dashboard
  • Token present in localStorage
  • API calls include Authorization: Bearer <token> header

​
1.3 CEO/Executive Login

  • Navigate to login page
  • Enter CEO/Executive credentials
  • Login succeeds without errors
  • Redirected to /ceo dashboard
  • Token present in localStorage
  • API calls include Authorization: Bearer <token> header

​
2. Agent dashboard (/agent)

​
2.1 Page Load

  • Page loads without console errors (check DevTools → Console)
  • No 404s in Network tab for API calls
  • Loading states display correctly
  • Error states display correctly (if any)

​
2.2 KPI Cards

  • Commission Card: Displays current month commission
  • Employees Card: Displays current month employee count
  • Businesses Card: Displays current month business count
  • Growth/Loss Cards: Display trend indicators (if applicable)
  • All cards populate with non-zero values (unless legitimately zero)
  • Tooltips work on hover (if applicable)

​
2.3 Trend Charts

  • Employee Growth/Loss Chart: Renders without errors
  • Chart tooltips work on hover
  • Chart data matches KPI card values (if applicable)
  • Chart displays reasonable values (not inflated, not zero/NaN)

​
2.4 Agent Insights Page

  • Navigate to Agent Insights page (if separate route)
  • Top 10 Growth table loads and displays data
  • Top 10 Loss table loads and displays data
  • Tables sort correctly (if sorting enabled)
  • Table tooltips/hover behavior works

​
2.5 AI Assistant

  • AI Assistant sidebar/component loads
  • Chat interface displays correctly
  • Current Status: Documented 403-for-self bug (see Known Issues below)
  • Error messages display correctly (401, 403, 404, 429)
  • Network tab shows requests to canonical endpoint: /api/v1/ai/query-public
  • Known Issue: Agent may get 403 when asking about own data (see docs/AI_CHATBOT_DEBUG_NEXT.md)

​
3. Executive/CEO dashboard (/ceo)

​
3.1 Page Load

  • Page loads without console errors
  • No 404s in Network tab for API calls
  • Loading states display correctly
  • Error states display correctly (if any)

​
3.2 Executive KPI Cards

  • Total Businesses: Displays tenant-wide business count
  • Total Employees: Displays tenant-wide employee count
  • Net Change: Displays delta vs baseline (if applicable)
  • Gained/Lost Counts: Display new/lost business counts (if applicable)
  • All cards populate with non-zero values (unless legitimately zero)
  • Tooltips work on hover (if applicable)

​
3.3 Employee Growth/Loss Charts

  • Employee Growth Chart: Renders without errors
  • Employee Loss Chart: Renders without errors (if separate)
  • Chart tooltips work on hover
  • Chart data shows reasonable values (not inflated, not zero/NaN)
  • Chart scales correctly (not showing unrealistic spikes)

​
3.4 Business Insights Page

  • Navigate to Business Insights page (if separate route)
  • Top 10 Growth Businesses table loads and displays data
  • Top 10 Loss Businesses table loads and displays data
  • Tables show correct scale (not inflated counts)
  • Tables sort correctly (if sorting enabled)
  • Table tooltips/hover behavior works

​
3.5 Tooltips and Chart Hover Behavior

  • All tooltips display correctly on hover
  • Chart hover shows data points correctly
  • Tooltip content is readable and formatted correctly

​
4. Admin dashboard (/admin)

​
4.1 Page Load

  • Page loads without console errors
  • No 404s in Network tab for API calls
  • Loading states display correctly

​
4.2 User Management Page

  • Navigate to /admin/users (or user management route)
  • User list loads and displays correctly
  • Create User button works (if applicable)
  • Edit User functionality works (if applicable)
  • No stale routing (all links work correctly)

​
4.3 Admin Widgets

  • All critical admin widgets load as expected
  • Widget data displays correctly
  • Widget interactions work (if applicable)

​
5. Data sanity checks

​
5.1 Active Period Display

  • Active Period displayed matches expected (e.g., 2025-11-01 in current screenshots)
  • Period selector works (if applicable)
  • Period changes update data correctly

​
5.2 Count Validation

  • Counts are not zero/NaN (unless legitimately zero)
  • Counts are not wildly inflated (e.g., showing 1M employees when should be 100)
  • Counts match across related views (e.g., KPI card matches chart)

​
5.3 Network Tab Verification

  • No 404s in Network tab for API calls
  • API responses return 200 OK (or expected status codes)
  • API response times are reasonable (< 5 seconds for most calls)
  • No CORS errors in console
  • No authentication errors (401) for authenticated users

​
5.4 AI Chatbot Endpoint Verification

  • AI Assistant requests hit canonical endpoint: /api/v1/ai/query-public
  • Requests include Authorization: Bearer <token> header
  • Legacy /ai/query-public path not used (if used, frontend bundle is stale)

​
6. Cross-role verification

​
6.1 RBAC Enforcement

  • Agent cannot access CEO dashboard (redirects or shows error)
  • Agent cannot access Admin dashboard (redirects or shows error)
  • CEO cannot access Admin dashboard (redirects or shows error)
  • Admin can access all dashboards (if intended)

​
6.2 Data Scoping

  • Agent dashboard shows only agent’s own data (not other agents)
  • CEO dashboard shows tenant-wide data (all agents)
  • Admin dashboard shows appropriate admin data

​
Verification

  • All required login flows succeed
  • Core dashboard pages render without critical console/network failures
  • KPI cards/charts/tables show sane values
  • RBAC and data-scoping behavior is correct
  • AI endpoint path is canonical (/api/v1/ai/query-public)

​
Failure modes & fixes

​
AI assistant 403-for-self bug

  • Symptom: Agent gets 403 Forbidden when asking about own data
  • Example: “how much did kenny young make this month” returns 403
  • Status: Documented for future debugging (see docs/AI_CHATBOT_DEBUG_NEXT.md)
  • Workaround: None currently (bug being debugged)
  • Impact: Agents cannot query their own commission data via chatbot

​
Failure criteria

Stop deployment if:
  • Any login flow fails
  • Any dashboard page fails to load
  • KPI cards show zero/NaN when data should exist
  • Charts fail to render
  • Network tab shows 404s for critical API calls
  • RBAC enforcement fails (agents can access other roles’ dashboards)
Non-blocking issues:
  • AI Assistant 403-for-self bug (documented, being debugged)
  • Minor tooltip formatting issues
  • Non-critical widget loading delays

​
Artifacts produced

  • Smoke test pass/fail checklist
  • Screenshots and log evidence for failures
  • Incident notes with blocking/non-blocking classification
  • docs/AI_CHATBOT_INCIDENT_2025_12_24.md - Production incident summary
  • docs/AI_CHATBOT_DEBUG_NEXT.md - AI Assistant 403-for-self bug debug guide
  • docs/DEPLOYMENT_RUNBOOK.md - Deployment procedures
  • docs/AI_CHATBOT_SECURITY.md - Security features and API contract

​
Supporting reference

​
Reporting

After completing smoke tests:
  1. Document Results:
    • List all passed checks
    • List all failed checks
    • Note any non-blocking issues
  2. Screenshot Evidence (if failures):
    • Console errors
    • Network tab 404s
    • UI rendering issues
  3. Log Collection (if failures):
    • Cloud Run logs for backend errors
    • Vercel build logs for frontend errors
    • Browser console logs