Skip to main content

​
GCS Bucket Baseline

​
Purpose

Define minimum bucket, IAM, and lifecycle controls for onboarding/staging object storage.

​
When to run this

  • Environment bootstrap and handoff
  • Before onboarding/intake deploys
  • During storage or ingestion incident triage

​
Prerequisites

  • gcloud authenticated to payroll-bi-gauntlet
  • Access to Cloud Storage IAM and bucket metadata

​
Inputs

  • Project: payroll-bi-gauntlet
  • Bucket: payroll-bi-gauntlet-onboarding-storage
  • Runtime SA: sa-worker@payroll-bi-gauntlet.iam.gserviceaccount.com
  • Env var contract: ONBOARDING_STORAGE_BUCKET

​
Procedure

​
1) Verify bucket exists and region posture

gcloud storage buckets describe gs://payroll-bi-gauntlet-onboarding-storage

​
2) Verify runtime IAM access

gcloud storage buckets get-iam-policy gs://payroll-bi-gauntlet-onboarding-storage
Confirm runtime SA has least-privilege object access required for upload/read flows.

​
3) Verify lifecycle policy baseline

gcloud storage buckets describe gs://payroll-bi-gauntlet-onboarding-storage \
  --format="yaml(lifecycle)"
Expected: lifecycle policy exists for staging/object cleanup to avoid orphan growth.

​
4) Verify path conventions

Required path patterns:
  • Intake uploads: intake/{tenant_id}/{batch_id}/{filename}
  • Staging NDJSON: intake-staging/{tenant_id}/{batch_id}/...
  • Bulk import: bulk-import/{tenant_id}/{org_key}/...

​
5) Verify Cloud Run env wiring

gcloud run services describe payroll-backend-prod \
  --region=us-central1 \
  --project=payroll-bi-gauntlet \
  --format="value(spec.template.spec.containers[0].env)"

gcloud run services describe payroll-pipeline-cbs-api \
  --region=us-central1 \
  --project=payroll-bi-gauntlet \
  --format="value(spec.template.spec.containers[0].env)"
Confirm ONBOARDING_STORAGE_BUCKET=payroll-bi-gauntlet-onboarding-storage on both services.

​
Verification

  • Bucket exists and expected region is confirmed
  • Runtime SA IAM is least-privilege and functional
  • Lifecycle policy is configured
  • Path conventions are respected
  • Both services have correct ONBOARDING_STORAGE_BUCKET env var

​
Failure modes & fixes

  • 403 bucket/object access errors
    • Reconcile bucket IAM for runtime SA.
  • Staging object growth
    • Add/fix lifecycle retention rules.
  • Wrong bucket used in runtime
    • Correct ONBOARDING_STORAGE_BUCKET and redeploy.

​
Artifacts produced

  • Bucket/IAM/lifecycle verification evidence
  • Env wiring verification output for both services