Out-of-Band Resources

Out-of-Band Resources

A living inventory of cloud resources that were provisioned manually (outside FSD / Terraform) rather than declared in a service or infra YAML. The goal is to keep these from becoming untracked shadow infra: every entry must have an owner and a reconcile plan to bring it into IaC.

Why an out-of-band resource ever exists

A managed resource declared in a service YAML (e.g. an aws_s3_bucket dep in consumer-graph-worker.yml) gets created in the target environment’s Terraform state the moment any branch deploys there. In a shared environment (stage), that blocks every other branch’s deploy: their YAML doesn’t declare the resource, so the FSD diff wants to destroy it → conflict. (This happened on 2026-06-23: PR #263’s managed coach-model-registry bucket, deployed from a feature branch, blocked all CGW stage deploys for the team; the bucket was deleted to unblock.)

The fix is to isolate infra: provision shared-env infra via an infra change merged to main first (so all branches inherit it), and have services only reference it (unmanaged, read-only). When a resource is needed before that infra PR lands, create it manually as a tracked stopgap — and record it here.

Rules for an out-of-band resource

1. Tag it at creation so it’s discoverable: managed_by=manual-out-of-band, owner=<you>, ticket=<JIRA>, purpose=<what>, reconcile=<the adopt-into-IaC plan>. (Query later via Resource Groups / Tag Editor on managed_by=manual-out-of-band.)
2. Log it in the table below — what / env / why / owner / reconcile plan / status.
3. Reconcile into IaC via an isolated infra PR merged to main. Use terraform import to adopt the existing resource — do not redeclare it as new, or the apply tries to recreate it (e.g. S3 BucketAlreadyOwnedByYou) → a second mini-incident. After import + merge, mark the entry Reconciled (or remove it).

Inventory

Resource	Env	Type	How created	Owner	Ticket	Reason	Reconcile plan	Status
stage-coach-model-registry	stage	S3 bucket	aws s3 mb + tags (manual — see Provisioning commands)	f.luo	PLT-854	Coach new-rules model-artifact registry. consumer-graph-worker.yml references it unmanaged (read-only, key prefix coach/new-rules/*) per #263; created standalone to avoid the managed-bucket-in-service-yaml shared-stage block (2026-06-23 incident).	Isolated FSD/Terraform infra change merged to main; terraform import the existing bucket; then this entry → Reconciled. Prod registry bucket TBD via the same path.	Active — recreated + tagged 2026-06-23, artifact re-uploaded. Unmanaged-ref ships in #263 (unmerged).

Provisioning commands

The exact commands used to create each out-of-band resource, so it’s reproducible and auditable (do not lose this — it’s the only record outside CloudTrail).

stage-coach-model-registry — stage-services (643098206224), us-east-1, 2026-06-23:

aws s3 mb s3://stage-coach-model-registry --region us-east-1 --profile stage-services-admin

aws s3api put-bucket-tagging --bucket stage-coach-model-registry --profile stage-services-admin \
  --tagging 'TagSet=[{Key=managed_by,Value=manual-out-of-band},{Key=owner,Value=f.luo},{Key=ticket,Value=PLT-854},{Key=purpose,Value=coach-new-rules-model-registry},{Key=reconcile,Value=isolated-infra-PR-to-main}]'

Artifact upload (data, not infra — re-run whenever a new model_version is promoted):

aws s3 cp <artifact.json> \
  s3://stage-coach-model-registry/coach/new-rules/<model_version>/artifact.json --profile stage-services-admin