Skip to content
Vouch Docs

Restore Postgres from a backup

Postgres backups are continuous: the CloudNativePG cluster (vouch-db) ships every WAL segment plus a daily 03:00 UTC base backup to the backups bucket owned by the persistent stack (infra/persistent), through the node instance role. Retention is 30 days. The bucket survives a teardown, so a fresh deploy can be seeded from the last state of the previous one.

Object layout: s3://vouch-backups-<account>/vouch-db/<serverName>/{base,wals}/. serverName carries the EIP of the deploy that wrote it (vouch-db-54-74-43-141), because barman refuses to archive into a folder that already holds another server’s WAL — every deploy after a teardown is a new cluster on a new EIP.

1. Find the backup to restore

Section titled “1. Find the backup to restore”
Terminal window
export AWS_PROFILE=<sso-profile>
BUCKET=$(tofu -chdir=infra output -raw backups_bucket)
aws s3 ls "s3://$BUCKET/vouch-db/"                      # one prefix per serverName
aws s3 ls "s3://$BUCKET/vouch-db/vouch-db-54-74-43-141/base/"   # base backups

If the old cluster still exists you can also kubectl -n vouch get backups — the Backup objects are owned by the ScheduledBackup and list their serverName and end time.

2. Decide the recovery target

Section titled “2. Decide the recovery target”
  • Latest — replay all WAL up to the end of the archive (default).
  • Point in time — recoveryTarget.targetTime: "2026-09-13 10:00:00.000+00" to stop before a bad migration or a mass delete.

3. Bootstrap the new cluster from the object store

Section titled “3. Bootstrap the new cluster from the object store”

Recovery is a bootstrap.recovery on a new Cluster: CloudNativePG restores the base backup, replays WAL from the archive, then promotes. The chart’s postgres-cluster.yaml renders bootstrap.initdb; for a restore, render it once with helm template, replace the bootstrap block, and apply the result by hand (the next helmfile apply keeps the running cluster: CNPG ignores bootstrap after the cluster exists).

apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: vouch-db
  namespace: vouch
spec:
  instances: 1
  imageName: ghcr.io/cloudnative-pg/postgresql:18
  storage:
    size: 5Gi
    storageClass: ebs-gp3
  affinity:
    nodeSelector:
      vouch.dev/role: apps
  resources:
    requests: { cpu: 100m, memory: 256Mi }
    limits: { memory: 512Mi }


  bootstrap:
    recovery:
      source: previous
      # optional point-in-time target; omit for "latest"
      # recoveryTarget:
      #   targetTime: "2026-09-13 10:00:00.000+00"


  # Where to read from: the previous deploy's server name.
  externalClusters:
    - name: previous
      barmanObjectStore:
        destinationPath: "s3://vouch-backups-<account>/vouch-db"
        serverName: vouch-db-54-74-43-141
        s3Credentials:
          inheritFromIAMRole: true
        wal:
          maxParallel: 4


  # Where the restored cluster writes ITS backups: a fresh server name
  # (never reuse the source's folder — barman would mix the two WAL streams).
  backup:
    retentionPolicy: "30d"
    barmanObjectStore:
      destinationPath: "s3://vouch-backups-<account>/vouch-db"
      serverName: vouch-db-<new-eip-dashes>
      s3Credentials:
        inheritFromIAMRole: true
      wal: { compression: gzip }
      data: { compression: gzip }

Steps:

Terminal window
export KUBECONFIG=$PWD/kubeconfig      # SSM tunnel open, see get_kubeconfig.sh
# 1. platform only, so the CNPG operator + ESO + EBS CSI are up
cd deploy && helmfile -l phase=infra apply && cd ..
kubectl create namespace vouch --dry-run=client -o yaml | kubectl apply -f -
# 2. the restored cluster (file from the YAML above)
kubectl apply -f restore-cluster.yaml
kubectl -n vouch get cluster vouch-db -w      # wait for "Cluster in healthy state"
# 3. the rest of the stack; helmfile sees the Cluster already exists
scripts/deploy_cluster.sh

deploy_cluster.sh runs the migration hook Job against the restored database, so a restore from an older schema is migrated forward automatically.

4. Verify

Section titled “4. Verify”
Terminal window
kubectl -n vouch exec -it vouch-db-1 -- psql -U postgres vouch -c 'select count(*) from users;'
kubectl -n vouch get scheduledbackup vouch-db-daily
kubectl -n vouch get backups          # a fresh base backup appears shortly (immediate: true)

Confirm select pg_is_in_recovery() is f (the cluster was promoted) and that the app secret in Secrets Manager (vouch/app) is the same one the backup was taken with: session JWTs and, above all, the phone pepper are keyed on it. The persistent stack never rotates it, but a restore into a different AWS account needs the same values or every user re-verifies.

Field reference

Section titled “Field reference”
Field Meaning
bootstrap.recovery.source name in externalClusters to restore from
bootstrap.recovery.backup.name alternative: restore from a Backup object that still exists in the cluster
bootstrap.recovery.recoveryTarget.targetTime PITR timestamp (YYYY-MM-DD HH:MM:SS.ffffff+TZ)
externalClusters[].barmanObjectStore.serverName folder under destinationPath written by the source cluster
backup.barmanObjectStore.serverName folder the restored cluster writes to — must differ from the source

CloudNativePG reference: Backup on object stores and Recovery in the 1.24 documentation.