Operations

This page provides a guide on how to configure and manage your PostgreSQL clusters using the pgBackRest CNPG plugin.

It covers essential administrative basic tasks, including creating a cluster with a S3 repository, performing on-demand or scheduled backups, restoring, and customizing WAL archiving for asynchronous performance.

note

To follow the instructions here, users need an up and running Kubernetes cluster, kubectl command-line, and probably a text editor to modify YAML configuration files.

To keep the documentation clear and consistent, all the examples below reference a CNPG cluster named cluster-sample.

Create a Cluster with pgBackRest S3 repository

The examples directory contains several pre-configured manifests designed to work with kind (Eg: the pull policy is set to Never). These files may require modifications to run on other types of Kubernetes clusters.

To use this plugin with a Cluster, CloudNativePG users must:

Create a Secret named pgbackrest-s3-secret in the namespace of the PostgreSQL Cluster. This secret must contain the key and secret-key of the s3 bucket.

Example:

---
apiVersion: v1
kind: Secret
metadata:
  name: minio
type: Opaque
stringData:
  ACCESS_KEY_ID: minioadmin
  ACCESS_SECRET_KEY: minioadmin
  ENCRYPTION_PASS: 3nCryptI0nk3y

Create a pgBackRest stanza

---
apiVersion: pgbackrest.dalibo.com/v1
kind: Stanza
metadata:
  name: stanza-sample
spec:
  stanzaConfiguration:
    name: main
    s3Repositories:
      - bucket: bucket-01
        endpoint: https://172.18.0.2:9000
        region: fr-par
        repoPath: /repo-01
        uriStyle: path
        verifyTLS: false
        retentionPolicy:
          full: 7
          fullType: count
          diff: 14
          archive: 2
          archiveType: full
          history: 30
        secretRef:
          accessKeyId:
            name: minio
            key: ACCESS_KEY_ID
          secretAccessKey:
            name: minio
            key: ACCESS_SECRET_KEY

note

The s3Repositories variable is a list. For Azure Blob Storage repositories, use the azureRepositories key instead. more information about the Azure Blob Storage can be found in the configuration references. Both storage types can be configured at the same time if needed. You can then select the repository to which your backup will be performed. By default:

the first repository is selected for backup;
WAL archiving always occurs on all repositories.

Create the PostgreSQL Cluster and adapt the manifest by:

adding the plugin definition pgbackrest.dalibo.com under the plugins entry;
referencing the pgBackRest stanza resource with stanzaRef.

Example:

---
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: cluster-sample
spec:
  # set to Never, because we use kind and image should be loaded on
  # the Kubernetes cluster and not pulled from an external registry.
  imagePullPolicy: IfNotPresent
  instances: 1
  plugins:
    - name: pgbackrest.dalibo.com
      enabled: true
      isWALArchiver: true
      parameters:
        stanzaRef: stanza-sample
        pluginConfigRef: sample-pluginconfig
  storage:
    size: 1Gi

If it runs without errors, the Pod dedicated to the PostgreSQL Cluster should have now two containers. One for the postgres service (which is the default setup), an other one for the pgbackrest plugin, named pgbackrest-plugin. The injected container now holds the responsibility for archiving the WALs and triggering backups when a backup request is made.

Backup a Cluster

There are two ways to backup a PostgreSQL Cluster with this plugin through the CloudNativePG operator:

One shot backup, equivalent to running it by hand but through a Backup object definition;
With Scheduled Backup object, equivalent to defining a crontab entry to run a backup periodically.

Whatever the kind of backup, users can list and see them with the appropriate kubectl command:

kubectl get backups.postgresql.cnpg.io

On-demand backup

Backup can be requested through a Backup object, using the default CloudNativePG CRD Backup definition. The pgbackrest plugin can be specified when declaring the Backup object. The method should be set to plugin and the pluginConfiguration.name field to pgbackrest.dalibo.com.

Here is a full example of a backup definition using the pgBackRest plugin:

---
apiVersion: postgresql.cnpg.io/v1
kind: Backup
metadata:
  name: backup-sample
spec:
  method: plugin
  pluginConfiguration:
    name: pgbackrest.dalibo.com
    parameters:
      backupType: full  # full, diff or incr (default: incr)
  cluster:
    name: cluster-sample

It's also possible to use the cnpg plugin for kubectl to perform your backup:

kubectl cnpg backup cluster-sample -m plugin --plugin-name pgbackrest.dalibo.com

When performing a backup, you can choose the repository to which to push it. To do this, you need to define the selectedRepository key using the number of the repository, according to its position in the list of configured repositories. For example, to use the first repository:

[...]
  pluginConfiguration:
    name: pgbackrest.dalibo.com
    parameters:
      selectedRepository: "1"

Or with the cnpg plugin:

kubectl cnpg backup cluster-sample -m plugin --plugin-name pgbackrest.dalibo.com \
  --plugin-parameters selectedRepository=1

Scheduled backup

A scheduled backup uses almost the same definition as a one-shot backup. Only the kind should be changed to ScheduledBackup. When using this object, the schedule field (with a crontab-like annotation) should also be defined under the specification (spec).

Here is a full example of a scheduled backup definition using the pgBackRest plugin:

---
apiVersion: postgresql.cnpg.io/v1
kind: ScheduledBackup
metadata:
  name: scheduled-backup-example
spec:
  method: plugin
  pluginConfiguration:
    name: pgbackrest.dalibo.com
  cluster:
    name: cluster-sample
  schedule: "0 15 * * * *"

Restore a Cluster

To restore a Cluster from a backup, create a new Cluster that references the Stanza containing the backup. Below is an example:

---
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: cluster-restored
spec:
  # set to Never, because we use kind and image should be loaded on
  # the Kubernetes cluster and not pulled from an external registry.
  imagePullPolicy: IfNotPresent
  instances: 1
  plugins:
    - name: pgbackrest.dalibo.com
      parameters:
        stanzaRef: stanza-sample
  storage:
    size: 1Gi
  bootstrap:
    recovery:
      source: origin
  externalClusters:
    - name: origin
      plugin:
        name: pgbackrest.dalibo.com
        parameters:
          stanzaRef: stanza-sample

When using the recovery options, the recoveryTarget can be specified to perform point-in-time recovery using a specific strategy (based on time, LSN, etc.). If it is not specified, the recovery will continue up to the latest available WAL.

---
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: cluster-restored
spec:
  # set to Never, because we use kind and image should be loaded on
  # the Kubernetes cluster and not pulled from an external registry.
  imagePullPolicy: IfNotPresent
  instances: 1
  plugins:
    - name: pgbackrest.dalibo.com
      parameters:
        stanzaRef: stanza-sample
  storage:
    size: 1Gi
  bootstrap:
    recovery:
      source: origin
      recoveryTarget:
        backupID: 20260210-101333F
  externalClusters:
    - name: origin
      plugin:
        name: pgbackrest.dalibo.com
        parameters:
          stanzaRef: stanza-sample

If no specific backup (BackupID) is specified, the plugin lets pgBackRest automatically choose the optimal backup using its standard algorithm. For more details, see the pgBackRest restore documentation.

WAL Archiving customization and async mode

WAL archiving can be customized through the Stanza CRD. It is possible to define the WAL archiving strategy (e.g. using the asynchronous mode) as well as configure the pgbackrest queue size.

---
apiVersion: pgbackrest.dalibo.com/v1
kind: Stanza
metadata:
  name: stanza-sample
spec:
  stanzaConfiguration:
    name: main
    archive:
      async: true
    processMax: 4
    s3Repositories:
      - bucket: bucket-01
        endpoint: https://172.18.0.2:9000
        region: fr-par
        repoPath: /repo-01
        uriStyle: path
        verifyTLS: false
        retentionPolicy:
          full: 7
          fullType: count
          diff: 14
          archive: 2
          archiveType: full
          history: 30
        secretRef:
          accessKeyId:
            name: minio
            key: ACCESS_KEY_ID
          secretAccessKey:
            name: minio
            key: ACCESS_SECRET_KEY

To configure asynchronous mode, users must explicitly enable it using the following two configuration options: archive.async: enables asynchronous WAL archiving mode (push/pull supported depending on configuration) processMax: defines the maximum number of parallel pgBackRest processes used for WAL processing

When asynchronous mode is enabled, the plugin will automatically request a PersistentVolumeClaim (PVC) to let pgBackRest store spool data. This PVC is used for transient WAL data buffering and is managed specifically for asynchronous processing workloads.

Stanza Consideration

We chose to adhere to the concepts of the pgBackRest project, especially regarding the scope of a Stanza object.

As stated in the documentation, a stanza is specific to a PostgreSQL Cluster.

A stanza is the configuration for a PostgreSQL database cluster that defines where it is located, how it will be backed up, archiving options, etc.

Therefore, you will need to create as many Stanza objects as you have deployed Cluster.

Stanza Initialization (or create-stanza operation)

Stanzas are initialized when archiving the first WAL. Since the stanza initialization state is tracked internally, restarting the sidecar container will cause the pgbackrest create-stanza command to run again.

Adding a new repository to a stanza can require running the create-stanza command again. Currently, this is not done automatically. Restarting the pgbackrest-plugin container will launch the create-stanza command.

Create a Cluster with pgBackRest S3 repository​

Backup a Cluster​

On-demand backup​

Scheduled backup​

Restore a Cluster​

WAL Archiving customization and async mode​

Stanza Consideration​

Stanza Initialization (or create-stanza operation)​