Skip to main content

Operations

Create a Cluster with pgBackRest​

The examples directory contains several pre-configured manifests designed to work with kind (Eg: the pull policy is set to Never). These files may require modifications to run on other types of Kubernetes clusters.

To use this plugin with a Cluster, CloudNativePG users must :

  1. Create a Secret named pgbackrest-s3-secret in the namespace of the PostgreSQL Cluster. This secret must contain the key and secret-key of the s3 bucket.

    Example:

    ---
    apiVersion: v1
    kind: Secret
    metadata:
      name: minio
    type: Opaque
    stringData:
      ACCESS_KEY_ID: minioadmin
      ACCESS_SECRET_KEY: minioadmin
      ENCRYPTION_PASS: 3nCryptI0nk3y

  2. Create a pgBackRest stanza

    ---
    apiVersion: pgbackrest.dalibo.com/v1
    kind: Stanza
    metadata:
      name: stanza-sample
    spec:
      stanzaConfiguration:
        name: main
        s3Repositories:
          - bucket: bucket-01
            endpoint: https://172.18.0.2:9000
            region: fr-par
            repoPath: /repo-01
            uriStyle: path
            verifyTLS: false
            retentionPolicy:
              full: 7
              fullType: count
              diff: 14
              archive: 2
              archiveType: full
              history: 30
            secretRef:
              accessKeyId:
                name: minio
                key: ACCESS_KEY_ID
              secretAccessKey:
                name: minio
                key: ACCESS_SECRET_KEY
    note

    The s3Repositories variable is a list. You can configure multiple repositories. You can then select the repository to which your backup will be performed. By default :

    • the first repository is selected for backup ;
    • WAL archiving always occurs on all repositories.

  3. Create the PostgreSQL Cluster and adapt the manifest by :

    • adding the plugin definition pgbackrest.dalibo.com under the plugins entry;
    • referencing the pgBackRest stanza resource with stanzaRef.

    Example:

    ---
    apiVersion: postgresql.cnpg.io/v1
    kind: Cluster
    metadata:
      name: cluster-sample
    spec:
      # set to Never, because we use kind and image should be loaded on
      # the Kubernetes cluster and not pulled from an external registry.
      imagePullPolicy: IfNotPresent
      instances: 1
      plugins:
        - name: pgbackrest.dalibo.com
          enabled: true
          isWALArchiver: true
          parameters:
            stanzaRef: stanza-sample
            pluginConfigRef: sample-pluginconfig
      storage:
        size: 1Gi

If it runs without errors, the Pod dedicated to the PostgreSQL Cluster should have now two containers. One for the postgres service (which is the default setup), an other one for the pgbackrest plugin, named pgbackrest-plugin. The injected container now holds the responsibility for archiving the WALs and triggering backups when a backup request is made.

Backup a Cluster​

There are two ways to backup a PostgreSQL Cluster with this plugin through the CloudNativePG operator :

  • One shot backup, equivalent to running it by hand but through a Backup object definition ;
  • With Scheduled Backup object, equivalent to defining a crontab entry to run a backup periodically.

Whatever the kind of backup, users can list and see them with the appropriate kubectl command :

kubectl get backups.postgresql.cnpg.io

On-demand backups​

Backup can be requested through a Backup object, using the default CloudNativePG CRD Backup definition. The pgbackrest plugin can be specified when declaring the Backup object. The method should be set to plugin and the pluginConfiguration.name field to pgbackrest.dalibo.com.

Here is a full example of a backup definition using the pgBackRest plugin:

---
apiVersion: postgresql.cnpg.io/v1
kind: Backup
metadata:
  name: backup-sample
spec:
  method: plugin
  pluginConfiguration:
    name: pgbackrest.dalibo.com
    parameters:
      backupType: full  # full, diff or incr (default: incr)
  cluster:
    name: cluster-sample

It's also possible to use the cnpg plugin for kubectl to perform your backup :

kubectl cnpg backup cluster-sample -m plugin --plugin-name pgbackrest.dalibo.com

When performing a backup, you can choose the repository to which to push it. To do this, you need to define the selectedRepository key using the number of the repository, according to its position in the list of configured repositories. For example, to use the first repository:

[...]
  pluginConfiguration:
    name: pgbackrest.dalibo.com
    parameters:
      selectedRepository: "1"

Or with the cnpg plugin:

kubectl cnpg backup cluster-sample -m plugin --plugin-name pgbackrest.dalibo.com \
  --plugin-parameters selectedRepository=1

Scheduled backup​

A scheduled backup uses almost the same definition as a one-shot backup. Only the kind should be changed to ScheduledBackup. When using this object, the schedule field (with a crontab-like annotation) should also be defined under the specification (spec).

Here is a full example of a scheduled backup definition using the pgBackRest plugin:

---
apiVersion: postgresql.cnpg.io/v1
kind: ScheduledBackup
metadata:
  name: scheduled-backup-example
spec:
  method: plugin
  pluginConfiguration:
    name: pgbackrest.dalibo.com
  cluster:
    name: cluster-sample
  schedule: "0 15 * * * *"

Restoring a Cluster​

To restore a Cluster from a backup, create a new Cluster that references the Stanza containing the backup. Below is an example:

---
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: cluster-restored
spec:
  # set to Never, because we use kind and image should be loaded on
  # the Kubernetes cluster and not pulled from an external registry.
  imagePullPolicy: IfNotPresent
  instances: 1
  plugins:
    - name: pgbackrest.dalibo.com
      parameters:
        stanzaRef: stanza-sample
  storage:
    size: 1Gi
  bootstrap:
    recovery:
      source: origin
  externalClusters:
    - name: origin
      plugin:
        name: pgbackrest.dalibo.com
        parameters:
          stanzaRef: stanza-sample

When using the recovery options, the recoveryTarget can be specified to perform point-in-time recovery using a specific strategy (based on time, LSN, etc.). If it is not specified, the recovery will continue up to the latest available WAL.

---
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: cluster-restored
spec:
  # set to Never, because we use kind and image should be loaded on
  # the Kubernetes cluster and not pulled from an external registry.
  imagePullPolicy: IfNotPresent
  instances: 1
  plugins:
    - name: pgbackrest.dalibo.com
      parameters:
        stanzaRef: stanza-sample
  storage:
    size: 1Gi
  bootstrap:
    recovery:
      source: origin
      recoveryTarget:
        backupID: 20260210-101333F
  externalClusters:
    - name: origin
      plugin:
        name: pgbackrest.dalibo.com
        parameters:
          stanzaRef: stanza-sample

If no specific backup (BackupID) is specified, the plugin lets pgBackRest automatically choose the optimal backup using its standard algorithm. For more details, see the pgBackRest restore documentation.

WAL Archiving​

WAL archiving can be customized through the Stanza CRD. It is possible to define the WAL archiving strategy (e.g. using the asynchronous mode) as well as configure the pgbackrest queue size.

Stanza Consideration​

We chose to adhere to the concepts of the pgBackRest project, especially regarding the scope of a Stanza object.

As stated in the documentation, a stanza is specific to a PostgreSQL Cluster.

A stanza is the configuration for a PostgreSQL database cluster that defines where it is located, how it will be backed up, archiving options, etc.

Therefore, you will need to create as many Stanza objects as you have deployed Cluster.

Stanza Initialization (or create-stanza operation)​

Stanzas are initialized when archiving the first WAL. Since the stanza initialization state is tracked internally, restarting the sidecar container will cause the pgbackrest create-stanza command to run again.

Adding a new repository to a stanza can require running the create-stanza command again. Currently, this is not done automatically. Restarting the pgbackrest-plugin container will launch the create-stanza command.