-
Notifications
You must be signed in to change notification settings - Fork 80
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* release-operator-and-add-br * Update 1.introduction-to-nebula-operator.md
- Loading branch information
1 parent
e8b379f
commit 4f2ee89
Showing
3 changed files
with
261 additions
and
28 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
230 changes: 230 additions & 0 deletions
230
docs-2.0/nebula-operator/10.backup-restore-using-operator.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,230 @@ | ||
| # Backup and restore data using NebulaGraph Operator | ||
|
|
||
| This article introduces how to back up and restore data of the NebulaGraph cluster on Kubernetes. | ||
|
|
||
| !!! enterpriseonly | ||
|
|
||
| This feature is only for the enterprise edition NebulaGraph clusters on Kubernetes. | ||
|
|
||
| ## Overview | ||
|
|
||
| [NebulaGraph BR (Enterprise Edition)](../backup-and-restore/nebula-br-ent/1.br-ent-overview.md) is a command line tool for data backup and recovery of NebulaGraph enterprise edition. NebulaGraph Operator is based on the BR tool to achieve data backup and recovery for NebulaGraph clusters on Kubernetes. | ||
|
|
||
| When backing up data, NebulaGraph Operator creates a Job to back up the data in the NebulaGraph cluster to the specified storage service. | ||
|
|
||
| When restoring data, NebulaGraph Operator checks the specified backup NebulaGraph cluster for existence, and whether the access to remote storage is normally based on the information defined in the NebulaRestore resource object. It then creates a new cluster and restores the backup data to the new NebulaGraph cluster. For more information, see [restore flowchart](https://github.com/vesoft-inc/nebula-operator/blob/v{{operator.release}}/doc/user/br_guide.md#restore-nebulagraph-cluster). | ||
|
|
||
|
|
||
| ## Prerequisites | ||
|
|
||
| To backup and restore data using NebulaGraph Operator, the following conditions must be met: | ||
|
|
||
| - Nebula Operator version >= 1.4.0. | ||
| - The enterprise edition NebulaGraph cluster deployed on Kubernetes is running. | ||
| - In the YAML file used to create the cluster, `spec.enableBR` is set to true. | ||
|
|
||
| ``` | ||
| // Partial content of a sample cluster YAML file. | ||
| apiVersion: apps.nebula-graph.io/v1alpha1 | ||
| kind: NebulaCluster | ||
| metadata: | ||
| name: nebula | ||
| spec: | ||
| enableBR: true // Set to true to enable the backup and restore function. | ||
| ... | ||
| ``` | ||
|
|
||
| - Only storage services that use the S3 protocol (such as AWS S3, Minio, etc.) can be used to back up and restore data. | ||
| - Sufficient computing resources are available in the cluster to restore data. | ||
|
|
||
| ## Backup | ||
|
|
||
| ### Notes | ||
|
|
||
| - NebulaGraph Operator supports full and incremental backups. | ||
| - During data backup, DDL and DML statements in the specified graph space will be blocked. We recommend performing the operation during off-peak hours, such as from 2:00 am to 5:00 am. | ||
| - The cluster executing incremental backups and the cluster specified for the last backup must be the same, and the (storage bucket) path for the last backup must be the same. | ||
| - Ensure that the time between each incremental backup and the last backup is less than a `wal_ttl`. | ||
| - Specifying the backup data of a specified graph space is not supported. | ||
|
|
||
| ### Full backup | ||
|
|
||
| When backing up data to a storage service compatible with the S3 protocol, you need to create a backup Job, which will back up the full NebulaGraph data to the specified storage location. | ||
|
|
||
| Here is an example of the YAML file for a full backup Job: | ||
|
|
||
| ```yaml | ||
| apiVersion: batch/v1 | ||
| kind: Job | ||
| metadata: | ||
| name: nebula-full-backup | ||
| spec: | ||
| parallelism: 1 | ||
| ttlSecondsAfterFinished: 60 | ||
| template: | ||
| spec: | ||
| restartPolicy: OnFailure | ||
| containers: | ||
| - image: vesoft/br-ent:v{{br_ent.release}} | ||
| imagePullPolicy: Always | ||
| name: backup | ||
| command: | ||
| - /bin/sh | ||
| - -ecx | ||
| - exec /usr/local/bin/nebula-br backup full | ||
| - --meta $META_ADDRESS:9559 | ||
| - --storage s3://$BUCKET | ||
| - --s3.access_key $ACCESS_KEY | ||
| - --s3.secret_key $SECRET_KEY | ||
| - --s3.region $REGION | ||
| - --s3.endpoint https://s3.$REGION.amazonaws.com | ||
| ``` | ||
| ### Incremental backup | ||
| Except for the name of the Job and the command specified in `spec.template.spec.containers[0].command`, the YAML file for incremental backup is the same as that for a full backup. Here is an example of the YAML file for incremental backup: | ||
|
|
||
| ```yaml | ||
| apiVersion: batch/v1 | ||
| kind: Job | ||
| metadata: | ||
| name: nebula-incr-backup | ||
| spec: | ||
| parallelism: 1 | ||
| ttlSecondsAfterFinished: 60 | ||
| template: | ||
| spec: | ||
| restartPolicy: OnFailure | ||
| containers: | ||
| - image: vesoft/br-ent:v{{br_ent.release}} | ||
| imagePullPolicy: Always | ||
| name: backup | ||
| command: | ||
| - /bin/sh | ||
| - -ecx | ||
| - exec /usr/local/bin/nebula-br backup incr | ||
| - --meta $META_ADDRESS:9559 | ||
| - --base $BACKUP_NAME | ||
| - --storage s3://$BUCKET | ||
| - --s3.access_key $ACCESS_KEY | ||
| - --s3.secret_key $SECRET_KEY | ||
| - --s3.region $REGION | ||
| - --s3.endpoint https://s3.$REGION.amazonaws.com | ||
| ``` | ||
|
|
||
| ### Parameter description | ||
|
|
||
| The main parameters are described as follows: | ||
|
|
||
|
|
||
| | Parameter |Default value | Description | | ||
| | ------------- | ---- | ---- | | ||
| | `spec.parallelism` |1 |The number of tasks executed in parallel. | | ||
| | `spec.ttlSecondsAfterFinished` | 60 | The time to keep task information after the task is completed. | | ||
| | `spec.template.spec.containers[0].image` | `vesoft/br-ent:{{br_ent.release}}`|The image address of the NebulaGraph BR Enterprise Edition tool. | | ||
| | `spec.template.spec.containers[0].command`| - | The command for backing up data to the storage service compatible with the S3 protocol.<br/>For descriptions of the options in the command, see [Parametr description](../backup-and-restore/nebula-br-ent/3.backup-data.md#_12). | | ||
|
|
||
|
|
||
| For more settings of the Job, see [Kubernetes Jobs](https://kubernetes.io/docs/concepts/workloads/controllers/job/). | ||
|
|
||
| After the YAML file for the backup Job is set, run the following command to start the backup Job: | ||
|
|
||
|
|
||
|
|
||
| ```bash | ||
| kubectl apply -f <backup_file_name>.yaml | ||
| ``` | ||
|
|
||
| When the data backup succeeds, a backup file is generated in the specified storage location. For example, the backup file name is `BACKUP_2023_02_12_10_04_16`. | ||
|
|
||
|
|
||
| ## Restore | ||
|
|
||
| ### Notes | ||
|
|
||
| - After the data recovery is successful, a new cluster will be created, and the old cluster will not be deleted. Users can decide whether to delete the old cluster themselves. | ||
| - There will be a period of service unavailability during the data recovery process, so it is recommended to perform the operation during a low period of business activity. | ||
|
|
||
| <!-- ### Recovery Process | ||
|
|
||
| The recovery process is as follows: | ||
|
|
||
| 1. Verify the NebulaRestore field, check if the specified NebulaGraph cluster exists, and verify if remote S3 storage access is normal. | ||
|
|
||
| 2. Download the metadata information of the backup cluster according to the backupName, and parse it to obtain the physical topology of the backup cluster. Perform an isomorphism check with the topology of the cluster to be restored. | ||
|
|
||
| 3. Copy the Spec configuration of the backup cluster and create a new cluster for restoring data. Confirm that there are sufficient resources for scheduling before starting the cluster. | ||
|
|
||
| 4. The init container of the Metad Pod downloads the backup data. After the download is successful, start the Metad container. After all containers are running, the Operator calls the Metad interface to load the backup data. | ||
|
|
||
| 5. The init container of the Storaged Pod downloads the backup data. After the download is successful, the agent container performs data playback. After the playback is successful, the Storaged container is started. After all containers are running, the Graphd container is started. | ||
|
|
||
| 6. When the cluster is ready, the Operator calls the agent interface to clear the downloaded backup data and remove the init container configuration of the current cluster. --> | ||
|
|
||
| ### Process | ||
|
|
||
| When restoring data from a compatible S3 protocol service, you need to create a Secret to store the credentials for accessing the compatible S3 protocol service. Then create a resource object (NebulaRestore) for restoring the data, which will instruct the Operator to create a new NebulaGraph cluster based on the information defined in this resource object and restore the backup data to the newly created cluster. | ||
|
|
||
| Here is an example YAML for restoring data based on the backup file `BACKUP_2023_02_12_10_04_16`: | ||
|
|
||
| ```yaml | ||
| apiVersion: v1 | ||
| kind: Secret | ||
| metadata: | ||
| name: aws-s3-secret | ||
| type: Opaque | ||
| data: | ||
| access-key: QVNJQVE0WFlxxx | ||
| secret-key: ZFJ6OEdNcDdxenMwVGxxx | ||
| --- | ||
| apiVersion: apps.nebula-graph.io/v1alpha1 | ||
| kind: NebulaRestore | ||
| metadata: | ||
| name: restore1 | ||
| spec: | ||
| br: | ||
| clusterName: nebula | ||
| backupName: "BACKUP_2023_02_12_10_04_16" | ||
| concurrency: 5 | ||
| s3: | ||
| region: "us-west-2" | ||
| bucket: "nebula-br-test" | ||
| endpoint: "https://s3.us-west-2.amazonaws.com" | ||
| secretName: "aws-s3-secret" | ||
| ``` | ||
|
|
||
| ### Parameter Description | ||
|
|
||
| - Secret | ||
|
|
||
| |Parameter|Default Value|Description| | ||
| |:---|:---|:---| | ||
| |`metadata.name`|-|The name of the Secret.| | ||
| |`type`|`Opaque`|The type of the Secret. See [Types of Secret](https://kubernetes.io/docs/concepts/configuration/secret/#secret-types) for more information.| | ||
| |`data.access-key`|-|The AccessKey for accessing the S3 protocol-compatible storage service.| | ||
| |`data.secret-key`|-|The SecretKey for accessing the S3 protocol-compatible storage service.| | ||
|
|
||
| - NebulaRestore | ||
|
|
||
| |Parameter|Default Value|Description| | ||
| |:---|:---|:---| | ||
| |`metadata.name`|-|The name of the resource object NebulaRestore.| | ||
| |`spec.br.clusterName`|-|The name of the backup cluster.| | ||
| |`spec.br.backupName`|-|The name of the backup file. Restore data based on this backup file.| | ||
| |`spec.br.concurrency`|`5`|The number of concurrent downloads when restoring data. The default value is `5`.| | ||
| |`spec.br.s3.region`|-| The geographical region where the S3 storage bucket is located.| | ||
| |`spec.br.s3.bucket`|-|The path of the S3 storage bucket where backup data is stored.| | ||
| |`spec.br.s3.endpoint`|-|The access address of the S3 storage bucket.| | ||
| |`spec.br.s3.secretName`|-|The name of the Secret that is used to access the S3 storage bucket.| | ||
|
|
||
| After setting up the YAML file for restoring the data, run the following command to start the restore job: | ||
|
|
||
| ```bash | ||
| kubectl apply -f <restore_file_name>.yaml | ||
| ``` | ||
|
|
||
| Run the following command to check the status of the NebulaRestore object. | ||
|
|
||
| ```bash | ||
| kubectl get rt <NebulaRestore_name> -w | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters