(GCP) Users should be able to select a snapshot type (archive or standard) when creating backups

[tuusberg]

Describe the problem/challenge you have

We are using Velero extensively to back up our entire production network. Hundreds of blockchain node backups produce thousands of snapshots and it's costing us a lot of money to keep them around. GCP introduced archive snapshots that share the same interface with the standard ones, which means Velero should be able to seamlessly work with both types. Using archive snapshots would reduce cloud costs for backups that have larger TTLs.

Describe the solution you'd like

I would like to be able to select a SnapshotType upon creating a backup. The standard type can be made the default and users would explicitly need to select archive type if they need to.

Anything else you would like to add:

Environment:

• Velero version (use velero version): 1.13.0
• Kubernetes version (use kubectl version): 1.27.7
• Cloud provider or hardware configuration: GCP/GKE

Vote on this issue!

This is an invitation to the Velero community to vote on issues, you can see the project's top voted issues listed here.
Use the "reaction smiley face" up to the right of this comment to vote.

• 👍 for "The project would be better with this feature added"
• 👎 for "This feature will not enhance the project in a meaningful way"

[danfengliu]

Thanks for bring the requirement which is important in certain scenario!

[blackpiglet]

@tuusberg
Hi, how does your environment create snapshots, using the Velero Native Snapshot or the Velero CSI plugin?

• The Velero can support the snapshot type chosen by the Velero Native Snapshot because the Velero GCP plugin can control the Snapshot Creation Request content. I can make some code changes to support it.
• Still, the Velero CSI plugin cannot support it yet. The reason is the CSI external snapshotter automatically creates the snapshot creation request, and the content is read from the VolumeSnapshotClass, but the GCP PD CSI driver doesn't support choosing the snapshot type.

PS: There is a supported parameter called ParameterKeySnapshotType, but its valid values are images and snapshots. It is not used to choose the snapshot types.

https://github.com/kubernetes-sigs/gcp-compute-persistent-disk-csi-driver/blob/e75b9a3ae6c770daaa9d336298ad94d0372ad809/pkg/common/parameters.go#L236-L241

[tuusberg]

Hi @blackpiglet,

We are creating snapshots using Velero Native Snapshot. I've already found the place that needs to be changed in order to get it working. The snapshot type could passed to the struct here, that would do the trick. More than happy to open a pull request, but wanted to make sure what I am planning to do is correct and this sort of change is something Velero community would benefit from.

[blackpiglet]

@tuusberg
Thanks. I think we can add the snapshot type setting in the VolumeSnapshotLocation.Spec.Config.

And init the snapshotType by the Init method.

type VolumeSnapshotter struct {
	log              logrus.FieldLogger
	gce              *compute.Service
	snapshotLocation string
	volumeProject    string
	snapshotProject  string
        snapshotType      string
}

[tuusberg]

Would that change make it configurable on a per-schedule basis? I am trying to achieve the following: Backups with long TTL (e.g. weekly and monthly backup schedules that have TTLs of 1 year/10 years, respectively) would use archive snapshots and hourly/daily would use standard ones. Would that be possible to achieve the described behavior by adding snapshotType to VolumeSnapshotLocation.Spec.Config?

[blackpiglet]

The VolumeSnapshotLocation setting is not per schedule or backup, but the Velero supports multiple VolumeSnapshotLocations.
Both the backup and the schedule can select which VolumSnapshotLocation to use.

[tuusberg]

that would work! So we need to make these code changes, then create a separate VolumeSnapshotLocation that will have its snapshotType set to Archive and use that VolumeSnapshotLocation for certain schedules. I will start working on a PR for it closer to the end of this week. Thanks for sharing the knowledge!

[okuuva]

I'm also in need of this and interested in contributing. @tuusberg let me know if I can help!