Improve Deletion Secret Handling

[saad-ali]

This is the same underlying issue as kubernetes-csi/external-provisioner#330 for external-provisioner. This bug is to track the work for the external-snapshotter. We should make sure the annotations used for both solution have the same name.

Today, the external-snapshotter will look at VolumeSnapshotClass for deletion secret. If the secret does not exist, delete is called any way.

The problem is that the VolumeSnapshotClass may be deleted or mutated (deleted and recreated with different parameters). This will result in snapshots being unable to delete. Ideally we want the deletion secret on the VolumeSnapshotContent object. However, @liggitt pointed out that this would result in asymmetry of the API (provision is handled by a higher layer controller and specified in VolumeSnapshotClass, having that controller then look at VolumeSnapshotContent object for the delete secret seems wrong). That said, we do want to better handle this case.

So as a compromise, the proposal is to 1) add a reference to the deletion secret as an annotation on the VolumeSnapshotContent object (instead of a first class field), and 2) to better document why you shouldn't have deletion secrets.

For 1, the proposed change is to add a new flag to external-snapshotter that says controller requires deletion secret, if this is set by SP, the external-snapshotter should store a reference to the provision secret in an annotation on the PV object, and when deleting, if the flag is set and the VolumeSnapshotContent object has the annotation, fetch and pass the secret in the CSI DeleteSnapshot call.

And 2 is being tacked in kubernetes-csi/docs#189 (comment).

CC @jingxu97 @xing-yang

[xing-yang]

CC @yuxiangqian