Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Copy labels from source to DataSource #3377

Merged
merged 6 commits into from
Aug 16, 2024

Conversation

0xFelix
Copy link
Member

@0xFelix 0xFelix commented Aug 8, 2024

What this PR does / why we need it:

Copy labels from the source of a DataSource (DV/PVC/VolumeSnapshot) to the labels of the DataSource. This allows to make use of labels which were originally defined as env vars on a containerdisk.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Special notes for your reviewer:

Release note:

Copy labels from the source of a DataSource (DV/PVC/VolumeSnapshot)

Extract the label copying logic from populator-base.go into the common
pkg as CopyAllowedLabels func.

Signed-off-by: Felix Matouschek <[email protected]>
@kubevirt-bot kubevirt-bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. dco-signoff: yes Indicates the PR's author has DCO signed all their commits. size/L labels Aug 8, 2024
@kubevirt-bot kubevirt-bot requested review from awels and mhenriks August 8, 2024 14:35
Comment on lines 155 to 157
Expect(ds.Labels).To(HaveKeyWithValue(testKubevirtIoKey, testKubevirtIoValue))
Expect(ds.Labels).To(HaveKeyWithValue(testInstancetypeKubevirtIoKey, testInstancetypeKubevirtIoValue))
Expect(ds.Labels).To(HaveKeyWithValue(testKubevirtIoKeyExisting, testKubevirtIoNewValueExisting))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you're not guaranteed to have the labels when reaching "Ready=true". You could use eventually here

@coveralls
Copy link

coveralls commented Aug 9, 2024

Coverage Status

coverage: 59.187% (-0.001%) from 59.188%
when pulling 05a9731 on 0xFelix:copy-labels
into e1a553a on kubevirt:main.

@0xFelix 0xFelix force-pushed the copy-labels branch 2 times, most recently from f2478af to 0f46fec Compare August 9, 2024 12:08
@0xFelix
Copy link
Member Author

0xFelix commented Aug 9, 2024

/retest-required

Copy link
Collaborator

@akalenyu akalenyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! main worry is ofc the func test failure/flake which essentially means a user snap could be missing the labels

pkg/controller/common/util.go Show resolved Hide resolved
pkg/controller/dataimportcron-controller.go Show resolved Hide resolved
tests/dataimportcron_test.go Show resolved Hide resolved
tests/datasource_test.go Outdated Show resolved Hide resolved
pkg/controller/datasource-controller.go Show resolved Hide resolved
@akalenyu
Copy link
Collaborator

/test pull-containerized-data-importer-e2e-nfs
ignore the nfs lane, this is a known issue, but the ceph lane failure is surely something we care about lol

@0xFelix
Copy link
Member Author

0xFelix commented Aug 12, 2024

/retest

Make copying of labels from a prime PVC to the target PVC more robust,
by moving it before rebinding the PV from prime to target. This way we
can ensure the labels are already present once the PVC becomes ready.

Signed-off-by: Felix Matouschek <[email protected]>
Do not pass labels from a DataImportCron to a DataSource in the
dataimportcron-controller anymore. In the future this will be
handled by the datasource-controller.

Signed-off-by: Felix Matouschek <[email protected]>
Copy labels from the source of a DataSource to the labels of the
DataSource in the datasource-controller.

Signed-off-by: Felix Matouschek <[email protected]>
Add e2e tests that cover all scenarios where labels should be copied
from the source of a DataSource to the DataSource itself.

Signed-off-by: Felix Matouschek <[email protected]>
@akalenyu
Copy link
Collaborator

/test all
just want to see that we got rid of the flakes

@0xFelix
Copy link
Member Author

0xFelix commented Aug 13, 2024

/retest

@0xFelix
Copy link
Member Author

0xFelix commented Aug 13, 2024

Fail of NFS lane was unrelated

@akalenyu
Copy link
Collaborator

/test all

@0xFelix
Copy link
Member Author

0xFelix commented Aug 14, 2024

Failures are unrelated to the changes, the cluster failed to come up.

/retest

@0xFelix 0xFelix force-pushed the copy-labels branch 2 times, most recently from 53051e8 to f8b57cb Compare August 14, 2024 07:56
@0xFelix
Copy link
Member Author

0xFelix commented Aug 14, 2024

/retest

@mhenriks
Copy link
Member

/retest-required

return reconcile.Result{}, err
}
}
}

if pvcCopy, err = r.updatePVCWithPVCPrimeAnnotations(pvcCopy, pvcPrime, r.updateImportAnnotations); err != nil {
if _, err = r.updatePVCWithPVCPrimeAnnotations(pvcCopy, pvcPrime, r.updateImportAnnotations); err != nil {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want the same treatment for annotations? #3381 (comment)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines 730 to 736
updatedSnapshot := currentSnapshot.DeepCopy()
cc.CopyAllowedLabels(pvc.GetLabels(), updatedSnapshot, true)
if !reflect.DeepEqual(currentSnapshot, updatedSnapshot) {
if err := r.client.Update(ctx, updatedSnapshot); err != nil {
return err
}
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this would solve the case for an existing snapshots with no labels, maybe we could just do something similar to 755fe0b by copying the labels from the data source to the snapshot. This would only be needed for a brief time window before the cron gets a sha update.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the approach to match 755fe0b, PTAL!

@0xFelix
Copy link
Member Author

0xFelix commented Aug 15, 2024

/test pull-cdi-goveralls

@0xFelix
Copy link
Member Author

0xFelix commented Aug 15, 2024

/test pull-cdi-apidocs

@0xFelix
Copy link
Member Author

0xFelix commented Aug 15, 2024

/test pull-containerized-data-importer-e2e-destructive

@@ -377,6 +379,11 @@ func (r *DataImportCronReconciler) update(ctx context.Context, dataImportCron *c
cc.AddAnnotation(snapshot, cc.AnnSourceVolumeMode, string(*volMode))
}
}
// Copy labels found on dataSource to the existing snapshot in case of upgrades.
dataSource, err := r.getDataSource(ctx, dataImportCron)
if err == nil {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we're "forgetting" to requeue in case the GET fails for a random network err

When using VolumeSnapshots copy the labels found on the source PVC to
the created or an existing VolumeSnapshot.

Signed-off-by: Felix Matouschek <[email protected]>
Copy link
Collaborator

@akalenyu akalenyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

thanks!

@kubevirt-bot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: akalenyu

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubevirt-bot kubevirt-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 15, 2024
@kubevirt-bot
Copy link
Contributor

kubevirt-bot commented Aug 15, 2024

@0xFelix: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-cdi-apidocs 05a9731 link false /test pull-cdi-apidocs

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@mhenriks
Copy link
Member

/test pull-cdi-unit-test

@mhenriks
Copy link
Member

/test pull-containerized-data-importer-e2e-upg

@mhenriks
Copy link
Member

/lgtm

@kubevirt-bot kubevirt-bot added the lgtm Indicates that a PR is ready to be merged. label Aug 15, 2024
@kubevirt-bot kubevirt-bot merged commit f357368 into kubevirt:main Aug 16, 2024
18 of 19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. dco-signoff: yes Indicates the PR's author has DCO signed all their commits. lgtm Indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants