Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generating label names #1200

Merged
merged 20 commits into from
May 29, 2020
Merged

Conversation

brybacki
Copy link
Contributor

@brybacki brybacki commented May 8, 2020

What this PR does / why we need it:

Removes unused labels holding resource names. If any label is used then its name will be shortened by provided function in package naming.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Special notes for your reviewer:

Release note:

Enhancement: DataVolume names are no longer restricted to 55 characters. The limit is determined by the runtime now.

@kubevirt-bot kubevirt-bot added release-note-none Denotes a PR that doesn't merit a release note. dco-signoff: yes Indicates the PR's author has DCO signed all their commits. size/S labels May 8, 2020
@kubevirt-bot kubevirt-bot requested review from awels and mhenriks May 8, 2020 13:32
@coveralls
Copy link

coveralls commented May 8, 2020

Coverage Status

Coverage increased (+0.2%) to 69.597% when pulling 6b8bc90 on brybacki:generating-label-names into b98e0a8 on kubevirt:master.

@awels
Copy link
Member

awels commented May 13, 2020

In general looks good, but can we think of any functional tests that we can do here? I don't want to merge again and having to add functional tests after the fact.

@brybacki
Copy link
Contributor Author

Datavolume webhook still guards names (len <=55) so I can only test import using pvc for this PR.
But when #1201 is finished, we can decide if we want to lift the name length limit, and follow with tests for import/upload/clone (to test if pod/pvc/label/service names are ok)

@awels
Copy link
Member

awels commented May 18, 2020

/test pull-containerized-data-importer-e2e-k8s-1.17-ceph

@awels
Copy link
Member

awels commented May 19, 2020

/retest

4 similar comments
@awels
Copy link
Member

awels commented May 19, 2020

/retest

@brybacki
Copy link
Contributor Author

/retest

@awels
Copy link
Member

awels commented May 20, 2020

/retest

@awels
Copy link
Member

awels commented May 20, 2020

/retest

@awels
Copy link
Member

awels commented May 20, 2020

@brybacki
Copy link
Contributor Author

pushed 'Focused' test by mistake, and now after turning off the focus I have found a problem to fix

Copy link
Member

@awels awels left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some questions, in particular why the annotation to store/retrieve the pod name? Is the call to the naming function not deterministic based on the pvc name?

@@ -247,6 +285,7 @@ func (r *ImportReconciler) updatePvcFromPod(pvc *corev1.PersistentVolumeClaim, p
log.V(1).Info("Pod requires scratch space, terminating pod, and restarting with scratch space", "pod.Name", pod.Name)
scratchExitCode = true
anno[AnnRequiresScratch] = "true"
anno[AnnScratchPvc] = scratchNameFromPvc(pvc)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one really needed? Looks like we create it above when we determine that we need scratch space.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see my last comment on storing names in annotations

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually not, we create scratch name once, and then it is always stored in a Volume definition on the pod

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so I have removed this annotation, as it is not needed, and change to read from pod spec, if available

pod := &corev1.Pod{}
err := r.client.Get(context.TODO(), types.NamespacedName{Name: podName, Namespace: pvc.GetNamespace()}, pod)

if k8serrors.IsNotFound(err) {
return nil, nil
}

/// TODO: what about other errors? // check upload
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what you mean by this comment.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if r.client.Get does not wok (service error, temporary network error) then we go straight to check if pod is controlled by pvc, and the only error we can see is Pod is not owned by PVC.

	if err := r.client.Get(context.TODO(), types.NamespacedName{Name: podName, Namespace: pvc.Namespace}, pod); err != nil {
		if !k8serrors.IsNotFound(err) {
			return nil, errors.Wrapf(err, "error getting import pod %s/%s", pvc.Namespace, podName)
		}
		return nil, nil
	}

	if !metav1.IsControlledBy(pod, pvc) {
		return nil, errors.Errorf("%s pod not controlled by pvc %s", podName, pvc.Name)
	}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have actually applied the change from previous comment

if err := r.createImporterPod(pvc); err != nil {
return reconcile.Result{}, err

_, ok := pvc.Annotations[AnnImportPod]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is more a stylistic thing than a code problem, but I would write this as:

if _, ok := pvc.Annotations[AnnImportPod]; ok {
...
} else {
...
}

Second, what is the difference between initPvcPodName vs createImporterPod. I see initPvcPodName is creating the name and adding it to the annotations, but can't we do that in createImporterPod as well? I am trying to understand why this is split.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was started with a plan to change name generation logic, but now as we can see the name generation is backwards compatible, so this does not make sense/does not help, I can simplify

pkg/controller/import-controller.go Show resolved Hide resolved
}

// createUploadResourceName returns the name given to upload resources
func createUploadResourceName(name string) string {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I am a little confused at this point, this function is called always with the pvc name, and the result is stored in the annotation. But why are we doing that? This should be a deterministic result based on the pvc name, so instead of storing and reading from the annotation, why not simply call this function whenever we are interested in finding the pod name based on the pvc name?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So you are right, our function is now deterministic, and currently storing in annotation does not help us. So it has no value now, but might help us in the future:

  • if we decide to resolve duplicate names/conflicts by trying some other name.
  • if we decide to make name generation more random/unique.
  • if we decide to 'change' the naming method, we might use annotation for backwards compatibility.

Now after you comment, I think, this "store in annotation" commit", and the next one, are nice to have, but really are maybe over engineering for current state.

I mean - keeping it here, and improving some tests might be beneficial (for the future needs?), but removing it - might simplify the code. So I am open for hearing arguments for one of this directions. Use annotations to store names, or do not (since naming is deterministic)

@mhenriks @awels What do you think?

@kubevirt-bot kubevirt-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 25, 2020
@brybacki brybacki force-pushed the generating-label-names branch from 40913de to 3962833 Compare May 26, 2020 10:32
@kubevirt-bot kubevirt-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 26, 2020
brybacki added 19 commits May 29, 2020 15:16
Signed-off-by: Bartosz Rybacki <[email protected]>
Signed-off-by: Bartosz Rybacki <[email protected]>
Signed-off-by: Bartosz Rybacki <[email protected]>
The situation of having a pod but not name on annotation can happen after the upgrade, when we have a legacy pvc and pod already existing, but clone operation not finished. But when we already have the pod, then in the code (currently) we do not need the name from annotation.

Signed-off-by: Bartosz Rybacki <[email protected]>
Signed-off-by: Bartosz Rybacki <[email protected]>
@brybacki brybacki force-pushed the generating-label-names branch from c4e82c0 to 6b8bc90 Compare May 29, 2020 13:23
Copy link
Member

@awels awels left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve

@kubevirt-bot kubevirt-bot added the lgtm Indicates that a PR is ready to be merged. label May 29, 2020
@kubevirt-bot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: awels

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubevirt-bot kubevirt-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 29, 2020
@awels
Copy link
Member

awels commented May 29, 2020

/test pull-containerized-data-importer-e2e-os-3.11.0-crio

@kubevirt-bot kubevirt-bot merged commit ab8b9c0 into kubevirt:master May 29, 2020
@kubevirt-bot kubevirt-bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed release-note-none Denotes a PR that doesn't merit a release note. labels May 29, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. dco-signoff: yes Indicates the PR's author has DCO signed all their commits. lgtm Indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XL
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants