-
Notifications
You must be signed in to change notification settings - Fork 412
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
initial pass at populating the os version label #514
initial pass at populating the os version label #514
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some comments in line.
Also is there an open issue somewhere for this? Also since there is a bug fix is there approval to put this in 4.0 because the MCO is putting all new enchancements/features into our master-4.1
branch at this time.
cc: @runcom
pkg/controller/template/test_data/templates/worker/01-worker-kubelet/aws/units/kubelet.service
Outdated
Show resolved
Hide resolved
...ntroller/template/test_data/templates/worker/01-worker-kubelet/libvirt/units/kubelet.service
Outdated
Show resolved
Hide resolved
pkg/controller/template/test_data/templates/worker/01-worker-kubelet/none/units/kubelet.service
Outdated
Show resolved
Hide resolved
Until there is confirmation that this should go into 4.0 /hold |
ab33c57
to
40755e2
Compare
@kikisdeliveryservice changed to the 4.1 branch |
There has been some discussion on aos-devel regarding how networking can figure out if the underlying OS is RHEL7 or RHEL8. |
Thanks @rphillips ! |
40755e2
to
e4f2339
Compare
changes made as requested, will allow others to review.
this is only needed for 4.1. In 4.0, we know a node not labeled is rhel7, and in 4.1, pods that require rhel8 will have nodeSelectors that will hold scheduling until a label is present. In 4.1, rhel7 workers are new nodes, and would have newly registered labels which should be fine with general approach. |
@@ -20,7 +20,7 @@ contents: | | |||
--container-runtime=remote \ | |||
--container-runtime-endpoint=/var/run/crio/crio.sock \ | |||
--allow-privileged \ | |||
--node-labels=node-role.kubernetes.io/worker \ | |||
--node-labels=node-role.kubernetes.io/worker,node.openshift.io/os_version=${OS_VERSION} \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not just add /etc/os-release
as an EnvironmentFile and pull it out from there?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
that does simplify things. fixed!
e4f2339
to
44f321c
Compare
Should we add similar support so that we can differentiate between RHCOS and RHEL Server or other? |
@sdodson which variable would you like to add from os-version? |
This looks good to me. Just to remind folks, you'll end up getting a version like this on RHCOS:
Related: |
The variable ID from /etc/os-release seems like a good source for the value but we probably want to name the label differently. os_id perhaps? |
@sdodson My general thought was to have os_version correspond to ${VERSION} within the os-release. If we want to add more variables from os-release, then they might have a 1 to 1 mapping. os_id would map to ${ID} within os-release. |
Yes, that's what I meant, an additional label for os_id. |
I think this PR is ready for approval unless there are any further comments. Edit: I can add that id in. |
@sdodson added the ID |
/retest |
2 similar comments
/retest |
/retest |
@kikisdeliveryservice after talking to @sjenning, we need this in 4.0. I've switched the target branch back to master. |
thanks for the update @rphillips Could you amend the body of the commit messages to add a little more info? For ex, os version using: "Enumerates the OS version from /etc/os-release and creates a node.openshift.io/os_version={OS_VERSION} label." and something substantive for the id label commit. |
Any comments on this PR? |
Some operators will need to know the underlying operating system version to work (ie: networking, etc). This information is especially needed if nodes are upgraded from RHEL7 to RHEL8. The kubelet adds labels when it registers for the first time. There will be changes in the machine-config-daemon or NFD to support relabelling the node when the OS has upgraded.
5795ac9
to
e781673
Compare
@kikisdeliveryservice updated the commit message and comment. |
thanks @rphillips ! This looks good. Can verify that labels were created for each node. Checking e2e-aws/artifacts/nodes.json. For ex:
|
code lgtm, I'd love @sjenning to approve |
/lgtm |
/approve |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: kikisdeliveryservice, rphillips, sjenning The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
/test e2e-rhel-scaleup |
@@ -19,7 +20,7 @@ contents: | | |||
--container-runtime=remote \ | |||
--container-runtime-endpoint=/var/run/crio/crio.sock \ | |||
--allow-privileged \ | |||
--node-labels=node-role.kubernetes.io/master \ | |||
--node-labels=node-role.kubernetes.io/master,node.openshift.io/os_version=${VERSION},node.openshift.io/os_id=${ID} \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This breaks RHEL scaleup:
kubelet_node_status.go:92] Unable to register node "ip-10-0-156-42.ec2.internal" with API server: Node "ip-10-0-156-42.ec2.internal" is invalid: metadata.labels: Invalid value: "7 (Core)": a valid label must be an empty string or consist of alphanumeric characters, ''-'', ''_'' or ''.'', and must start and end with an alphanumeric character (e.g. ''MyValue'', or ''my_value'', or ''12345'', regex used for validation is ''(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?'')'
This removes the functionality added in openshift#514 which I don't think really worked for what people needed; a node selector can't parse the value. All we care about is currently 7 or 8; this signals things like `iptables` vs `nftables`. And the presence of this label also signals "RHEL" too; though we keep the `os_id` label in case e.g. a daemonset should only run on RHCOS (or traditional RHEL). Closes: openshift#582
- What I did
Enumerates the OS version from /etc/os-release and creates a
node.openshift.io/os_version={OS_VERSION}
label.- How to verify it
oc get node [node name] -o yaml
- Description for the changelog
Add an initial OS Version to label (
node.openshift.io/os_version={OS_VERSION}
) to nodes.- Some background info
Kubelet --node-labels only get created on kubelet registration, so changes will need to happen with a daemonset. Liggit has a number of comments in upstream saying the kubelet should not update labels, so a custom patch would probably never be merged upstream (IMO). There are differing opinions on the ticket.
This PR is phase 1) create the label with the starting version. The second phase will be to create a daemonset to update the label on RHEL7->RHEL8 upgrades. A final third phase, when RHEL7 is not supported anymore, we can retire the daemonset and delete the code/daemonset.
/cc @jeremyeder @sjenning @smarterclayton @derekwaynecarr