Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

WIP: add API to configure RDMA subsystem network namespace mode #156

Closed
wants to merge 4 commits into from
Closed

WIP: add API to configure RDMA subsystem network namespace mode #156

wants to merge 4 commits into from

Conversation

ykulazhenkov
Copy link
Collaborator

This PR introduces API to configure RDMA mode on nodes.

RDMA subsystem network namespace mode is a node-level configuration. Reboot required to change this option.

This PR relies on and extend SriovNetworkPoolConfig API proposed in #125

Replacement for #6

Status: WIP

API changes:

  • Add RDMAExclusiveMode to SriovNetworkPoolConfig

  • Add NodeSettings to SriovNetworkNodeState.Spec

Proposed workflow to apply node level configuration on k8s clusters:

  1. User create SriovNetworkPoolConfig CR
  2. SriovNetworkPoolConfig controller (single instance, part of sriov-network-operator POD) reconcile CR, and update SriovNetworkNodeState.Spec.NodeSettings for nodes which match selectors.
  3. sriov-network-config-daemon (runs on every node) read updated config from SriovNetworkNodeState CR and apply required node level configuration. We can use plugins to handle node level configuration.

zshi-redhat and others added 4 commits July 2, 2021 14:37
@zshi-redhat
Initialize sriovnetworkpoolconfig API
6dc134d
@zshi-redhat
OpenShift: enable OVS HW offload for multiple MCPs
4bd9cc9 
One node can only be added in one custom MCP pool by
design of MCO. Since current ovs-hw-offload MCs are
applied to hardcoded MCP ovs-hw-offload-worker, other
existing custome MCPs cannot be enabled with ovs hw
offload configuration.

This commits allows ovs-hw-offload MCs be applied
to multiple MCPs as specified in default SriovNetworkNodeConfig.
@ykulazhenkov
Add RDMAExclusiveMode field to SriovNetworkPoolConfig CRD
9fe9b3d 
RDMAExclusiveMode field holds policy for RDMA mode
configuration on nodes.

Add NodeSettings field to SriovNetworkNodeStateSpec.
This field is updated by SriovNetworkPoolConfig controller.
After update config daemon will apply required configuration
changes.

Signed-off-by: Yury Kulazhenkov <[email protected]>
@ykulazhenkov
Add RDMAPlugin plugin to support RDMA system mode switching
d563f27 
This plugin handle NodeSettings.RDMAExclusiveMode setting

Signed-off-by: Yury Kulazhenkov <[email protected]>
@zshi-redhat
Copy link
Collaborator

@ykulazhenkov Thanks for the PR!

I discussed with @pliurh on this PR and here are some suggestions:

  1. The mechanism used for passing interfaces configuration via SriovNetworkNodeState can also be used to pass node level configuration (SriovNetworkPoolConfig) to sriov config daemonset. We can have another field (in the same level with Interfaces) in SriovNetworkNodeState.Spec (just like what you did) for node specific configuration. The advantages of this are we don't have to create another CRD and informer (in config daemon) for the same purpose, existing plugins can be used w/o adding additional plugin interfaces.

  2. Regarding the plugin-rdma, we presume the rdma isolatoin is a generic feature that would apply to all vendor devices, would it make sense to add its function in generic_plugin (vs writing a separate plugin)?

@ykulazhenkov
Copy link
Collaborator Author

@zshi-redhat Thanks for the feedback.

Regarding the plugin-rdma, we presume the rdma isolatoin is a generic feature that would apply to all vendor devices, would it make sense to add its function in generic_plugin (vs writing a separate plugin)?

I agree, there is no need for the separate plugin for this. I will rework this.

@jjuele
Copy link
Contributor

jjuele commented Oct 28, 2022

Can one of the admins verify this patch?

@SchSeba
Copy link
Collaborator

SchSeba commented Nov 23, 2022

I don't think this PR is still active
I will close it please reopen if this is needed

@SchSeba SchSeba closed this Nov 23, 2022
@adrianchiris adrianchiris mentioned this pull request Dec 21, 2022
Closed
@adrianchiris adrianchiris mentioned this pull request Jul 20, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@ykulazhenkov @zshi-redhat @jjuele @SchSeba