Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DAOS-13292 control: Use cart API to detect fabric #13989

Merged
merged 12 commits into from
Apr 29, 2024
Merged

DAOS-13292 control: Use cart API to detect fabric #13989

merged 12 commits into from
Apr 29, 2024

Conversation

kjacque
Copy link
Contributor

@kjacque kjacque commented Mar 15, 2024 •
edited
Loading

  • Add a lib/hardware package to collect fabric interface information through CART API.
  • Remove custom OFI and UCX packages and dependencies.
  • Update Go githook to ignore deleted files.

Required-githooks: true

Before requesting gatekeeper:

  • Two review approvals and any prior change requests have been resolved.
  • Testing is complete and all tests passed or there is a reason documented in the PR why it should be force landed and forced-landing tag is set.
  • Features: (or Test-tag*) commit pragma was used or there is a reason documented that there are no appropriate tags for this PR.
  • Commit messages follows the guidelines outlined here.
  • Any tests skipped by the ticket being addressed have been run and passed in the PR.

Gatekeeper:

  • You are the appropriate gatekeeper to be landing the patch.
  • The PR has 2 reviews by people familiar with the code, including appropriate owners.
  • Githooks were used. If not, request that user install them and check copyright dates.
  • Checkpatch issues are resolved. Pay particular attention to ones that will show up on future PRs.
  • All builds have passed. Check non-required builds for any new compiler warnings.
  • Sufficient testing is done. Check feature pragmas and test tags and that tests skipped for the ticket are run and now pass with the changes.
  • If applicable, the PR has addressed any potential version compatibility issues.
  • Check the target branch. If it is master branch, should the PR go to a feature branch? If it is a release branch, does it have merge approval in the JIRA ticket.
  • Extra checks if forced landing is requested
    • Review comments are sufficiently resolved, particularly by prior reviewers that requested changes.
    • No new NLT or valgrind warnings. Check the classic view.
    • Quick-build or Quick-functional is not used.
  • Fix the commit message upon landing. Check the standard here. Edit it to create a single commit. If necessary, ask submitter for a new summary.

@kjacque
DAOS-13292 control: Use cart API to detect fabric
63ead1b 
- Add a lib/hardware package to collect fabric interface
  information through CART API.
- Remove custom OFI and UCX packages and dependencies.
- Update Go githook to ignore deleted files.

Required-githooks: true

Signed-off-by: Kris Jacque <[email protected]>
@kjacque kjacque self-assigned this Mar 15, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 15, 2024 •
edited
Loading

Ticket title is 'Update control plane fabric scans to use new mercury API'
Status is 'In Review'
https://daosio.atlassian.net/browse/DAOS-13292

@daosbuild1
Copy link
Collaborator

Test stage Unit Test on EL 8.8 completed with status UNSTABLE. https://build.hpdd.intel.com/job/daos-stack/job/daos//view/change-requests/job/PR-13989/1/testReport/

mjmac
mjmac reviewed Mar 15, 2024
View reviewed changes
Copy link
Contributor

@mjmac mjmac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me... Much simpler to have a single provider.

src/control/lib/hardware/cart/cart.go Outdated Show resolved Hide resolved
kjacque added 2 commits March 29, 2024 23:44
@kjacque
Merge branch 'master' into kjacque/control-cart-fabric
6a52cc4 
Required-githooks: true
@kjacque
Address review comment
29ac9ac 
Features: control

Required-githooks: true

Signed-off-by: Kris Jacque <[email protected]>
@kjacque kjacque marked this pull request as ready for review March 30, 2024 00:01
@kjacque kjacque requested review from a team as code owners March 30, 2024 00:01
@kjacque kjacque requested review from tanabarr and knard38 March 30, 2024 00:02
@daosbuild1
Copy link
Collaborator

Test stage Unit Test on EL 8.8 completed with status UNSTABLE. https://build.hpdd.intel.com/job/daos-stack/job/daos//view/change-requests/job/PR-13989/2/testReport/

@daosbuild1
Copy link
Collaborator

Test stage Functional Hardware Medium completed with status UNSTABLE. https://build.hpdd.intel.com/job/daos-stack/job/daos//view/change-requests/job/PR-13989/2/testReport/

@daosbuild1
Copy link
Collaborator

Test stage Functional Hardware Medium Verbs Provider completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-13989/2/execution/node/1545/log

@kjacque
Compensate for DAOS-15588
2eca0b4 
For systems without Infiniband, getting info for verbs produces a Mercury
error. For all other providers, including UCX verbs, it returns no error
and instead returns no results. We'll simulate that behavior here until
the underlying bug is fixed.

Features: control

Signed-off-by: Kris Jacque <[email protected]>
@kjacque kjacque requested a review from mjmac April 2, 2024 23:50
@daosbuild1
Copy link
Collaborator

Test stage Functional Hardware Medium Verbs Provider completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-13989/4/execution/node/1540/log

@kjacque
Copy link
Contributor Author

kjacque commented Apr 8, 2024

Test failure is https://daosio.atlassian.net/browse/DAOS-15598

knard38
knard38 previously approved these changes Apr 9, 2024
View reviewed changes
Copy link
Contributor

@knard38 knard38 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

tanabarr
tanabarr previously approved these changes Apr 17, 2024
View reviewed changes
Copy link
Contributor

@tanabarr tanabarr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice work

@daosbuild1
Copy link
Collaborator

Test stage Functional Hardware Medium completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-13989/7/execution/node/1451/log

@kjacque kjacque dismissed stale reviews from knard38 and tanabarr via 3ded050 April 19, 2024 18:36
@daosbuild1
Copy link
Collaborator

Test stage Functional on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-13989/8/execution/node/1173/log

@kjacque
Merge branch 'master' into kjacque/control-cart-fabric
3f45b1e 
Excluding DAOSVersion.test_version test, which has a known failure.

Test-tag: pr control,-test_version
@kjacque
Copy link
Contributor Author

kjacque commented Apr 22, 2024

Test failure is DAOS-15686. I'm going to exclude the affected test and re-run.

@kjacque kjacque requested review from knard38 and tanabarr April 22, 2024 18:43
knard38
knard38 previously approved these changes Apr 23, 2024
View reviewed changes
tanabarr
tanabarr previously approved these changes Apr 23, 2024
View reviewed changes
mjmac
mjmac previously approved these changes Apr 23, 2024
View reviewed changes
@daosbuild1
Copy link
Collaborator

Test stage Functional Hardware Medium completed with status UNSTABLE. https://build.hpdd.intel.com/job/daos-stack/job/daos//view/change-requests/job/PR-13989/9/testReport/

@kjacque
Fix dmg_network_scan test
3f93fcd 
Don't include ofi+tcp;ofi_rxm when ofi+tcp is requested.

Features: control

Required-githooks: true

Signed-off-by: Kris Jacque <[email protected]>
@kjacque kjacque dismissed stale reviews from mjmac, tanabarr, and knard38 via 3f93fcd April 24, 2024 23:01
@kjacque kjacque requested review from a team as code owners April 24, 2024 23:01
@kjacque kjacque requested review from mjmac, tanabarr and knard38 April 24, 2024 23:02
daltonbohning
daltonbohning reviewed Apr 24, 2024
View reviewed changes
Copy link
Contributor

@daltonbohning daltonbohning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ftest LGTM

@daosbuild1
Copy link
Collaborator

Test stage Functional on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-13989/10/execution/node/1198/log

knard38
knard38 approved these changes Apr 25, 2024
View reviewed changes
Copy link
Contributor

@knard38 knard38 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
The functional test failing should not be related to this PR and to be a transient error corrupting the json output.

@kjacque kjacque requested a review from a team April 29, 2024 16:06
@frostedcmos frostedcmos merged commit e01970d into master Apr 29, 2024
53 checks passed
@frostedcmos frostedcmos deleted the kjacque/control-cart-fabric branch April 29, 2024 16:39
@frostedcmos
Copy link
Contributor

Discussed at gatekeeping and with @kjacque . This change re-uses same information that engines are already using, as such the landing risk is low in terms of breaking other systems/clusters. This change is also a right way to go forward, as we don't want to rely on libfabric/ucx apis and need to use unified cart-level ones for retrieval of the info.

@mjmac mjmac mentioned this pull request Apr 30, 2024
Closed
mjmac pushed a commit that referenced this pull request Apr 30, 2024
@kjacque @mjmac
DAOS-13292 control: Use cart API to detect fabric (#13989)
0227079 
- Add a lib/hardware package to collect fabric interface
  information through CART API.
- Remove custom OFI and UCX packages and dependencies.
- Update Go githook to ignore deleted files.

* Compensate for DAOS-15588

For systems without Infiniband, getting info for verbs produces a Mercury
error. For all other providers, including UCX verbs, it returns no error
and instead returns no results. We'll simulate that behavior here until
the underlying bug is fixed.

Signed-off-by: Kris Jacque <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@daltonbohning daltonbohning daltonbohning left review comments

@knard38 knard38 knard38 approved these changes

@tanabarr tanabarr tanabarr approved these changes

@mjmac mjmac Awaiting requested review from mjmac

Assignees

@kjacque kjacque

Labels
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@kjacque @daosbuild1 @frostedcmos @mjmac @tanabarr @daltonbohning @knard38