DAOS-16326 control: Update pool query UX for MD-on-SSD phase2 #14844

tanabarr · 2024-07-30T18:51:41Z

Display MD-on-SSD specifics on pool query including VOS index file
sizes. Tier-0 space usage reported is that of MD-on-SSD blobs and in
phase2 mode these two sizes will be distinct.

Required-githooks: true

Before requesting gatekeeper:

Two review approvals and any prior change requests have been resolved.
Testing is complete and all tests passed or there is a reason documented in the PR why it should be force landed and forced-landing tag is set.
Features: (or Test-tag*) commit pragma was used or there is a reason documented that there are no appropriate tags for this PR.
Commit messages follows the guidelines outlined here.
Any tests skipped by the ticket being addressed have been run and passed in the PR.

Gatekeeper:

github-actions · 2024-07-30T18:51:59Z

Ticket title is 'Update dmg pool query for MD-on-SSD P2'
Status is 'In Review'
Labels: 'control-plane,md_on_ssd2,usability'
https://daosio.atlassian.net/browse/DAOS-16326

daosbuild1 · 2024-07-30T18:55:30Z

Test stage Build on EL 8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14844/1/execution/node/358/log

daosbuild1 · 2024-07-30T18:55:57Z

Test stage Build on Leap 15.5 with Intel-C and TARGET_PREFIX completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14844/1/execution/node/387/log

daosbuild1 · 2024-07-30T18:56:37Z

Test stage Build RPM on EL 9 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14844/1/execution/node/348/log

daosbuild1 · 2024-07-30T18:57:23Z

Test stage Build RPM on EL 8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14844/1/execution/node/363/log

daosbuild1 · 2024-07-30T18:58:11Z

Test stage Build RPM on Leap 15.5 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14844/1/execution/node/273/log

daosbuild1 · 2024-07-30T19:02:10Z

Test stage Build DEB on Ubuntu 20.04 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14844/1/execution/node/313/log

daosbuild1 · 2024-08-08T11:23:54Z

Test stage Build on Leap 15.5 with Intel-C and TARGET_PREFIX completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14844/2/execution/node/372/log

daosbuild1 · 2024-08-08T11:25:15Z

Test stage Build RPM on EL 9 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14844/2/execution/node/365/log

daosbuild1 · 2024-08-08T11:25:31Z

Test stage Build RPM on EL 8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14844/2/execution/node/362/log

daosbuild1 · 2024-08-08T11:26:53Z

Test stage Build RPM on Leap 15.5 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14844/2/execution/node/334/log

daosbuild1 · 2024-08-08T11:31:06Z

Test stage Build DEB on Ubuntu 20.04 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14844/2/execution/node/361/log

daosbuild1 · 2024-08-08T11:31:20Z

Test stage Build on EL 8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14844/2/execution/node/494/log

daosbuild1 · 2024-08-08T16:16:16Z

Test stage Build on Leap 15.5 with Intel-C and TARGET_PREFIX completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14844/3/execution/node/372/log

daosbuild1 · 2024-08-08T16:17:41Z

Test stage Build RPM on EL 9 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14844/3/execution/node/361/log

daosbuild1 · 2024-08-08T16:19:36Z

Test stage Build RPM on EL 8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14844/3/execution/node/355/log

daosbuild1 · 2024-08-08T16:21:10Z

Test stage Build RPM on Leap 15.5 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14844/3/execution/node/362/log

daosbuild1 · 2024-08-08T16:23:35Z

Test stage Build DEB on Ubuntu 20.04 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14844/3/execution/node/348/log

daosbuild1 · 2024-08-08T16:23:58Z

Test stage Build on EL 8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14844/3/execution/node/494/log

daosbuild1 · 2024-08-08T23:39:04Z

Test stage Functional on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14844/4/execution/node/1208/log

daosbuild1 · 2024-08-09T17:37:42Z

Test stage Functional Hardware Large completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14844/4/execution/node/1425/log

daosbuild1 · 2024-08-10T00:53:51Z

Test stage Functional Hardware Medium completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14844/4/execution/node/1526/log

daosbuild1 · 2024-08-22T05:19:02Z

Test stage Functional Hardware Large completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14844/8/execution/node/1417/log

knard-intel

Mostly good for me.
Just one detail not preventing landing of this PR.

knard-intel · 2024-08-22T12:07:46Z

src/control/cmd/dmg/pretty/pool_test.go

@@ -582,6 +582,33 @@ one  6.0 TB Ready 83%%  16%%       0/16
 			verbose:     true,
 			expPrintStr: msgNoPools + "\n",
 		},
+		"verbose, two pools": {


Suggested change

"verbose, two pools": {

"verbose, one pool": {

daosbuild1 · 2024-08-22T14:47:22Z

Test stage Functional Hardware Medium completed with status UNSTABLE. https://build.hpdd.intel.com/job/daos-stack/job/daos//view/change-requests/job/PR-14844/8/testReport/

sherintg · 2024-08-23T08:37:48Z

src/control/cmd/dmg/pool.go

-	defaultTierRatios = []float64{0.06, 0.94}
+	defaultTierRatios         = []float64{0.06, 0.94}
+	errPoolCreateIncompatOpts = errors.New("unsupported option combination, use (--scm-size and " +
+		"--nvme-size) or (--scm-size and --nvme-size) or (--size)")


shouldn't this be (--scm-size and " + --nvme-size) or (--meta-size and --data-size)?

yes, well spotted thanks

kjacque

Looks okay to me outside of the issues noted by other reviewers. The issue @sherintg noted needs to be fixed.

…abarr/control-display-poolquery-mdonssd Required-githooks: true Signed-off-by: Tom Nabarro <[email protected]>

Features: control pool Allow-unstable-test: true Required-githooks: true Signed-off-by: Tom Nabarro <[email protected]>

tanabarr · 2024-08-27T22:17:47Z

functional test failures need addressing

daosbuild1 · 2024-08-28T16:51:52Z

Test stage Functional Hardware Large completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14844/9/execution/node/1454/log

daosbuild1 · 2024-09-02T12:01:49Z

Test stage Functional Hardware Large completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14844/10/execution/node/505/log

daosbuild1 · 2024-09-02T19:32:39Z

Test stage Functional Hardware Medium completed with status UNSTABLE. https://build.hpdd.intel.com/job/daos-stack/job/daos//view/change-requests/job/PR-14844/10/testReport/

…abarr/control-display-poolquery-mdonssd Signed-off-by: Tom Nabarro <[email protected]>

Features: control pool Allow-unstable-test: true Required-githooks: true Signed-off-by: Tom Nabarro <[email protected]>

daltonbohning

ftest LGTM

daltonbohning · 2024-09-04T14:33:25Z

src/tests/ftest/control/dmg_pool_query_test.yaml

@@ -35,7 +35,7 @@ exp_vals:
  replicas: [0]
  query_mask: "rebuild,space"
  scm:
-    total: 16000008192
+    total: 16000000000


Is this test this picky or are you just making this a clean number?

This is now being specified as base-10 not base-2, I don't know whether that's a result of this PR or an existing failure on the branch. I don't have anything to compare with that is running features pool and control. I expect that it's a result of now reading the SCM quantity from meta-blob size which I imagine is specified in base-10 as its storage rather than the VOS-file size in RAM/memory which is specified in base-2.

Because pool create uses scm_size: 16GB, this looks correct to me. Thanks for the explanation

daosbuild1 · 2024-09-05T19:17:23Z

Test stage Functional Hardware Large completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14844/12/execution/node/1416/log

daosbuild1 · 2024-09-06T06:17:51Z

Test stage Functional Hardware Medium completed with status UNSTABLE. https://build.hpdd.intel.com/job/daos-stack/job/daos//view/change-requests/job/PR-14844/12/testReport/

…abarr/control-display-poolquery-mdonssd Required-githooks: true Signed-off-by: Tom Nabarro <[email protected]>

Features: control pool Allow-unstable-test: true Required-githooks: true Signed-off-by: Tom Nabarro <[email protected]>

daosbuild1 · 2024-09-08T00:29:19Z

Test stage Functional Hardware Large completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14844/13/execution/node/1508/log

tanabarr · 2024-09-08T18:03:27Z

https://build.hpdd.intel.com/blue/organizations/jenkins/daos-stack%2Fdaos/detail/PR-14844/13/tests/ hardware large failed in CI due to https://daosio.atlassian.net/browse/DAOS-16496 related issues

Logging macro checking and Linting Summary checks fail on unchanged code

Landing to feature branch

tanabarr · 2024-09-08T19:14:08Z

commit title mistakenly referenced DAOS-14422 but should reference DAOS-16326

tanabarr self-assigned this Jul 30, 2024

tanabarr added control-plane work on the management infrastructure of the DAOS Control Plane meta-on-ssd Metadata on SSD Feature usability Changes specific to user facing tools or behaviour. labels Jul 30, 2024

tanabarr force-pushed the tanabarr/control-display-poolquery-mdonssd branch from 6992000 to 91a5b89 Compare August 8, 2024 11:20

tanabarr force-pushed the tanabarr/control-display-poolquery-mdonssd branch from 91a5b89 to 02681f6 Compare August 8, 2024 16:12

tanabarr force-pushed the tanabarr/control-display-poolquery-mdonssd branch from 02681f6 to 993793f Compare August 8, 2024 17:06

tanabarr mentioned this pull request Aug 20, 2024

DAOS-14422 control: Update pool create UX for MD-on-SSD phase2 #14740

Merged

18 tasks

tanabarr requested a review from sherintg August 21, 2024 22:42

knard-intel approved these changes Aug 22, 2024

View reviewed changes

sherintg reviewed Aug 23, 2024

View reviewed changes

kjacque reviewed Aug 26, 2024

View reviewed changes

kjacque approved these changes Aug 27, 2024

View reviewed changes

tanabarr added 2 commits August 27, 2024 19:20

Merge remote-tracking branch 'origin/feature/vos_on_blob_p2' into tan…

76dced3

…abarr/control-display-poolquery-mdonssd Required-githooks: true Signed-off-by: Tom Nabarro <[email protected]>

correct help message

62c48c7

Features: control pool Allow-unstable-test: true Required-githooks: true Signed-off-by: Tom Nabarro <[email protected]>

sherintg approved these changes Sep 2, 2024

View reviewed changes

tanabarr added 2 commits September 4, 2024 13:05

Merge remote-tracking branch 'origin/feature/vos_on_blob_p2' into tan…

5762199

…abarr/control-display-poolquery-mdonssd Signed-off-by: Tom Nabarro <[email protected]>

fix ftest failures

8b78d43

Features: control pool Allow-unstable-test: true Required-githooks: true Signed-off-by: Tom Nabarro <[email protected]>

tanabarr requested review from a team as code owners September 4, 2024 13:58

daltonbohning reviewed Sep 4, 2024

View reviewed changes

tanabarr added 2 commits September 6, 2024 12:06

Merge remote-tracking branch 'origin/feature/vos_on_blob_p2' into tan…

8ea108f

…abarr/control-display-poolquery-mdonssd Required-githooks: true Signed-off-by: Tom Nabarro <[email protected]>

fix another ftest failure

bfbf355

Features: control pool Allow-unstable-test: true Required-githooks: true Signed-off-by: Tom Nabarro <[email protected]>

tanabarr added the forced-landing The PR has known failures or has intentionally reduced testing, but should still be landed. label Sep 8, 2024

tanabarr merged commit aa2cba4 into feature/vos_on_blob_p2 Sep 8, 2024
54 of 58 checks passed

tanabarr deleted the tanabarr/control-display-poolquery-mdonssd branch September 8, 2024 18:12

tanabarr changed the title ~~DAOS-14422 control: Update pool query UX for MD-on-SSD phase2~~ DAOS-16326 control: Update pool query UX for MD-on-SSD phase2 Sep 8, 2024

DAOS-16326 control: Update pool query UX for MD-on-SSD phase2 #14844

DAOS-16326 control: Update pool query UX for MD-on-SSD phase2 #14844

Conversation

tanabarr commented Jul 30, 2024 • edited Loading

Before requesting gatekeeper:

Gatekeeper:

github-actions bot commented Jul 30, 2024 • edited Loading

daosbuild1 commented Jul 30, 2024

daosbuild1 commented Jul 30, 2024

daosbuild1 commented Jul 30, 2024

daosbuild1 commented Jul 30, 2024

daosbuild1 commented Jul 30, 2024

daosbuild1 commented Jul 30, 2024

daosbuild1 commented Aug 8, 2024

daosbuild1 commented Aug 8, 2024

daosbuild1 commented Aug 8, 2024

daosbuild1 commented Aug 8, 2024

daosbuild1 commented Aug 8, 2024

daosbuild1 commented Aug 8, 2024

daosbuild1 commented Aug 8, 2024

daosbuild1 commented Aug 8, 2024

daosbuild1 commented Aug 8, 2024

daosbuild1 commented Aug 8, 2024

daosbuild1 commented Aug 8, 2024

daosbuild1 commented Aug 8, 2024

daosbuild1 commented Aug 8, 2024

daosbuild1 commented Aug 9, 2024

daosbuild1 commented Aug 10, 2024

daosbuild1 commented Aug 22, 2024

knard-intel left a comment

Choose a reason for hiding this comment

knard-intel Aug 22, 2024

Choose a reason for hiding this comment

daosbuild1 commented Aug 22, 2024

sherintg Aug 23, 2024 • edited Loading

Choose a reason for hiding this comment

tanabarr Aug 27, 2024

Choose a reason for hiding this comment

kjacque left a comment • edited Loading

Choose a reason for hiding this comment

tanabarr commented Aug 27, 2024

daosbuild1 commented Aug 28, 2024

daosbuild1 commented Sep 2, 2024

daosbuild1 commented Sep 2, 2024

daltonbohning left a comment

Choose a reason for hiding this comment

daltonbohning Sep 4, 2024

Choose a reason for hiding this comment

tanabarr Sep 4, 2024

Choose a reason for hiding this comment

daltonbohning Sep 4, 2024

Choose a reason for hiding this comment

daosbuild1 commented Sep 5, 2024

daosbuild1 commented Sep 6, 2024

daosbuild1 commented Sep 8, 2024

tanabarr commented Sep 8, 2024

tanabarr commented Sep 8, 2024

tanabarr commented Jul 30, 2024 •

edited

Loading

github-actions bot commented Jul 30, 2024 •

edited

Loading

sherintg Aug 23, 2024 •

edited

Loading

kjacque left a comment •

edited

Loading