DAOS-xxx client: Dump metrics to container #15525
Conversation
|
Errors are Ticket number suffix is not a number. See https://daosio.atlassian.net/wiki/spaces/DC/pages/11133911069/Commit+Comments,Unable to load ticket data |
|
Test stage NLT on EL 8.8 completed with status UNSTABLE. https://build.hpdd.intel.com/job/daos-stack/job/daos//view/change-requests/job/PR-15525/1/testReport/ |
mjmac
force-pushed
the
mjmac/dfs_stats
branch
2 times, most recently
from
22e2ae2 to
78f09d2
Compare
|
Test stage Unit Test on EL 8.8 completed with status UNSTABLE. https://build.hpdd.intel.com/job/daos-stack/job/daos//view/change-requests/job/PR-15525/3/testReport/ |
|
Test stage NLT on EL 8.8 completed with status UNSTABLE. https://build.hpdd.intel.com/job/daos-stack/job/daos//view/change-requests/job/PR-15525/3/testReport/ |
mjmac
force-pushed
the
mjmac/dfs_stats
branch
from
78f09d2 to
cd972b7
Compare
mjmac
changed the title
mjmac
changed the title
|
Test stage Unit Test on EL 8.8 completed with status UNSTABLE. https://build.hpdd.intel.com/job/daos-stack/job/daos//view/change-requests/job/PR-15525/4/testReport/ |
|
Test stage NLT on EL 8.8 completed with status UNSTABLE. https://build.hpdd.intel.com/job/daos-stack/job/daos//view/change-requests/job/PR-15525/4/testReport/ |
mjmac
force-pushed
the
mjmac/dfs_stats
branch
from
cd972b7 to
3f819e1
Compare
|
Test stage NLT on EL 8.8 completed with status UNSTABLE. https://build.hpdd.intel.com/job/daos-stack/job/daos//view/change-requests/job/PR-15525/5/testReport/ |
Add a boolean container property that can be used to toggle client-side metrics collection and reporting behavior on a per-container basis. By default, containers will not participate in metrics collection, and must be opted-in via the property. The initial use case for this property is to enable DFS-level metrics in POSIX containers. Features: container Required-githooks: true Signed-off-by: Michael MacDonald <[email protected]>
If metrics are enabled for a POSIX container, create a new container/$UUID/dfs metrics root in the client telemetry to provide DFS-oriented metrics (POSIX ops, file I/Os, etc). Features: container Required-githooks: true Signed-off-by: Michael MacDonald <[email protected]>
Required-githooks: true Signed-off-by: Michael MacDonald <[email protected]>
mjmac
force-pushed
the
mjmac/dfs_stats
branch
from
3f819e1 to
f4549d9
Compare
|
Test stage NLT on EL 8.8 completed with status UNSTABLE. https://build.hpdd.intel.com/job/daos-stack/job/daos//view/change-requests/job/PR-15525/6/testReport/ |
| /* | ||
| * dc_task_schedule() calls tse_task_complete() even on error (which also calls the | ||
| * completion cb that frees params in this case, so we can just ignore the rc here. | ||
| */ | ||
| dc_task_schedule(task, true); | ||
|
|
||
| dfs_update_file_metrics(dfs, *params->read_size, 0); |
There was a problem hiding this comment.
i think those metrics update calls should be in the completion callback when the read completes
| @@ -412,6 +412,12 @@ open_stat(dfs_t *dfs, dfs_obj_t *parent, const char *name, mode_t mode, int flag | |||
|
|
|||
| out: | |||
| if (rc == 0) { | |||
| if (flags & O_CREAT) { | |||
| DFS_OP_STAT_INCR(dfs, DOS_CREATE); | |||
There was a problem hiding this comment.
i think it would be better to not have a CREATE metric, but re-use the mkdir (because that is what open with O_CREAT for dir is) and File_Create or just CREATE for creating a file.
| if (flags & O_CREAT) { | ||
| DFS_OP_STAT_INCR(dfs, DOS_CREATE); | ||
| } else { | ||
| DFS_OP_STAT_INCR(dfs, DOS_OPEN); |
There was a problem hiding this comment.
i would also split this into opendir and open
| @@ -1662,6 +1679,7 @@ dfs_punch(dfs_t *dfs, dfs_obj_t *obj, daos_off_t offset, daos_size_t len) | |||
| return daos_der2errno(rc); | |||
| } | |||
|
|
|||
| DFS_OP_STAT_INCR(dfs, DOS_PUNCH); | |||
There was a problem hiding this comment.
punch is not a posix term, so maybe better one would be truncate?
| @@ -1708,6 +1726,7 @@ dfs_sync(dfs_t *dfs) | |||
| if (dfs->amode != O_RDWR) | |||
| return EPERM; | |||
|
|
|||
| DFS_OP_STAT_INCR(dfs, DOS_SYNC); | |||
There was a problem hiding this comment.
since this is a no-op, I would not increment the metric for it as it can cause confusion.
| ### Per-Container Metrics Collection | ||
|
|
||
| If enabled, per-container metrics (currently only POSIX containers are supported) | ||
| will be collected on the client. Set the `DAOS_PROP_CO_METRICS_ENABLED` container |
There was a problem hiding this comment.
i don't think this is a good idea. we can discuss offline
|
@mchaarawi: This is the combined patch for playing around with the feature. I'll incorporate your feedback into the standalone PR for DFS metrics: #15544 |
If configured, dump metrics to a container instead of
a directory.