-
Notifications
You must be signed in to change notification settings - Fork 554
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
config-linux: add schemata field to IntelRdt #1230
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
@@ -645,6 +645,7 @@ The following parameters can be specified for the container: | |||||||||||||||||||||||||||||||||||||||
The value SHOULD start with `L3:` and SHOULD NOT contain newlines. | ||||||||||||||||||||||||||||||||||||||||
* **`memBwSchema`** *(string, OPTIONAL)* - specifies the schema of memory bandwidth per L3 cache id. | ||||||||||||||||||||||||||||||||||||||||
The value MUST start with `MB:` and MUST NOT contain newlines. | ||||||||||||||||||||||||||||||||||||||||
* **`schemata`** *(string, OPTIONAL)* - specifies the schemata to be written to the `schemata` file in resctrlfs. | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
The following rules on parameters MUST be applied: | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
|
@@ -654,15 +655,17 @@ The following rules on parameters MUST be applied: | |||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
* If either `l3CacheSchema` or `memBwSchema` is set, runtimes MUST write the value to the `schemata` file in the that sub-directory discussed in `closID`. | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
* If neither `l3CacheSchema` nor `memBwSchema` is set, runtimes MUST NOT write to `schemata` files in any `resctrl` pseudo-filesystems. | ||||||||||||||||||||||||||||||||||||||||
* If `schemata` field is set, runtimes MUST write the value to the `schemata` file in the that sub-directory discussed in `closID`. If also l3CacheSchema` or `memBwSchema` is set the value of `schemata` field must be written last, after the values from `l3CacheSchema` and `memBwSchema` has been written. | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
* If none of `l3CacheSchema`, `memBwSchema` or `schemata` is set, runtimes MUST NOT write to `schemata` files in any `resctrl` pseudo-filesystems. | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
* If `closID` is not set, runtimes MUST use the container ID from [`start`](runtime.md#start) and create the `<container-id>` directory. | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
* If `closID` is set, `l3CacheSchema` and/or `memBwSchema` is set | ||||||||||||||||||||||||||||||||||||||||
* If `closID` is set, `l3CacheSchema` and/or `memBwSchema` and/or `schemata` is set | ||||||||||||||||||||||||||||||||||||||||
* if `closID` directory in a mounted `resctrl` pseudo-filesystem doesn't exist, the runtimes MUST create it. | ||||||||||||||||||||||||||||||||||||||||
* if `closID` directory in a mounted `resctrl` pseudo-filesystem exists, runtimes MUST compare `l3CacheSchema` and/or `memBwSchema` value with `schemata` file, and [generate an error](runtime.md#errors) if doesn't match. | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
* If `closID` is set, and neither of `l3CacheSchema` and `memBwSchema` are set, runtime MUST check if corresponding pre-configured directory `closID` is present in mounted `resctrl`. If such pre-configured directory `closID` exists, runtime MUST assign container to this `closID` and [generate an error](runtime.md#errors) if directory does not exist. | ||||||||||||||||||||||||||||||||||||||||
* If `closID` is set, and none of `l3CacheSchema`, `memBwSchema` or `schemata` are set, runtime MUST check if corresponding pre-configured directory `closID` is present in mounted `resctrl`. If such pre-configured directory `closID` exists, runtime MUST assign container to this `closID` and [generate an error](runtime.md#errors) if directory does not exist. | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
* **`enableCMT`** *(boolean, OPTIONAL)* - specifies if Intel RDT CMT should be enabled: | ||||||||||||||||||||||||||||||||||||||||
* CMT (Cache Monitoring Technology) supports monitoring of the last-level cache (LLC) occupancy | ||||||||||||||||||||||||||||||||||||||||
|
@@ -674,18 +677,23 @@ The following rules on parameters MUST be applied: | |||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
### Example | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
Consider a two-socket machine with two L3 caches where the default CBM is 0x7ff and the max CBM length is 11 bits, | ||||||||||||||||||||||||||||||||||||||||
and minimum memory bandwidth of 10% with a memory bandwidth granularity of 10%. | ||||||||||||||||||||||||||||||||||||||||
Consider a two-socket machine with: | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
- two L3 caches where the default CBM is 0x7ff (11 bits) | ||||||||||||||||||||||||||||||||||||||||
- eight L2 caches where the default CBM is 0xFF (8 bits) | ||||||||||||||||||||||||||||||||||||||||
- minimum memory bandwidth of 10% with a memory bandwidth granularity of 10% | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
Tasks inside the container: | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
Tasks inside the container only have access to the "upper" 7/11 of L3 cache on socket 0 and the "lower" 5/11 L3 cache on socket 1, | ||||||||||||||||||||||||||||||||||||||||
and may use a maximum memory bandwidth of 20% on socket 0 and 70% on socket 1. | ||||||||||||||||||||||||||||||||||||||||
- have access to the "upper" 7/11 of L3 cache on socket 0 and the "lower" 5/11 L3 cache on socket 1 | ||||||||||||||||||||||||||||||||||||||||
- have access to the "lower" 4/8 of L2 cache on socket 0 (socket 1 is left out from this example) | ||||||||||||||||||||||||||||||||||||||||
- may use a maximum memory bandwidth of 20% on socket 0 and 70% on socket 1. | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
```json | ||||||||||||||||||||||||||||||||||||||||
"linux": { | ||||||||||||||||||||||||||||||||||||||||
"intelRdt": { | ||||||||||||||||||||||||||||||||||||||||
"closID": "guaranteed_group", | ||||||||||||||||||||||||||||||||||||||||
"l3CacheSchema": "L3:0=7f0;1=1f", | ||||||||||||||||||||||||||||||||||||||||
"memBwSchema": "MB:0=20;1=70" | ||||||||||||||||||||||||||||||||||||||||
"schemata": "L3:0=7f0;1=1f\nL2:0=f;1=f;2=f;3=f\nMB:0=20;1=70" | ||||||||||||||||||||||||||||||||||||||||
Comment on lines
695
to
+696
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Just for my understanding, none of the other fields ( runtime-spec/specs-go/config.go Lines 796 to 814 in c0e9043
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Correct 👍 |
||||||||||||||||||||||||||||||||||||||||
} | ||||||||||||||||||||||||||||||||||||||||
} | ||||||||||||||||||||||||||||||||||||||||
``` | ||||||||||||||||||||||||||||||||||||||||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll write a longer comment, but I'm considering that we should document that if
schemata
is set, runtimesMUST
ignore both thel3CacheSchema
andmemBwSchema
fields, and instead to use theschemata
field instead (and write its content "as-is" (that should probably also be documented on thel3CacheSchema
andmemBwSchema
fields)There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This field also needs to be added to the JSON-schema;
runtime-spec/schema/config-linux.json
Lines 259 to 278 in c0e9043
As well as the go implementation;
runtime-spec/specs-go/config.go
Line 795 in c0e9043
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Woops 🙄 Added