Update crutest+dsc to auto-generate volumes #1457
Comments
|
If dsc returns the number of "region sets" it is responsible for, this could enable a single dsc to serve a number of (identical) region sets. This could be combined with also supporting multiple dsc servers and crutest could be smart enough to build a VCR based on both of these. |
leftwo
pushed a commit
that referenced
this issue
Sep 25, 2024
These changes should be a fix for both issues: #1451 #1457 This adds support for crutest to use a provided dsc endpoint to construct a Volume object. Moved the existing volume creation steps to a new function, and added another option on how we can create a Volume. The two previous ways of creating a volume are not changed (though I changed a log message and added some warnings). The new code is in taking the dsc provided endpoint and using that to construct a volume. Additional dsc changes were made to help provide Volume info. Renamed things in dsc to better reflect what information they hold. Specifically, update a bunch of region set comments, as dsc just controls crucible-downstairs processes, and does not know which ones are part of what region set. New dsc commands: get_ds_uuid: Returns the UUID for the given client ID. all_running: Returns true if all downstairs that dsc knows about are currently in Running state. get_region_count: Returns the total number of regions that dsc knows about. New dsc behavior. dsc will now wait on all downstairs starting before taking any commands. The ability for dsc to answer a request can be used by a test to confirm that all downstairs had started. Add the ability to supply a dsc endpoint to crutest-cli #1459 tools/test_replay.sh transitioned to using the new --dsc option, as that test already required a dsc endpoint and was using a hard coded default value for it. tools/test_restart_repair.sh was updated to wait for dsc to report that all downstairs are online after a restart. This avoids a race where we told dsc to start, and then start crutest, but the downstairs are not yet online. All the tests that use dsc will eventually transition to using it to construct a Volume, but I'm pushing that work to another PR. There are more changes coming, specifically: Updating replace-before-acive and replace-reconcile to use the new dsc option correctly instead of a default. Updating tools/test_* to not require targets for crutest and instead use dsc option. More updates in crutest to update the current RegionInfo struct to be aware of multiple sub-volumes. #1454 Possibly some updates to the BlockIO trait. #1455 Other work this enables Making tests that use multiple sub-volumes. Layering the current set of tests in a way so we can run the same tests with a single sub-volume and with multiple sub-volumes
leftwo
added a commit
that referenced
this issue
Sep 26, 2024
This adds support for crutest to use a provided dsc endpoint to construct a Volume object. These changes should be a fix for both issues: #1451 and #1457 Moved the existing volume creation steps to a new function, and added another option on how we can create a Volume. The two previous ways of creating a volume are not changed (though I changed a log message and added some warnings). The new code is in taking the dsc provided endpoint and using that to construct a volume. Additional dsc changes were made to help provide Volume info. Renamed things in dsc to better reflect what information they hold. Specifically, update a bunch of region set comments, as dsc just controls crucible-downstairs processes, and does not know which ones are part of what region set. **New dsc commands** * get_ds_uuid: Returns the UUID for the given client ID * all_running: Returns true if all downstairs that dsc knows about are currently in Running state. * get_region_count: Returns the total number of regions that dsc knows about. **New dsc behavior** * dsc will now wait on all downstairs starting before taking any commands. The ability for dsc to answer a request can be used by a test to confirm that all downstairs had started. * Add the ability to supply a dsc endpoint to crutest-cli (fix #1459) **Other changes** `tools/test_replay.sh` transitioned to using the new --dsc option, as that test already required a dsc endpoint and was using a hard coded default value for it. `tools/test_restart_repair.sh` was updated to wait for dsc to report that all downstairs are online after a restart. This avoids a race where we told dsc to start, and then start crutest, but the downstairs are not yet online. All the tests that use dsc will eventually transition to using it to construct a Volume, but I'm pushing that work to another PR. --------- Co-authored-by: Alan Hanson <[email protected]>
|
With #1472, this work is done |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Crutest will soon support working at the volume layer, with support for reading a VCR from a file.
Once this work lands, we need to grow it to have crutest be able to ask dsc for either a VCR, or enough information so it can create a VCR based on what dsc tells it. It seems like having crutest build the VCR by asking dsc for the region information it needs could result in a framework that can create the scenarios we have now.
For example, If we have crutest itself generate the VCR, that could allow crutest to have multiple dsc "region sets" where each could have it's own region information that need not be the same as other region sets.
This flexibility could also allow us run a number of different tests of crutest in a loop where we change the dsc (and hence VCR) on each loop and crutest will just figure it out each time, possibly reducing the size of bash scripts we have now.
The text was updated successfully, but these errors were encountered: