Experiment form review

[matthew-mizielinski]

Experiment registration form

The new/modify experiment form has been added at

https://github.com/WCRP-CMIP/CMIP7_CVs/issues/new/choose

This issue is to capture feedback for any updates to this form

[taylor13]

Thanks for this solid first draft. Here are some suggestions:

For “Experiment ID”, we should instruct those registering experiments to consult the full guidance document. I would replace the current text with: “There are rules for constructing acceptable experiment names which can be found in this guidance document.

For “Description”, I suggest adding a sentence to the instruction:
“Provide all information not included elsewhere in this form that would be required to correctly configure the model and impose the experiment conditions. Include a description of the experiment, links to any appropriate references and indicate similarities to CMIP6 experiments. Please also include details of forcing differences relative to the parent experiment.”

I think we should add an entry at this point in the form for “Experiment Title”, which we could describe as “A short phrase that can help in interpreting the unique experiment_id’s. As examples, the CMIP6 titles are listed here in the third column labeled “experiments”.

For the registered “MIP / Activity ID” list, we should capitalize certain letters consistent with CMIP6: AerChemMIP, C4MIP, CFMIP, DAMIP, DCPP, GeoMIP, PMIP, RFMIP, ScenarioMIP. Also, shouldn’t “imip” be “ISMIP”?

I don’t think the “Parent Activity ID” should be requested. Doesn’t the parent experiment ID imply a unique parent activity ID? If we retain this entry, we should indicate that “no parent” should be entered if an experiment is not initialized from another CMIP7 experiment.

For “Allowed Model Component”, replace “must be included” with “may be included, but are not required”. Also we should note: If only the required model components should be included, enter “none”.

For “Parent Experiment ID”, I think it is confusing to include “forcings” in the instructions. The parent of the historical run for example is the piControl, but most of the forcings are different in the historical period. I would simply eliminate “forcings” in the description. And I would mention that “no parent” is an acceptable entry. The text could read: The ID of the experiment providing the initial conditions. If not initialized from another CMIP7 experiment, enter “no parent”.

In the case of “Sub-Experiment”, sometime ago I suggested that we eliminate this dataset descriptor in CMIP7. It was used only for DCPP runs in CMIP6. I propose that instead of defining a separate “sub-experiment”, we include it as part of the “ripf” variant identifier. We would replace the previously defined “initialization method” index (e.g., “i1”, “i2”, …) with the first full forecast year (e.g., “i1960”, “i1965”, ….). If alternative initialization methods were used, then a single letter suffix would be appended to the year (e.g., “i1960”, “i1960a”, “i1960b” …). Most modeling groups only use a single “method”, so it will be uncommon to see a letter suffix. Assuming this is acceptable (we need to check with DCPP coordinators), then we could eliminate “sub-experiment” on this form and ask that if there are multiple start dates for experiments, they should all be listed under “Start Date”, which could be renamed “Start Date(s)”. If we leave in “Sub-Experiment”, it should be renamed “Sub-Experiments”.

We should provide more guidance for the “Tier” entry. Something like: “How essential is the experiment to the primary scientific goals of the activity? 1=absolutely essential, 2=of major value, 3=of lesser value.”

I would rearrange the date/year input and provide further guidance as follows:

Start Date
For experiments initialized on a particular historical date, enter that date. If a set of sub-experiments are initialized at different times, list all the start times requested. For example, for an historical simulation initialized at the beginning of 1850, enter “1850-01-01”. For a set of DCPP “prediction” runs, enter all dates of the first full forecast/hindcast year (e.g., 1960, 1970, 1980, 1990, 2000, 2010). Enter “arbitrary” for a run not tied to any particular historical period. [PLEASE CHECK THAT I’VE GOT THE START OF THE CMIP7 HISTORICAL RUN RIGHT HERE!]

Branch Date
For experiments initialized from a non-arbitrary point in a parent experiment, enter the branch date in the parent. For example, “2022-01-01” should be entered for an experiment initialized from the end of a historical run and extending into the future. Enter “arbitrary” when the branch point is arbitrary. [PLEASE CHECK THAT I’VE GOT THE END OF THE CMIP7 HISTORICAL RUN RIGHT HERE!]

(Minimum) Number of Years
For most experiments this will be the actual length of the desired simulation. For example, “172” would be specified for a run initialized in 1850-01-01 and extending to 2022-01-01.

Note that there is no reason to request an “End Date” since that can be calculated from “Start Date” and “Number of Years”.

[matthew-mizielinski]

Feedback from Martin:
• The "Parent Activity ID (registered/unregistered)" questions needed? They should be provided via the entry for the parent activity.
• The terms listed as "model components" are referred to as "source type" in the CMIP6 paper cited. I.e. They refer to a category of model, not a component of a model.
• "Sub-experiment": it will be inefficient if they have to fill out the form for each sub-experiment. Could they provide the full list of all sub-experiment IDs here?
• Tier: does this have a clear definition? Is it really needed?
• "End Date" and "Minimum Number of Years": Does the end date correspond to the maximum number of years?

[matthew-mizielinski]

Thanks for this solid first draft. Here are some suggestions:

For “Experiment ID”, we should instruct those registering experiments to consult the full guidance document. I would replace the current text with: “There are rules for constructing acceptable experiment names which can be found in this guidance document.

Done, but that document should be confirmed and added to the IPO webpages.

For “Description”, I suggest adding a sentence to the instruction: “Provide all information not included elsewhere in this form that would be required to correctly configure the model and impose the experiment conditions. Include a description of the experiment, links to any appropriate references and indicate similarities to CMIP6 experiments. Please also include details of forcing differences relative to the parent experiment.”

Done

I think we should add an entry at this point in the form for “Experiment Title”, which we could describe as “A short phrase that can help in interpreting the unique experiment_id’s. As examples, the CMIP6 titles are listed here in the third column labeled “experiments”.

Added above the description

For the registered “MIP / Activity ID” list, we should capitalize certain letters consistent with CMIP6: AerChemMIP, C4MIP, CFMIP, DAMIP, DCPP, GeoMIP, PMIP, RFMIP, ScenarioMIP. Also, shouldn’t “imip” be “ISMIP”?

I don’t think the “Parent Activity ID” should be requested. Doesn’t the parent experiment ID imply a unique parent activity ID? If we retain this entry, we should indicate that “no parent” should be entered if an experiment is not initialized from another CMIP7 experiment.

Done

For “Allowed Model Component”, replace “must be included” with “may be included, but are not required”. Also we should note: If only the required model components should be included, enter “none”.

For “Parent Experiment ID”, I think it is confusing to include “forcings” in the instructions. The parent of the historical run for example is the piControl, but most of the forcings are different in the historical period. I would simply eliminate “forcings” in the description. And I would mention that “no parent” is an acceptable entry. The text could read: The ID of the experiment providing the initial conditions. If not initialized from another CMIP7 experiment, enter “no parent”.. If not included in

Done

In the case of “Sub-Experiment”, sometime ago I suggested that we eliminate this dataset descriptor in CMIP7. It was used only for DCPP runs in CMIP6. I propose that instead of defining a separate “sub-experiment”, we include it as part of the “ripf” variant identifier. We would replace the previously defined “initialization method” index (e.g., “i1”, “i2”, …) with the first full forecast year (e.g., “i1960”, “i1965”, ….). If alternative initialization methods were used, then a single letter suffix would be appended to the year (e.g., “i1960”, “i1960a”, “i1960b” …). Most modeling groups only use a single “method”, so it will be uncommon to see a letter suffix. Assuming this is acceptable (we need to check with DCPP coordinators), then we could eliminate “sub-experiment” on this form and ask that if there are multiple start dates for experiments, they should all be listed under “Start Date”, which could be renamed “Start Date(s)”. If we leave in “Sub-Experiment”, it should be renamed “Sub-Experiments”.

I'd like to see something more concrete for a proposal for this before making this change. In particular I'd like to confirm that CMOR can handle the proposed changes.

We should provide more guidance for the “Tier” entry. Something like: “How essential is the experiment to the primary scientific goals of the activity? 1=absolutely essential, 2=of major value, 3=of lesser value.”

Done

I would rearrange the date/year input and provide further guidance as follows:

Start Date
For experiments initialized on a particular historical date, enter that date. If a set of sub-experiments are initialized at different times, list all the start times requested. For example, for an historical simulation initialized at the beginning of 1850, enter “1850-01-01”. For a set of DCPP “prediction” runs, enter all dates of the first full forecast/hindcast year (e.g., 1960, 1970, 1980, 1990, 2000, 2010). Enter “arbitrary” for a run not tied to any particular historical period. [PLEASE CHECK THAT I’VE GOT THE START OF THE CMIP7 HISTORICAL RUN RIGHT HERE!]

Done

Branch Date
For experiments initialized from a non-arbitrary point in a parent experiment, enter the branch date in the parent. For example, “2022-01-01” should be entered for an experiment initialized from the end of a historical run and extending into the future. Enter “arbitrary” when the branch point is arbitrary. [PLEASE CHECK THAT I’VE GOT THE END OF THE CMIP7 HISTORICAL RUN RIGHT HERE!]

Done

(Minimum) Number of Years
For most experiments this will be the actual length of the desired simulation. For example, “172” would be specified for a run initialized in 1850-01-01 and extending to 2022-01-01.

Note that there is no reason to request an “End Date” since that can be calculated from “Start Date” and “Number of Years”.

[matthew-mizielinski]

Feedback from Martin:
• The "Parent Activity ID (registered/unregistered)" questions needed? They should be provided via the entry for the parent activity.
Removed as noted above

• The terms listed as "model components" are referred to as "source type" in the CMIP6 paper cited. I.e. They refer to a category of model, not a component of a model.

This needs a little more thought as to how to address. Will come back to this one.

• "Sub-experiment": it will be inefficient if they have to fill out the form for each sub-experiment. Could they provide the full list of all sub-experiment IDs here?

I've added a small clarification.

• Tier: does this have a clear definition? Is it really needed?

I've kept it and expanded the description as noted above.

• "End Date" and "Minimum Number of Years": Does the end date correspond to the maximum number of years?

As suggested above I've removed the end date field in favour of the minimum number of years

[matthew-mizielinski]

• The terms listed as "model components" are referred to as "source type" in the CMIP6 paper cited. I.e. They refer to a category of model, not a component of a model.

I've tried to adjust the wording to make this more consistent. Am happy to take further suggestions for changes

[taylor13]

Looking good. Perhaps a few additional tweaks are needed:

At the top of the form it says "Add a title", and then "Add/Modify: Experiment: ". If the "item" is the name of the experiment, then this should read "new experiment_id", or something along those lines. Otherwise, what does "item" refer to?

It's a bit confusing that for "activity id" and "parent experiment id", only 1 of the 2 options in each case is "required". Shouldn't both have an asterisk or neither? Perhaps all items should be required, but an acceptable answer for some is "NA" (not applicable), or something similar.

Under "Source type codes for additional allowed model components", we say "Components that may be included in models when running this experiment." This sentence could be misinterpreted as asking for all components that can be included (including required components). Could we say instead "Components that may be included in models in addition to the required components listed above."

I think those filling out the form would really appreciate it if we put the most common "parent experiments" first in the list. I would put: no parent, piControl, historical, amip, esm-piControl, esm-hist, followed by all the others.

Also please use proper capitialization in all experiment_ids. (See CMIP6 listing here: https://wcrp-cmip.github.io/CMIP6_CVs/docs/CMIP6_experiment_id.html .

Since "Start Date" can be left blank, its confusing to include the asterisk, indicating that it is required input. Perhaps the directions should say: If there is no parent experiment please enter "NA", or some other similar text.