Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Control postscripts/userscripts options from config.yaml #73

Open
atteggiani opened this issue Aug 26, 2024 · 0 comments
Open

Control postscripts/userscripts options from config.yaml #73

atteggiani opened this issue Aug 26, 2024 · 0 comments

Comments

@atteggiani
Copy link

Context

The current NetCDF conversion step uses the submission script UM_conversion_job.sh, which calls esm1p5_convert_nc $PAYU_CURRENT_OUTPUT_DIR --delete-ff.
The --delete-ff option is used to remove the original fields files at the end of the conversion process.

Most runtime options for ACCESS-ESM1.5 can be currently tweaked by changing the config.yaml, where the submission script above is listed, added to the postscripts:

postscript: -v PAYU_CURRENT_OUTPUT_DIR,PROJECT -lstorage=${PBS_NCI_STORAGE} ./scripts/NetCDF-conversion/UM_conversion_job.sh

Issue

The issue is that If someone does not want to remove the original fields files, they currently would need to manually omit (comment out) the --delete-ff option from the submission script.
The fact that, to disable a default option, a user needs to comment out a portion of a script instead of making changes in the config.yaml makes it clunky and makes goes against the purpose of having the config.yaml file in the first place.

Solution proposed

I propose the control flow of options for the postscripts (and possibly userscripts) to be embedded directly inside the config.yaml file.
This could be made possible, for example, by adding the desired options directly inside the postscript declaration:

postscript: -v PAYU_CURRENT_OUTPUT_DIR,PROJECT -lstorage=${PBS_NCI_STORAGE} ./scripts/NetCDF-conversion/UM_conversion_job.sh --delete-ff

I am not sure how payu handles the postscript/userscript calls. Therefore, I am placing this issue in this repo, but a modification on how these scripts are executed by payu might be needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants
@atteggiani and others