Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

XML Caching needs to be more robust #2850

Closed
jgfouca opened this issue Oct 12, 2018 · 4 comments
Closed

XML Caching needs to be more robust #2850

jgfouca opened this issue Oct 12, 2018 · 4 comments
Assignees

Comments

@jgfouca
Copy link
Contributor

jgfouca commented Oct 12, 2018

User should never have to disable caching. Propose adding sha1 to FILEMAP to check that file was not changed. Maybe need addition class-level data in GenericXML to ensure that we never have multiple writable GenericXML objects open to the same file at the same time.

@jgfouca jgfouca self-assigned this Oct 12, 2018
@gold2718
Copy link

Doesn't computing the sha1 take time and slow performance?

@jgfouca
Copy link
Contributor Author

jgfouca commented Oct 13, 2018

@gold2718 , it should be much faster than a full XML tree parse.

@billsacks
Copy link
Member

I'm curious: Are there current issues with the caching that prompted you to open this issue, or is this just a future precaution?

@billsacks
Copy link
Member

I'm curious: Are there current issues with the caching that prompted you to open this issue, or is this just a future precaution?

Okay, just skimmed Slack and see the question is answered there.

jgfouca added a commit that referenced this issue Dec 7, 2018
Big improvement to robustness of CIME's XML handling

Change list:

Propagate Case read_only to its XML objects
Cache all files, but use file mod-time to detect changes.
Try to pass Case, not caseroot, to run_sub_or_cmd
Add a CIME performance test
Add tests for CIME XML handling
Test suite: scripts_regression_tests
Test baseline:
Test namelist changes:
Test status: bit for bit

Fixes #2161
Fixes #2850

User interface changes?: GenericXML will now throw an error if XML files are changed behind its back without a re-read/invalidate. It is now expected that all python buildlib calls take a Case object, not caseroot.

Instructions for updating python buildlib scripts:

The main function should open a Case object (preferably, read-only) and pass that object to the buildlib function, not the caseroot.
The buildlib function should take a case object, not a caseroot, and should therefore not have to open a Case object.
Update gh-pages html (Y/N)?:

Code review: @jedwards4b @billsacks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants