Some desired hist file comparisons are not done for _N2 tests; also, confusing results in .base.cprnc.out

[billsacks]

This issue does NOT appear to be new to the ESMCI version of cime, but I just noticed it.

It looks like some desired hist file comparisons are not done for the .base vs. .rest file comparison for ERS_N2 tests (and I suspect the same is true for other two-run tests).

With cime5.0.3 (in the CESM alpha01 sandbox), I ran ERS_D_N2.f10_f10.ICLM45.yellowstone_gnu.clm-default; I also ran the same test out of clm4_5_8_r182 (with cime4).

In both, it appears that the only history file comparisons that were done (based on the *.rest.cprnc.out files that were produced) were the clm2_0002 hist files. In particular, I see no evidence that comparisons were done for clm2_0001 hist files (a relatively minor issue, but I believe that should be fixed), or for cpl hist files (a more significant issue, I believe).

I was also confused by the presence of *.base.cprnc.out files in these tests, despite the fact that I didn't ask for baseline comparisons. It looks like these compared the clm2_0001 .base history files with the clm2_0002 .base history files. Perhaps this is an intentional feature of _N2 tests. If so, that's fine, but it's confusing for these results to appear in the .base.cprnc.out file, which typically are used for comparisons with baselines. Again, this behavior seems to be the same in cime5 and cime4.

[jgfouca]

Assigning @jedwards4b because I have too many issues already. Feel free to reassign.

[billsacks]

The relevant code has been rewritten as of PR #413 . But a similar (though maybe slightly less severe) issue remains with _N2 tests: For components that generate multi-instance history files (e.g., clm2_0001...nc and clm2_0002...nc) only the _0001 files are used in the comparisons. Similarly, only the _0001 files are considered for baseline comparison and generation. I believe the problem is that hist_utils.py does not distinguish based on the multi-instance suffix.

It appears that the earlier issue of cpl hist files not being compared has been fixed.

I tested this with ERS_P24x1_N2_Ld3.f10_f10.ICLM45.hobart_nag.clm-default

Also cc'ing @mvertens on this.

[mvertens]

I think this is a step forwards - but we need to have all instances
compared before cime5.1.

On Wed, Aug 24, 2016 at 3:33 PM, Bill Sacks [email protected]
wrote:

The relevant code has been rewritten as of PR #413
#413 . But a similar (though maybe
slightly less severe) issue remains with _N2 tests: For components that
generate multi-instance history files (e.g., clm2_0001...nc and
clm2_0002...nc) only the _0001 files are used in the comparisons.
Similarly, only the _0001 files are considered for baseline comparison and
generation. I believe the problem is that hist_utils.py does not
distinguish based on the multi-instance suffix.

It appears that the earlier issue of cpl hist files not being compared has
been fixed.

I tested this with ERS_P24x1_N2_Ld3.f10_f10.ICLM45.hobart_nag.clm-default

Also cc'ing @mvertens https://github.com/mvertens on this.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#309 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AHlxEyVQJTQxxIjjhPVa1GXXrIFNxbkgks5qjLi-gaJpZM4JZJZ-
.