API 36.1.0 merge

[glemieux]

Integrating this PR will merge in #1136 ad #1236. To be associated with ESCOMP/CTSM#2700.

Description:

This is a pull request to coordinate the update of multiple parameter file updates at the same time. Whereas #1136 and #1236 were tested separately with their own special parameter file updates, this PR will combined the parameter file updates into the single default source. Part of the reason for setting things up this way is to provide more clarity in the testing, but also to capture the workflow necessary to update the default parameter file as it requires a sequential calls of UpdateParamAPI followed by BatchPatchParams tool on the #1136 and #1236 patch file updates respectively.

Collaborators:

@JessicaNeedham @jenniferholm @rgknox @XiulinGao

Expectation of Answer Changes:

Yes, answers are expected to change due to the parameter file updates. See the respective pull requests for more detail.

Checklist

If this is your first time contributing, please read the CONTRIBUTING document.

All checklist items must be checked to enable merging this pull request:

Contributor

• The in-code documentation has been updated with descriptive comments
• The documentation has been assessed to determine if updates are necessary

Integrator

• FATES PASS/FAIL regression tests were run
• Evaluation of test results for answer changes was performed and results provided

Documentation

• Technical Note update:
• User's Guide update:

Test Results:

CTSM (or) E3SM (specify which) test hash-tag:

CTSM (or) E3SM (specify which) baseline hash-tag:

FATES baseline hash-tag:

Test Output:

[glemieux]

• Update the default parameter file with the correct names for the new pfts

I think this requires a more up-to-date version of numpy than I'm currently using (numpy==1.24.4) as modify_fates_paramfile.py returns a warning and skips the update:

../tools/modify_fates_paramfile.py --fin /tmp/tmp.hdMg69N3Mi --fout /tmp/tmp.hdMg69N3Mi --var fates_pftname --silent  --val " broadleaf_colddecid_arctic_shrub" --nohist --overwrite --pft 11
output variable not interpretable as real. trying array
../tools/modify_fates_paramfile.py:84: DeprecationWarning: string or file could not be read to its end due to unmatched data; this will raise a ValueError in the future.
  outputval = np.fromstring(args.val, sep=',', dtype=np.float64)
Traceback (most recent call last):
  File "../tools/modify_fates_paramfile.py", line 355, in <module>
    main()
  File "../tools/modify_fates_paramfile.py", line 162, in main
    raise ValueError('variable dimension not handled in this script')
ValueError: variable dimension not handled in this script

[rgknox]

@glemieux , if your stuck, can you print out the value of the string:

print("args.val: --{}--".format(args.val))
outputval = np.fromstring(args.val, sep=',', dtype=np.float64)

[glemieux]

@glemieux , if your stuck, can you print out the value of the string:

print("args.val: --{}--".format(args.val))
outputval = np.fromstring(args.val, sep=',', dtype=np.float64)

Weirdly, the np.fromstring call on the new pft varname string doesn't result in a length of zero, which spot testing with a toy case suggests it should. It's actually returning a single element array with a value of -1, which is odd 🤔, and printing that deprecation warning. As such, it never triggers an exception to cascade down into the next try level here and set rename_pft to True:

fates/tools/modify_fates_paramfile.py

Lines 84 to 91 in e06e0df

	outputval = np.fromstring(args.val, sep=',', dtype=np.float32)
	if len(outputval) == 0:
	raise RuntimeError('output variable needs to have size greater than zero')
	except:
	if args.varname != 'fates_pftname':
	raise RuntimeError('output variable not interpretable as real or array')
	else:
	rename_pft = True

That said, I think I have a fix in which just involves checking to see if we're modifying the fates_pftname a bit early in that routine.

[glemieux]

@rgknox looks like the fix I brought in via 204a061 this works. Would you take a look and give this PR an approval if it looks good to you?

[glemieux]

Regression testing on derecho is complete. There are no unexpected failures. All testmods have DIFFs against the baseline (with the exception of the one ctsm-only testmod), which is expected given the extent of the parameter file changes and noted in #1136 (comment). Spot checking the satellite phenology SMS test shows results similar to those found in #1236 (comment).

Testing on izumi is still needed.

[glemieux]

Testing on izumi is underway.

[glemieux]

Izumi testing is complete. There are no unexpected failures.

Location: /scratch/cluster/glemieux/ctsm-tests/tests_1004-141329iz