Added the phen_model_day timer to work with all restart conditions

[rgknox]

Description:

The phen_model_day timer was added to the site structure. This timer is initialized as the day of year on a cold-start, and is read in from the restart file. This replaces the hlm_model_day, which for some types of restarts, will not be continuous from the last simulation. (ie it may be reset) By using phen_model_day, we have consistency across restarts, and this absolute integer day will always be preserved.

Note: Users who restart the model via an finidat, or non CONTINUE_RUN simulation, AND, have a gap in the DAY OF YEAR from the end of one simulation to the start of the next, will likely still encounter phenology weirdness. However, it seems reasonable that they should expect weirdness if they decide to magically remove a portion of the year from the plants life-cycle.

Fixes #627
Fixes #894

Collaborators:

@rosiealice @adrifoster

Expectation of Answer Changes:

Yes, this will change answers, but only on non CONTINUE_RUN restart scenarios, and only on deciduous vegetation.

Checklist:

• I have updated the in-code documentation .AND. (the technical note .OR. the wiki) accordingly.
• I have read the CONTRIBUTING document.
• FATES PASS/FAIL regression tests were run
• If answers were expected to change, evaluation was performed and provided

Test Results:

TBD

CTSM (or) E3SM (specify which) test hash-tag:

CTSM (or) E3SM (specify which) baseline hash-tag:

FATES baseline hash-tag:

Test Output:

[rgknox]

See test results from 2 simulations at bci, 10 years each, 4 pfts (our drought deciduous pfts), the base is master, the test is this branch.
PMD-test_plots.pdf

Running a hybrid test now, where I start a second simulation in the 5th out of 10th year. In a hybrid, the reference date should change, so this test should indicate if the our timers are preserved through the restart.

[rgknox]

hybrid test looks good. I ran two simulations, 1) a base simulation that was a 10 year cold-start, and a 5 year hybrid run that used the base simulations 5th year as an finidat (initial condition). Model output from the hybrid run and the base run are indistinguishable, included "days since leaf on and leaf off" diagnostics.

hybr-test2_plots.pdf

[rosiealice]

Sorry for taking so long to have the headspace to get to this! It all looks fine to me, and I think will prove useful for ongoing phenology updates...

[rgknox]

tested on cheyenne, "fates" test suite:
/glade/scratch/rgknox/tests_0825-130322ch
I believe tests are nominal (still checking), all tests have expected diffs with BASELINE, but the diffs are only in the variables that had their calculations changed:

 FATES_DAYSINCE_COLDLEAFOFF
 FATES_DAYSINCE_COLDLEAFON
 FATES_DAYSINCE_DROUGHTLEAFOFF
 FATES_DAYSINCE_DROUGHTLEAFON

Expected FAILS:
ERS_Lm12.1x1_brazil.I2000Clm50FatesCruRsGs.cheyenne_intel.clm-FatesFireLightningPopDens COMPARE_base_rest
ERS_D_Ld15.f45_f45_mg37.I2000Clm50FatesRs.cheyenne_intel.clm-FatesColdDefTreeDamage