crash when running GFS PBL/sfclay and clm lake in RRFS ensemble

[JiliDong-NOAA]

Description

A couple of RRFS ensemble members crashed with GFS PBL (TKE-EDMF)/sfclay and clm lake combined. This happened when lake water was frozen to lake ice in clm lake.

The reason for the crash is because clm lake is called between surface layer scheme and PBL scheme. If there is a state change in clm lake, particularly water turning into ice, stability related variables over ice will be missing and cause the crash.
Please see the attached slides for more information.

clm_lake_bug_fix.pptx

To Reproduce:

1. perform a forecast with GFS PBL/sfclayer combined with clm lake
2. wait for lake water turning into lake ice, which could happen in winter time further north
3. model crashed with AVOST error message from RUC LSM

Additional context

Add any other context about the problem here.
Directly reference any issues or PRs in this or other repositories that this is related to, and describe how they are related. Example:

• needs to be fixed also in noaa-emc/nems/issues/<issue_number>
• needed for noaa-emc/fv3atm/pull/<pr_number>

Output

Screenshots
If applicable, drag and drop screenshots to help explain your problem.

output logs
If applicable, include relevant output logs.
Either drag and drop the entire log file here (if a long log) or

      AVOST IN VILKA     Table index=  -2147483648

 I,J=           1           1 LU_index =           17 Psfc[hPa] =

                     NaN Tsfc =    262.537014420372

Testing:

1. Have you tested the code changes? On what platforms?
   Yes. On Hera.
2. Have you run regression test in ufs-weather-model or ufs-s2s-model with code changes?
   Yes.

• Will the baseline results change?
  No.
• If the baseline results change, is it expected? Please give brief explanation.

Dependent PRs:

ufs-community/ccpp-physics#148