Fix vertical refinement, broken from v4.0 through v4.1

[davegill]

TYPE: bug fix

KEYWORDS: hybrid vertical coordinate, vertical refinement

SOURCE: Katie Lundquist (LLNL) and internal

DESCRIPTION OF CHANGES:
Problem:
Since the formal introduction of the hybrid vertical coordinate option in
WRF (with version v4.0), the vertical refinement option (also referred to
as vertical nesting) has been broken for both of the options when vert_refine_method
= 1 or 2 (ndown or WRF, respectively). The symptomology is that model
dies in the first time step, whether in the dynamics, physics, or initialization.

Solution:
Katie discovered that the 1d arrays (c1, c2, c3, c4 which are used to
construct the pressure level of the vertical levels) were never set for
vertical refinement.

Editor's note: These values were set for traditional nests. Since the number of
vertical levels in the parent and child domains were the same, the already
existing code was sufficient:

 nest%c1h(1:parent%e_vert) = parent%c1h(1:parent%e_vert)
 nest%c2h(1:parent%e_vert) = parent%c2h(1:parent%e_vert)
 nest%c3h(1:parent%e_vert) = parent%c3h(1:parent%e_vert)
 nest%c4h(1:parent%e_vert) = parent%c4h(1:parent%e_vert)
 nest%c1f(1:parent%e_vert) = parent%c1f(1:parent%e_vert)
 nest%c2f(1:parent%e_vert) = parent%c2f(1:parent%e_vert)
 nest%c3f(1:parent%e_vert) = parent%c3f(1:parent%e_vert)
 nest%c4f(1:parent%e_vert) = parent%c4f(1:parent%e_vert)

However, when the two domains have a different number of vertical levels these assignments
are incorrect. When there is no input for the second domain, there is no opportunity inside of
the WRF model to correct these wrong assignments by overwriting the nest values of the 1d
arrays with a subsequent read.

The source code that is required to compute the vertical 1d arrays already exists.
For the real program, the computation of the 1d arrays is handled in the
dyn_em/module_initialize_real.F file. The entirety of those computations are now removed from
the module_initialize_real.F subroutine and placed in dyn_em/nest_init_utils.F, as a new
subroutine. This refactoring does two things:

1. The computations necessary for the vertical refinement are available and consistent with the
   existing computations done elsewhere in the real program.
2. A single shared routine is in place instead of duplicated software.

Due to the existing makefile dependencies, locating the new subroutine inside of the
nest_init_utils.F file required no mods to the makefiles or depend.common for the
case of the (online) vertical refinement. For the ndown (offline vertical refinement), the
depend.common file is updated to include the files nest_init_utils.o (where the new
1d array computing routine is located) and module_model_constants.o (where the
constants are located for R, CP, etc.

Lastly, a previously required test in check_a_mundo.F is now able to be removed. That test
did not allow users to use vertical nesting with the hybrid coordinate. As it turns outs, that test
was insufficient. Any use of the vertical refinement with input_from_file=.true.,.false. would
fail.

LIST OF MODIFIED FILES:
modified: dyn_em/module_initialize_real.F
modified: dyn_em/nest_init_utils.F
modified: main/depend.common
modified: main/ndown_em.F
modified: share/module_check_a_mundo.F

ISSUE:
Fixes #917 "NDOWN v4.1: vertical refinement is broken"

TESTS CONDUCTED:

• With the original code, the WRF model dies in the first time step.
• With the mods, the model runs a successful 12-h simulation

Here are the pieces to make the code fail for online vertical refinement (within WRF):

1. A nested domain, but only a single input file. The 1d arrays must be internally computed.

 &time_control
 input_from_file     = .true.,.false.,
 /

2. The vertical refinement option must be active, with different values of e_vert on each domain.
   Whether the nested value of vert_refine_method is 1 or 2 does not matter - fail city.

 &domains
 max_dom             = 2,
 e_vert              = 35,    45,
 feedback            = 0,
 vert_refine_method  = 0,     2,
 /

3. Nothing special, other than radiation LW and SW must be 1 - just a friendly reminder.

 &physics
 ra_lw_physics       = 1,     1,
 ra_sw_physics       = 1,     1,
 /

4. Importantly, the setting of the hybrid option does not matter. The original code will fail with either:

 &dynamics
 hybrid_opt          = 0
 /

or

 &dynamics
 hybrid_opt          = 2
 /

5. With this mod, using hybrid_opt=0 and use_theta_m=0, I was able to get bit-wise identical results
   with fix for vertical nesting without the hybrid coordinate - fix for use … #891 "fix for vertical nesting without the hybrid coordinate - fix for use". I used a test case
   provided by Katie Lundquist:

curl -O ftp://ftp.llnl.gov/outgoing/lundquist3/vertnesting_example.tgz

Per Katie:

I ran two cases in v3.9.1.1 (unchanged) and 4.1 (my vert. nesting fix, not your nice refactoring) and they look the same. MU certainly has noticeable effects at the edges from vertical nesting. Effects are much more muted on the other prognostic variables, such as velocity. The causes of this are documented in the Daniels et al (2016) paper in MWR. Some of it is unavoidable and I think some could be fixed by modifying the smoothing behavior between nested domains. This commit should return the vertical nesting functionality to that before v4.0 when the hybrid coordinate was set as the default. I have only looked at the solutions for hybrid_opt = 0, not 2.

6. I ran Katie's case with the hybrid vertical coordinate with metgrid data that was provided:

curl -O ftp://ftp.llnl.gov/outgoing/lundquist3/vertnesting_example_input.tgz

This namelist snippet is used:

&time_control
 run_seconds            = 60,
 input_from_file        = .true.,  .false.,  .false.,  .false.,
 history_interval_s     = 20,    20,    10,      10,       
/

&domains
 time_step              = 20,
 e_we                   = 196,     100,     100,     100,     
 e_sn                   = 196,     100,     100,     100,     
 e_vert                 = 88,      89,     90,      91,      
 vert_refine_method     = 0,       2,       2,       2,       
 dx                     = 27000,   3000,    333,    37,    
 dy                     = 27000,   3000,    333,    37,    
 grid_id                = 1,       2,       3,       4,       
 parent_id              = 1,       1,       2,       3,       
 parent_grid_ratio      = 1,       9,       9,       9,       
 parent_time_step_ratio = 1,       5,       4,       4,       
 feedback               = 0,
 rebalance              = 1,
 /

&dynamics
 use_theta_m            = 0,
 hybrid_opt             = 2,
 /

The figure shows domain 4, after 1 hour (240 d04 time steps), MU field
LEFT - differences between terrain following and hybrid vertical coordinate
CENTER - MU field terrain following coordinate (considered an exemplar)
RIGHT - MU field hybrid coordinate

Here are the pieces to make the code fail (for the original code) for offline vertical refinement
(using ndown):

1. A nested domain, requesting the same number of vertical levels, and turning on the hybrid option.

 &domains
 vert_refine_fact = 2
 e_vert           = 45,    45,    33,
 feedback         = 0,               
 /

 &dynamics
 hybrid_opt       = 2,               
 /

For ndown, cross section plots of geopotential perturbation of the generated wrfbdy_d02 file (the
ranges are not the same, to show detail of the figures).
Left: new mods
Center: original code
Right: difference

Recap: This PR provides results similar to v3.9.1.1 for hybrid_opt=0.

This error condition and fix are consistent with the required modification to fill in the 1d vertical
coordinate arrays. Those arrays became mandatory with v4.0 (when the vertical refinement option
was first broken). Those 1d arrays are mandatory for both the hybrid and terrain-following vertical
coordinates.

RELEASE NOTE: From version 4.0 through 4.1, both vertical refinement options were broken. For online vertical refinement (within the WRF model) with only a single input domain, the incorrect WRF code had uninitialized arrays that defined the vertical coordinate in the child domain. However, any users with vertical refinement activated AND with input_from_file = t,t were never impacted. For offline vertical refinement (using ndown with vert_refine_fact), the initial conditions also only had the lower portion of the 1d arrays initialized (and those were initialized incorrectly). Modifications were introduced to fix both refinement options, and the modifications support the hybrid and the terrain-following vertical coordinate.

[davegill]

@kalundquist
Katie,
I need to test this out. But I think this is supposed to be what we want.

[kalundquist]

@davegill
Dave,
This looks like it will work to me. Thanks for taking the time to fix this.
Katie

[davegill]

Hmmm, not ready yet. Better, but still not well.
Look at d02 after a few times steps. Low level u.

Low level v is similar (just on the n/s sides). The mu field has a picture frame, too.

[kalundquist]

@davegill
Is that with the hybrid coordinate (option 2)? Do things look fine with option 0?

[kalundquist]

@davegill
And do you have the rebalancing option on? You have to have that on. I don't see it mentioned in the namelist options above.

[davegill]

@kalundquist
Katie,
The results for hybrid=0 and hybrid=2 are essentially the same. The rebalance option just introduces the noise at the initial time. I need to poke around with this some

[davegill]

@kalundquist
Katie,
I used your repo and branch, built the code to run serially, and I get basically the same results for a few time steps.

Do your results look OK? For the PR you put together, could you send the namelist you used to test?

[kalundquist]

Dave,
I ran two cases in v3.9.1.1 (unchanged) and 4.1 (my vert. nesting fix, not your nice refactoring) and they look the same. MU certainly has noticeable effects at the edges from vertical nesting. Effects are much more muted on the other prognostic variables, such as velocity. The causes of this are documented in the Daniels et al (2016) paper in MWR. Some of it is unavoidable and I think some could be fixed by modifying the smoothing behavior between nested domains. This commit should return the vertical nesting functionality to that before v4.0 when the hybrid coordinate was set as the default. I have only looked at the solutions for hybrid_opt = 0, not 2. I am putting my 2 test cases (input and namelists) and the results (wrfout_d04 only) up for you to view at
ftp://ftp.llnl.gov/outgoing/lundquist3/vertnesting_example.tgz
Sorry- it is 1 GB.
Katie

[davegill]

@kalundquist
Katie,
I'll take a look at your results.
Thanks

[kalundquist]

@davegill
Let me know if you want me to look at your test case. You are right that the results above do not look good.

[davegill]

@kalundquist
Katie,
Take a look at this PR now. I have not changed anything, I just went out farther into the forecast. My results with hybrid coordinate look similar to your results with terrain following.

[kalundquist]

@davegill
Dave,
This all looks good. Would you like me to look at your poorly performing test case? Or, move forward and merge?

[davegill]

@kalundquist
Katie,
I think that we are good, no need to look at the poorly performing case.

1. I use the same code for hybrid_opt 0 and 2 in nest_init_utils.F
2. I get IDENTICAL results to your PR (fix for vertical nesting without the hybrid coordinate - fix for use … #891) with this PR for your 4 domain test case, hybrid_opt=0
3. I get really similar results with hybrid_opt = 0 and 2 on d04 after 240 time steps

We'll conduct some more tests back here. Would you also test this version of the code on some known cases. If both sides are happy, we can merge this into a release branch. This PR may not make v4.1.1 (possibly released in a couple of weeks), but I don't see any holdups for making v4.1.2. While v4.1.2 may be officially released in July or August, the branch will be available earlier.

Thanks again for digging in and finding the root problem.

[kkeene44]

@davegill
I ran a basic vertical nesting test with this branch. All seems okay.

[davegill]

@weiwangncar @kkeene44 @smileMchen
Folks,

For the WRF-based vertical refinement and for the ndown-based vertical refinement, we are getting results similar to 3.9.1.1 (no hybrid coordinate). We can get similar results when using the hybrid coordinate with v4.1.1. Katie Lundquist (the original developer) has run a longer test case with the WRF-based vertical refinement. The user having troubles with ndown (#917, "NDOWN v4.1: vertical refinement is broken"), says that this PR fixes his trouble.

Let me know if there are other things that you would like to see. This feature has been broken since v4.0 with the introduction of the hybrid coordinate.

[smileMchen]

I have looked through the codes changes and tests related to this issue. Everything looks fine. I am going to approve this PR at around noon time. If anyone has any concerns, please let me know before that time. Thanks.

[weiwangncar]

@davegill Can you then add something like 'both vertical refinement methods are working with hybrid coordinate' to the 'RELEASE NOTE'? Also is the ugliness along the nest boundaries gone now?

[lukasumek]

Hi,
i just saw that starting with v4.2 a check for using the terrain-following coordinate is introduced in https://github.com/wrf-model/WRF/blob/release-v4.2/share/module_check_a_mundo.F#L2283

Can you confirm that vertical nesting (either method with vert_refine_method 1 or 2) is not available with the new hybrid vertical coordinate?

[davegill]

@lukasumek
These changes are from about a year and a half ago. Reading through the commit messages, it looks like the WRF model is now supposed to work with the hybrid coordinate and vertical refinement.

[kalundquist]

Yes, vertical nesting should work with the new coordinate now. From: lukasumek <[email protected]> Reply-To: wrf-model/WRF <[email protected]> Date: Sunday, October 11, 2020 at 9:31 AM To: wrf-model/WRF <[email protected]> Cc: "Lundquist, Katie" <[email protected]>, Mention <[email protected]> Subject: Re: [wrf-model/WRF] Fix vertical refinement, broken from v4.0 through v4.1 (#901) Hi, i just saw that starting with v4.2 a check for using the terrain-following coordinate is introduced in https://github.com/wrf-model/WRF/blob/release-v4.2/share/module_check_a_mundo.F#L2283 Can you confirm that vertical nesting (either method with vert_refine_method 1 or 2) is not available with the new hybrid vertical coordinate? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#901 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AFPMMVMSWJEJPL2SLLELQQDSKHMV3ANCNFSM4HLWBFOA>.

[hawbecker]

Sorry to resurrect the dead thread, but I am having some issues with vertical refinement in ndown when using specified eta levels and vert_refine_method = 0,0,0,2 using v4.2.

First, is it maybe not possible to use vert_refine_method = 2 and specified eta levels with ndown (only vert_refine_fact seems to work right now)?

What I am doing is running WPS and real.exe for 4 domains with vert_refine_method = 0,0,0,2, and defined eta levels as follows:
eta_levels(1:60) = [60 levels]
eta_levels(61:120) = [same 60 levels]
eta_levels(121:180) = [same 60 levels]
eta_levels(181:369) = [188 levels]

This all works fine and gives the expected output for all domains.

I then do the following:

1. Run WRF for just the first 3 domains to get the input for ndown
2. I move the wrfinput_d04 to wrfndi_d02, rename the wrfout_d03* files to wrfout_d01*, and then delete the first 2 columns in the namelist for all max_dom variables
3. Run ndown

The result is wrfinput_d02 has 60 levels. The eta levels are the same as d01 (the original 60 specified levels), but the heights [(PH+PHB)/g] correspond to the first 60 heights from wrfinput_d04 (the 188 levels). This effectively squeezes the model data down to have a lower model top.

wrfndi_d02 (previously wrfinput_d04)

[[1.      0.99739 0.9948  0.9922  0.98962 0.98704 0.98446 0.98189 0.97933
  0.97677 0.97422 0.97168 0.96914 0.96661 0.96408 0.96156 0.95905 0.95654
  0.95404 0.95154 0.94905 0.94656 0.94408 0.94161 0.93914 0.93668 0.93423
  0.93178 0.92933 0.92689 0.92446 0.92203 0.91961 0.9172  0.91479 0.91238
  0.90998 0.90759 0.9052  0.90282 0.90044 0.89807 0.89571 0.89335 0.89099
  0.88865 0.8863  0.88396 0.88163 0.87931 0.87699 0.87467 0.87236 0.87006
  0.86776 0.86546 0.86317 0.86089 0.85861 0.85634 0.85407 0.85181 0.84956
  0.84731 0.84506 0.84282 0.84058 0.83835 0.83613 0.83391 0.8317  0.82949
  0.82728 0.82509 0.82289 0.8207  0.81852 0.81634 0.81417 0.812   0.80984
  0.80768 0.80553 0.80338 0.80124 0.7991  0.79697 0.79484 0.79272 0.7906
  0.78849 0.78638 0.78428 0.78219 0.78009 0.77801 0.77592 0.77385 0.77177
  0.7697  0.76764 0.76558 0.76343 0.76117 0.7588  0.75633 0.75374 0.75102
  0.74818 0.74521 0.7421  0.73885 0.73545 0.73189 0.72817 0.72429 0.72023
  0.71598 0.71155 0.70692 0.70209 0.69705 0.69179 0.6863  0.68058 0.67462
  0.66841 0.66194 0.65521 0.6482  0.64091 0.63334 0.62546 0.61729 0.6088
  0.6     0.59087 0.58141 0.57162 0.56149 0.55101 0.54019 0.52901 0.51749
  0.50561 0.49339 0.48081 0.46789 0.45463 0.44104 0.42711 0.41288 0.39834
  0.38351 0.36841 0.35305 0.33746 0.32165 0.30567 0.28952 0.27324 0.25687
  0.24044 0.22445 0.2092  0.19465 0.18077 0.16752 0.15489 0.14284 0.13134
  0.12037 0.10991 0.09992 0.0904  0.08132 0.07265 0.06438 0.0565  0.04897
  0.0418  0.03495 0.02842 0.02219 0.01624 0.01057 0.00516 0.     ]]

wrfinput_d03 (result from real)

[[1.     0.9974 0.9945 0.9914 0.9879 0.9841 0.98   0.9754 0.9704 0.965
  0.959  0.9525 0.9454 0.9376 0.9291 0.9198 0.9097 0.8987 0.8868 0.8738
  0.8597 0.8444 0.8279 0.81   0.7908 0.7701 0.7479 0.7241 0.6987 0.6716
  0.6429 0.6125 0.5805 0.5469 0.5118 0.4754 0.4378 0.401  0.3666 0.3346
  0.3047 0.2768 0.2507 0.2265 0.2038 0.1827 0.163  0.1446 0.1274 0.1114
  0.0965 0.0826 0.0696 0.0574 0.0461 0.0356 0.0257 0.0166 0.008  0.    ]]

wrfinput_d02 (result from ndown)

[[1.         0.99738383 0.9945132  0.991364   0.98791033 0.98412377
  0.9799736  0.97542655 0.9704467  0.9649952  0.9590304  0.95250714
  0.9453776  0.9375902  0.9290904  0.91982013 0.9097182  0.8987205
  0.88675964 0.8737664  0.8596691  0.84439504 0.827871   0.8100244
  0.79078466 0.7700848  0.7478632  0.72406626 0.6986502  0.6715851
  0.642857   0.6124726  0.58046204 0.54688364 0.5118273  0.47541872
  0.43782222 0.40099037 0.3666298  0.3345748  0.30467057 0.2767728
  0.25074688 0.22646725 0.20381668 0.18268591 0.16297296 0.14458269
  0.12742637 0.11142118 0.09648987 0.08256042 0.06956565 0.05744274
  0.04613322 0.03558259 0.02573987 0.01655755 0.00799132 0.        ]]

Heights (m)

	 ndi	 ndown	 d03
0	0.0	0.0	0.0
1	21.1	20.9	20.7
2	42.0	41.7	43.8
3	63.1	62.5	68.6
4	84.0	83.3	96.6
5	105.0	104.0	127.1
...
57	1164.8	1218.0	15288.6
58	1184.7	1244.3	15720.9
59	1204.6	1270.8	16150.9
60	1224.5
61	1244.4
...
186	15906.9
187	16187.4

Notice the eta levels for the wrfinput_d02 from ndown and those from wrfinput_d03 are essentially the same, but the heights from the wrfinput_d02 (ndown) are essentially equal to those from wrfinput_d04 (or wrfndi_d02).

If there's anything else I can provide to help with this, please let me know. I know the other option is to run WRF concurrently (not using ndown) which seems like it will work given real.exe worked well, but we're hoping to compare WRF LES with another LES and are hoping to do both in an offline mode for a fair comparison.

[kalundquist]

Patrick, This issue was for fixing vertical refinement for concurrently running domains, where ndown is not used. As far as I am aware, ndown only works with integer refinement for increasing vertical resolution, while with the method for concurrent runs you can either use integer refinement or specify eta levels independently. If you want ndown to work with specified eta levels, you will likely need to program it. Katie From: Patrick Hawbecker <[email protected]> Reply-To: wrf-model/WRF <[email protected]> Date: Thursday, December 10, 2020 at 4:32 PM To: wrf-model/WRF <[email protected]> Cc: "Lundquist, Katie" <[email protected]>, Mention <[email protected]> Subject: Re: [wrf-model/WRF] Fix vertical refinement, broken from v4.0 through v4.1 (#901) Sorry to resurrect the dead thread, but I am having some issues with vertical refinement in ndown when using specified eta levels and vert_refine_method = 0,0,0,2 using v4.2. First, is it maybe not possible to use vert_refine_method = 2 and specified eta levels with ndown (only vert_refine_fact seems to work right now)? What I am doing is running WPS and real.exe for 4 domains with vert_refine_method = 0,0,0,2, and defined eta levels as follows: eta_levels(1:60) = [60 levels] eta_levels(61:120) = [same 60 levels] eta_levels(121:180) = [same 60 levels] eta_levels(181:369) = [188 levels] This all works fine and gives the expected output for all domains. I then do the following: 1. Run WRF for just the first 3 domains to get the input for ndown 2. I move the wrfinput_d04 to wrfndi_d02, rename the wrfout_d03* files to wrfout_d01*, and then delete the first 2 columns in the namelist for all max_dom variables 3. Run ndown The result is wrfinput_d02 has 60 levels. The eta levels are the same as d01 (the original 60 specified levels), but the heights [(PH+PHB)/g] correspond to the first 60 heights from wrfinput_d04 (the 188 levels). This effectively squeezes the model data down to have a lower model top. wrfndi_d02 (previously wrfinput_d04) [[1. 0.99739 0.9948 0.9922 0.98962 0.98704 0.98446 0.98189 0.97933 0.97677 0.97422 0.97168 0.96914 0.96661 0.96408 0.96156 0.95905 0.95654 0.95404 0.95154 0.94905 0.94656 0.94408 0.94161 0.93914 0.93668 0.93423 0.93178 0.92933 0.92689 0.92446 0.92203 0.91961 0.9172 0.91479 0.91238 0.90998 0.90759 0.9052 0.90282 0.90044 0.89807 0.89571 0.89335 0.89099 0.88865 0.8863 0.88396 0.88163 0.87931 0.87699 0.87467 0.87236 0.87006 0.86776 0.86546 0.86317 0.86089 0.85861 0.85634 0.85407 0.85181 0.84956 0.84731 0.84506 0.84282 0.84058 0.83835 0.83613 0.83391 0.8317 0.82949 0.82728 0.82509 0.82289 0.8207 0.81852 0.81634 0.81417 0.812 0.80984 0.80768 0.80553 0.80338 0.80124 0.7991 0.79697 0.79484 0.79272 0.7906 0.78849 0.78638 0.78428 0.78219 0.78009 0.77801 0.77592 0.77385 0.77177 0.7697 0.76764 0.76558 0.76343 0.76117 0.7588 0.75633 0.75374 0.75102 0.74818 0.74521 0.7421 0.73885 0.73545 0.73189 0.72817 0.72429 0.72023 0.71598 0.71155 0.70692 0.70209 0.69705 0.69179 0.6863 0.68058 0.67462 0.66841 0.66194 0.65521 0.6482 0.64091 0.63334 0.62546 0.61729 0.6088 0.6 0.59087 0.58141 0.57162 0.56149 0.55101 0.54019 0.52901 0.51749 0.50561 0.49339 0.48081 0.46789 0.45463 0.44104 0.42711 0.41288 0.39834 0.38351 0.36841 0.35305 0.33746 0.32165 0.30567 0.28952 0.27324 0.25687 0.24044 0.22445 0.2092 0.19465 0.18077 0.16752 0.15489 0.14284 0.13134 0.12037 0.10991 0.09992 0.0904 0.08132 0.07265 0.06438 0.0565 0.04897 0.0418 0.03495 0.02842 0.02219 0.01624 0.01057 0.00516 0. ]] wrfinput_d03 (result from real) [[1. 0.9974 0.9945 0.9914 0.9879 0.9841 0.98 0.9754 0.9704 0.965 0.959 0.9525 0.9454 0.9376 0.9291 0.9198 0.9097 0.8987 0.8868 0.8738 0.8597 0.8444 0.8279 0.81 0.7908 0.7701 0.7479 0.7241 0.6987 0.6716 0.6429 0.6125 0.5805 0.5469 0.5118 0.4754 0.4378 0.401 0.3666 0.3346 0.3047 0.2768 0.2507 0.2265 0.2038 0.1827 0.163 0.1446 0.1274 0.1114 0.0965 0.0826 0.0696 0.0574 0.0461 0.0356 0.0257 0.0166 0.008 0. ]] wrfinput_d02 (result from ndown) [[1. 0.99738383 0.9945132 0.991364 0.98791033 0.98412377 0.9799736 0.97542655 0.9704467 0.9649952 0.9590304 0.95250714 0.9453776 0.9375902 0.9290904 0.91982013 0.9097182 0.8987205 0.88675964 0.8737664 0.8596691 0.84439504 0.827871 0.8100244 0.79078466 0.7700848 0.7478632 0.72406626 0.6986502 0.6715851 0.642857 0.6124726 0.58046204 0.54688364 0.5118273 0.47541872 0.43782222 0.40099037 0.3666298 0.3345748 0.30467057 0.2767728 0.25074688 0.22646725 0.20381668 0.18268591 0.16297296 0.14458269 0.12742637 0.11142118 0.09648987 0.08256042 0.06956565 0.05744274 0.04613322 0.03558259 0.02573987 0.01655755 0.00799132 0. ]] Heights (m) ndi ndown d03 0 0.0 0.0 0.0 1 21.1 20.9 20.7 2 42.0 41.7 43.8 3 63.1 62.5 68.6 4 84.0 83.3 96.6 5 105.0 104.0 127.1 ... 57 1164.8 1218.0 15288.6 58 1184.7 1244.3 15720.9 59 1204.6 1270.8 16150.9 60 1224.5 61 1244.4 ... 186 15906.9 187 16187.4 Notice the eta levels for the wrfinput_d02 from ndown and those from wrfinput_d03 are essentially the same, but the heights from the wrfinput_d02 (ndown) are essentially equal to those from wrfinput_d04 (or wrfndi_d02). If there's anything else I can provide to help with this, please let me know. I know the other option is to run WRF concurrently (not using ndown) which seems like it will work given real.exe worked well, but we're hoping to compare WRF LES with another LES and are hoping to do both in an offline mode for a fair comparison. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#901 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AFPMMVJIVZ7UYYAEOFLBNZTSUFSCRANCNFSM4HLWBFOA>.

[hawbecker]

Sorry, @kalundquist - I didn't realize this would send you a notification as we had already discussed via email.

Re-reading the first comment I see where I misunderstood the use of this fix: "vert_refine_method = 1 or 2 (ndown or WRF, respectively)" saying ndown only uses vert_refine_method = 1. I will look into the code to see what needs to be changed for ndown to use option 2.

[dudhia]

Pat, that may require some consulting with @davegill. I don't think it will be an easy or quick fix but if you can familiarize yourself with ndown, I and Dave may be able to guide you between us, Jimy

…

On Fri, Dec 11, 2020 at 9:07 AM Patrick Hawbecker ***@***.***> wrote: Sorry, @kalundquist <https://github.com/kalundquist> - I didn't realize this would send you a notification as we had already discussed via email. Re-reading the first comment I see where I misunderstood the use of this fix: "vert_refine_method = 1 or 2 (ndown or WRF, respectively)" saying ndown only uses vert_refine_method = 1. I will look into the code to see what needs to be changed for ndown to use option 2. — You are receiving this because your review was requested. Reply to this email directly, view it on GitHub <#901 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEIZ77CUWRYGOZLUQK7LX73SUI7TFANCNFSM4HLWBFOA> .