Should we allow the HLM to handle 'fast fluxes'?

[wwieder]

Following a CLM discussion a few weeks ago that was led by @rgknox and Gordon Bonan I was struck by the level of duplication related to fast fluxes for leaf level photosynthesis, stomatal conductance, radiation, etc. This made me wonder if the HLM should be responsible for these short time scale fluxes, with FATES providing information on canopy structure & slower fluxes related to allocation, stand structure, mortality, etc?

I appreciate this is not a trivial question, but thought it could be worth having this discussion. Looking forward to hearing your thoughts.

[rosiealice]

My feeling is somewhat that this might create as many issues as it solves. Specifically relating to parameters, if FATES is handling all the vegetation processes, but then a subset of those are reallocated ro the HLM, we end up in a situation where the HLM parameter file has to know about the FATES PFTs. The HLM currently contains no information about the FATES PFTs, which can be substantially and arbitrarily different from the HLM, and so duplicating thesis definition across both models might add, for users, a whole world of chaos while trying to parameterise plant functional strategies?

One option one might explore is a shared module (like MAAT) for leaf level fluxes that takes all the parameters and climate forcing external to the leaf as an input and gives A, rd and gs back?

This is plausibly something that will be a topic in the upcoming ILMF modularity discussion/webinar. In principle it's applicable across numerous models...

[adrifoster]

Specifically relating to parameters, if FATES is handling all the vegetation processes, but then a subset of those are reallocated ro the HLM, we end up in a situation where the HLM parameter file has to know about the FATES PFTs.

I think perhaps we could solve this by having the HLM use the FATES parameters if FATES is being used. The HLM is already in charge of reading in the parameter file and sending that information back to FATES so it stands to reason that it could also use those parameters?

[jenniferholm]

Hi, just (very quickly here) agree with Rosie's comments that this might lead to a lot of confusion, and possible duplication that might cause unintended consequences/errors..... (more trouble than it's worth?? maybe.... not sure...)

[ekluzek]

Interesting discussion. I like @adrifoster point about using the FATES parameters from the FATES file in the HLM. That makes total sense to me, and would seem to resolve any issues, as far as I would see.

I want to make sure I'm understanding @rosiealice comment, which makes sense to me from a code management point of view. I think you are saying that we would have a FATES object that will be sent back to the HLM, that would have a few methods for doing things that the HLM needs from FATES. The data and methods inside that object would be limited to only things that the HLM needs from FATES. And it might be something that the HLM would be querying inside its faster time updates. @rosiealice is that what you are getting at? @adrifoster is that something that would make sense to you?

[adrifoster]

I think you are saying that we would have a FATES object that will be sent back to the HLM, that would have a few methods for doing things that the HLM needs from FATES. The data and methods inside that object would be limited to only things that the HLM needs from FATES. And it might be something that the HLM would be querying inside its faster time updates

I like this idea, the HLM would essentially pass back any information FATES needs to do ecosystem demography, daily, and then FATES would pass back it's height, pft, etc. structure to the HLM. The HLM would then get to decide what to do with that structure in order to do physics. The major thing would be how to differentially allocate carbon to the different cohorts and PFTs on the FATES side.

[adrifoster]

I will note that this has been done in the past with models like the HYBRID model (Friend et al. 1993), Fire-BGC, and the like.

[ekluzek]

@jenniferholm I want to react to your point about duplication. Since, I work on the HLM side of things I see the current structure as leading to MORE duplication between FATES and the HLM. We have processes that are in both the HLM and in FATES. And part of what that means is that if one of those things is developed on the HLM side of things -- it can't be used by FATES. Obviously we want FATES to determine the vegetation characteristics, but the faster cycle things like radiation, surface energy balance, and the like we have duplicated in both. Actually, it might facilitate more science if you could choose between using the FATES parameterization for these things or the HLM option for these things. When something is developed for the HLM side, you can't immediately put it inside of FATES without a lot of work. And the same for the reverse.

[rosiealice]

Hi all, Good points you raise Erik. I guess I was actually maybe talking about introducing a third repo to handle the leaf level gas exchange itself. This is partly because we were discussing modularity on the call I was on yesterday and leaf gas exchange is its obvious use case. Now I write it like that it seems like that might be excessive. I am still also unsure that moving some part of the vegetation physiology to the HLM won’t cause confusion. While the HLM has a big leaf model, in effect all of the vegetation physiology is to some degree duplicated (FATES and CLM both do allocation , turnover, growth etc)? So it seems slightly arbitrary to specifically put the gas exchange stuff back into the HLM? Why not then also put phenology, fire, etc. back? If you go down that road, you end up with a fairly disjointed activity where things that can be modularised are in the HLM and things that can’t are in FATES. So I guess I think we should think through what the end goal of this would be before we embarked on it? At any rate, we should all definitely tune into the webinar on modularity!

…

------------------------- Dr Rosie A. Fisher Senior Researcher CICERO Center for International Climate Research Oslo, Norway https://cicero.oslo.no<https://cicero.oslo.no/>

________________________________ From: Adrianna Foster ***@***.***> Sent: Friday, August 25, 2023 9:38:19 PM To: NGEET/fates ***@***.***> Cc: Rosie Fisher ***@***.***>; Mention ***@***.***> Subject: Re: [NGEET/fates] Should HLM handle 'fast fluxes'? (Discussion #1067) I will note that this has been done in the past with models like the HYBRID model (Friend et al. 1993<https://www.jstor.org/stable/1940806>), Fire-BGC<https://www.firelab.org/project/firebgcv2-landscape-fire-model>, and the like. — Reply to this email directly, view it on GitHub<#1067 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ADC2YQ5ATY6BTZVDD62LDY3XXD5KXANCNFSM6AAAAAA35DTSCE>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[ekluzek]

I want to make sure we acknowledge @adrifoster contributions to this discussion. I'm coming at it mainly from the software management angle, while she has a better picture of both the science AND the software angles here. And she has really nice points about the FATES parameter file, and references for other models that do this.

I'd like to see the software side of those examples she gave in that reference and see how it might apply here. I'm thinking @adrifoster can help us all to do that.

@rosiealice unfortunately "modularity" is an overloaded buzzword. It's certainly an important thing to strive for, but can be implemented in different ways and means different things to different people. @rosiealice you mention a specific webinar on this, is that something you can point us to?

This discussion does propose a way to decide whether something belongs in FATES or the HLM. FATES does the daily cycle for determining what plants are there and where they are in space, and what the canopy looks like. The HLM would then interact with FATES for things like the gas exchange that's on a faster time cycle. So I think there is a clear line of what goes in FATES and what belongs in the HLM and not confusion. You are right there are also some things that are on the border between the two. But, those would be the things that you setup a clear interface to handle. When you do that there is a clear definition of how these things are intended to interact with each other. It actually brings clarity to both the HLM and to FATES.

You bring up things that are duplicated in the HLM for "big-leaf". Those are things that are so tied to the vegetation model, that I don't see a clear way on how they could be shared. I feel like the vegetation model HAS to be duplicated, but that's OK as they are so different. It's possible that there are small parts that could be interchanged, if they were small functions that could be shared. But, the larger work can't really be.

Yes, I agree that adding another repository wouldn't help here, the fewer repositories the better in terms of being able to get things done. But, defining the interface for the fast time cycle things between FATES and HLM does sound worthwhile.

I also think it's true that FATES development is mainly with the slower cycle vegetation part of the model. What maybe isn't worked on "as much" are these faster cycle things, but they are being developed in the HLM's. And if you can only work on them in EITHER the HLM or in FATES it's a problem for adoption of FATES in the HLM's and in moving those developments over to FATES. @rosiealice and @adrifoster does this seem like a fair assessment?

I also disagree with your statement "If you go down that road, you end up with a fairly disjointed activity where things that can be modularized are in the HLM and things that can’t are in FATES". I think we want modularity (at least in a software sense) in both the HLM and FATES. Although maybe I'm not quite understanding what you are getting at here?

I also think the adoption of this could be incremental. Just enabling making it possible to use the HLM for some processes while retaining those same capabilities in FATES allows for more science. That also gives you a road to experiment with this idea before it becomes the default option. If it's a failed experiment that's fine, but like all failed experiments in science -- failed experiments mean progress. @adrifoster I think I'm channelling or maybe even taking credit for some of your ideas here, can you add to this?

I also totally agree with your point @rosiealice "So I guess I think we should think through what the end goal of this would be before we embarked on it?". That is always good to do and this discussion is starting to get that defined. It's certainly getting us all thinking "outside the FATES box" and at other ways of doing things that could be really helpful for both FATES and the HLM's.

[ekluzek]

Ahh, September 23rd for the webinar! OK, I'll put that on my calendar. It's at 9am MDT so that works, although we'll have to move our CTSM software meeting, which of course is fine.

[ekluzek]

@rosiealice on putting all fast cycle code into FATES. This is a good thing to think about as a path to reduce code duplication. This is a brainstorming exercise so we should throw out all ideas and consider each one. Two considerations that need to come to play, are that the HLM options for fast cycling would then need to be moved to inside of FATES. So that's an upfront cost. The other thing that I'm not so sure about is that I think the HLM's will still want to retain a non-FATES option. That will at least be required for several years anyway. I think the main issue with this though is that will require that developers working on a particular HLM will need to work both in FATES and the HLM. So in my case both inside of CTSM and inside of FATES. A lot of developers are going to have an easier time just staying inside of one or the other. I think honestly that's the argument that many FATES scientists are saying is the reason -- they wouldn't like to do this -- "I'm more comfortable working just inside of FATES". And that's not a bad justification. But, we need to be cognizant of both sets of people, people that primarily work inside of FATES and the people working primarily inside the HLM. When I think about it that way, I think this allowing for the HLM to handle fast fluxes is better for both communities. People that want to work just inside the HLM can do so, others can work just inside FATES, and still others can have their hands in both. Since, it's harder to do both -- the people that really do that will be more limited. So I'm thinking the proposal to allow HLM's to handle fast fluxes is more flexible for all communities that use the HLM's. So that's an initial reaction to your question. I'm actually really glad that everyone is thinking about how we can reduce code duplication. So thanks for asking @rosiealice.

[rosiealice]

Hi Erik,

Thanks. And just quickly as I've a meeting, I do not mean all fast fluxes could be in FATES code, just the leaf level physiology (radiation, gs and psn), as those are vegetation elements.

[rosiealice]

Backing up slightly, I don't actually think the status quo is that bad. It is really -only- the photosynthesis code that is duplicated, and there are not all that many examples of people having to implement things twice.

What is more clear is that putting the leaf gas exchange solely in either in HLM or FATES code would 1) be work and 2) have downsides either way. Those might well be bigger than the minor duplication issue we have now?

My suspicion, going back to @wwieder's original post, is that if we start to grapple properly with how to interface with the multi-layer turbulence code, we might well make decisions that look different. But that is a different motivation than reducing duplication,in the sense that it is really creating a very different model altogether.

Maybe it is best to park this until we have the bandwidth to do that properly?

[wwieder]

Good point, @rosiealice this larger aim related to multi-layer turbulence would take considerable effort on all sides that in the short term may be better directed elsewhere.

[wwieder]

This is a helpful discussion. I have to admit that my understanding of all this is still pretty naive.
I also appreciate that there is an organization to the FATES effort that this inquiry potentially alters.

I guess my motivation ultimately comes down to the description of FATES as an ecosystem demography model, and wondering if we're creating too many challenges for ourselves by making the demography model also handle canopy fluxes? The HLMs already handle canopy fluxes, given simplifying assumptions about the structure and composition of that canopy. If we worked on the HLM side to include additional information from FATES about evolving structure and composition of the canopy, would this ultimately accelerate our progress?

I'm admittedly thinking of this from the HLM side and how to handle:

• new pull requests (e.g., roughness length calculations and multi-layer canopy)
• continued duplication of existing code (e.g., @rgknox's two-stream radiation PR, presumably canopy interception of snow, etc)
• building towards desired model features (e.g., BGC runs with FATES that also include explicit crop management).

This last example is maybe instructive to consider? Could we create a system where HLM simulates canopy fluxes for natural vegetation or crops given information about phenology, LAI and canopy structure from either FATES or a crop model?

This would give different FATES-side answers across HLM platforms that could complicate the calibration FATES demography parameters. Definitely not something to do all at once, but I do wonder if it's a direction we should consider working towards?

[rosiealice]

Hi Will,

I guess we might need a call to sort through some of the things in this thread, but just to respond to a couple of the points you raise here...

• The first is on the idea of there being duplication of the radiation code. While Ryan's new code is called 'twostream', it doesn't really have any analogue in the HLM. Both FATES RTM codes (the old norman and new twostream) implement the capability to have plants compete for light, for different size cohorts to inhabit the same vertical column, for these cohorts to have different widths and canopy depths and for these all to be organized into canopy layers (which themselves may have gaps in).
  Managing how the light streams are organised in this space is enormously complex and, moreover, it is very much the scientific core of FATES, as it is the competition for light that drives the demography in the first instance. Expertise on this is in very short supply, and so having alternate versions of that code living in different HLM repos would, to put it mildly, be a considerable challenge...

• on snow, I don't think FATES does the snow, short of considering the impact of snow on the canopy albedo.

• on the description of the model, I generally always refer to FATES as a model of vegetation, not as an ecosystem demography model. To me this makes more sense as an entity, because ultimately, competition is driven by both fast and slow timescale processes. And so the existing delineation seems like a good way of integrating everything relevant to plants in one place.

[rgknox]

My take is that FATES handles snow fairly crudely, but... FATES isn't in charge of surface reflectance on areas that are bare ground. And FATES does modify its scattering coefficient (more or less the effective albedo of the vegetation) for vegetation if there is snow, it does it in the same way CLM has done it in the past. But this does not take into account if the frozen water is ice, snow, or anything related to how "dirty" it is. It also doesn't have any capability to modify things like back-scatter fraction, or optical depth. The new fates two-stream code does not break from this, but I do think it would be a big improvement over the existing code to update it and incorporate elements of other
existing canopy snow/ice scattering algorithms.

[rgknox]

There are a few things I wanted to point out related to the software design of the soon-to-be two-stream module in FATES:

Earlier in this thread there is a discussion on how to avoid parameter chaos, if FATES finds itself relying on an external module to handle various processes. @adrifoster has a suggestion that is sounds pretty consistent with the new fates two-stream design. With the new two-stream, the relevant parameters are defined inside the module itself, so either FATES or CLM could have an interface with the module that fills the parameters and associates them with a PFT Index:

https://github.com/rgknox/fates/blob/two-stream-clean/radiation/TwoStreamMLPEMod.F90#L85-L103

Also, there was some discussion around the scaling assumptions involved. Radiative transfer is somewhat special because it has to carry scaling assumptions (ie how we chop up the canopy into bits, orient them in space and connect them) with it. However, the design for the data structures that capture and define this scaling was intended to accommodate a single scattering element (ie a big leaf), or a stack of scattering elements interacting with each other in serial (kind of like how Gordon lays out his vertical canopy structure for turbulence), or a mix where scattering elements can share vertical space, and also co-interact with higher or lower layers (which is associated with the PPA assumptions we have in FATES now). The key is to have a clean interface between the host model (CLM or FATES) and how we populate and fill in these data structures. The hope is that the extra machinery necessary to accommodate complex scattering environments does no slow down calculations on trivial environments like the big-leaf (I'm optimistic this won't be an issue, the math on this won't be more complicated for big-leaf than it already is).

Note that in the two-stream module, where all the core two-stream calculations happen, there are no external dependencies on FATES or CLM, with the exception of the share environments used for logging and error reporting.

https://github.com/rgknox/fates/blob/two-stream-clean/radiation/TwoStreamMLPEMod.F90#L27-L28

[alistairrogers]

I would advocate for keeping leaf-level photosynthesis and stomatal conductance in FATES. Both the structural representation of leaf physiology and its parameterization vary considerably among HLMs and both structural and parametric choices have a marked impact on competition. I don't think it is as simple as sharing a PFT parameter file. My experience is that FATES is currently very nimble in that adjustments to fast processes and parameterization can occur quickly and the impact on demography be determined consistently. Relying on HLM for the fast processes would make advancing understanding and model representation of leaf physiology much more challenging for NGEE tropics.

[ekluzek]

@alistairrogers thanks for your comment here. We've changed the title of this to make this optional rather than required. Your comment is addressing if it was required. There's enough comments that we wouldn't want to make this required. And even if you did want to make it required a transition plan where its optional first would be the way it would need to be done.

Taking that as an assumption that this would be optional do you see any issue with that? It would be option both for each HLM to take advantage of, and optional for each HLM to use FATES "fast fluxes" or that HLM's "fast fluxes". If this is optional I can't think of an objection, but maybe I've missed something. What are your thoughts on that?

[gbonan]

There are a lot of interesting thoughts here. My overall reading of the thread is that the big context is the coupling between FATES and CLM and the boundaries between the two models: what processes does FATES calculate, what does CLM do, what is a FATES parameter, what is a CLM parameter, etc. By treating FATES and CLM as two distinct models, we inevitably talk about the software engineering interface rather than a unified model.

The big-picture is that the tight coupling (interdependency) among leaf energy fluxes, leaf temperature, stomatal conductance, and photosynthesis is absent. FATES does photosynthesis (and by necessity stomatal conductance) in a layered, mixed PFT canopy (therefore requiring its own radiative transfer) to model competition for light, but CLM does leaf energy fluxes and leaf temperature in a big-leaf canopy to pass fluxes to CAM. CLM, for all its warts, integrates the canopy biogeophysics (leaf energy balance, leaf temperature, stomatal conductance) and photosynthesis in a consistent way. This close coupling was the defining feature of the third-generation land models developed in the 1990s. Instead, there is now a somewhat arbitrary distinction between canopy biogeophysics (CLM) and leaf physiology (FATES).

There is a strong rationale to separate fast timescale flux calculations needed for coupling with the atmosphere at every time step and slower timescale changes in canopy structure. But that is not the way in which FATES and CLM are coupled. The multilayer canopy modeling further confuses things because radiative transfer, photosynthesis, stomatal conductance, plant hydraulics, leaf energy balance, leaf temperature, and turbulent transfer are all integrated and solved in a 5-minute sub-timestep loop.

I am reminded of a statement by John Finnigan and Mike Raupach in 1987 (in a book on stomatal functioning): “the physiological state of a plant community substantially influences the microclimate within it; in turn, the microclimate influences the physiological state, so that neither is independent of the other”. By treating leaf physiology separate from canopy biogeophysics, the essential interdependency is removed.

[rosiealice]

Hi Gordon, - Thanks for this nice summary,. Indeed, the reason I thought we should recast this discussion is to make sure we don’t go down the wrong path wrt to any eventual integration of the multi layer model… - Just to make sure we are not talking at cross purposes though, the leaf fluxes in FATES are still very tightly coupled to the surface exchange. When FATES is on, the interaction between the leaf level fluxes and the canopy temperature loop works in exactly the same way it does in CLM. The leaf routines are called in the same place and the logic of passing the canopy resistance terms is identical. - It is indeed a challenge to come up with a scheme that ‘redraws’ the code design we have now, (which I think works quite well in the context of the single energy balance canopy) into one where there is a multi-layer energy balance scheme maybe in one HLM and not in another. I think we all need to mull that over a little to come up with a design that makes sense both in terms of software, science, and the community dynamics around the model. - I am wary of trying to run before we can walk though! We have some short-term high priority things we need to push through to get FATES working for TRENDY and coupled simulations (calibration, land use change, nutrient coupling with CLM, BVOCs). The equilibrium we have right now where all the HLMs can work together on this is a great, necessary (given the scope of the task and the subcritical resources in each group and the very intense demand for the model from many quarters) and -hard won- situation. That is why we are quite wary of perturbing it right now. I still think after all this back and forth that the answer -may- lie in a single shared photosynthesis code module...

[rgknox]

"- Just to make sure we are not talking at cross purposes though, the leaf fluxes in FATES are still very tightly coupled to the surface exchange. When FATES is on, the interaction between the leaf level fluxes and the canopy temperature loop works in exactly the same way it does in CLM. The leaf routines are called in the same place and the logic of passing the canopy resistance terms is identical."

That was my understanding as well

[gbonan]

Rosie and Ryan: Thanks for the clarification. Speaking of running before walking… I should have looked at the code first. I see now that photosynthesis is called within the leaf temperature iteration of CanopyFluxes. All that is different is whether the call is to a CLM module or a FATES module. I am not sure how I got the idea that this was not being done. I think it is because we talk about FATES as a separate model from CLM. For photosynthesis, we are really talking about simply a call to a new module (just as we already have multiple CLM photosynthesis modules). So a shared photosynthesis module is a great goal to work for and would will help avoid confusion. We (in TSS) also need to have a “coupling guide” that makes it clear where in the code FATES modules are used, what they replace in CLM, and what is being passed between the models and why (e.g., roughness length, leaf dimension). Maybe such a document already exists, and of course the exact answer is in the code, but it would be useful to have an easy-to-read document.

[rosiealice]

Hi Gordon,
I think you are most of the way there in terms of understanding the interaction between FATES and CLM. The other pieces are indeed, roughness length for which fates calculates and returns a canopy average value, and albedo for which it does the same...

The other interface is with the soil BGC. At the moment this is a one way carbon only coupling, but high up on our list is writing the N uptake code for CLM.

We have documentation of course:

https://github.com/NGEET/fates-docs

but I think what you were talking about is a bit more specific. Maybe a shared document with questions (of the 'I was too afraid to ask' category!) might work best, to figure out the scope of what people are still unclear on and then we can create the appropriate information to fill those gaps? You could even anonymise it first if you wanted ;)