Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor icepack interfaces #285

Merged
merged 10 commits into from
Dec 8, 2019
Merged

Conversation

apcraig
Copy link
Contributor

@apcraig apcraig commented Nov 24, 2019

DO NOT MERGE THIS

PR checklist

  • Short (1 sentence) summary of your PR:
    Update Icepack Interfaces
  • Developer(s):
    apcraig
  • Suggest PR reviewers from list in the column to the right.
  • Please copy the PR test results link or provide a summary of testing completed below.
    ENTER INFORMATION HERE
  • How much do the PR code changes differ from the unmodified code?
    • bit for bit
      except fix for icepack bgcskl, nblyr was set incorrectly
    • different at roundoff level
    • more substantial
  • Does this PR create or have dependencies on CICE or any other models?
    • Yes
    • No but will require an update in CICE
  • Does this PR add any new test cases?
    • Yes
    • No
  • Is the documentation being updated? ("Documentation" includes information on the wiki or in the .rst files from doc/source/, which are used to create the online technical docs at https://readthedocs.org/projects/cice-consortium-cice/.)
    • Yes
    • No, does the documentation need to be updated at a later time?
      • Yes
      • No
  • Please provide any additional information or relevant details below:

For now, this serves as a new place to discuss code changes.

This implementation addresses

  • updates icepack_step_radiation interface by passing bgc tracer slices as well as removing redundant tracer indices and flags. Also changes zbion argument name to trcrn_bgcsw.
  • allocate trcrn_bgcsw in driver code to size nbtrcr_sw which is more appropriate
  • rename icepack_init_ocean_conc and icepack_init_oceanConcArray to icepack_init_ocean_bio and icepack_load_ocean_bio_array
  • adds defaults for tracer indices and flags

A couple of bgc cpp sizes had to be corrected for this to work. I believe the size of NBGCLYR in bgcsklNICE was incorrect and impacted the results.

See also CICE-Consortium/CICE#379
See prior discussion at CICE-Consortium/CICE#378

@apcraig
Copy link
Contributor Author

apcraig commented Nov 24, 2019

Would love some feedback whether this is heading in the correct direction. I will be starting to generate some documentation on the interfaces and in particular, the tracer indexing. That will be added to the icepack user guide.

I have several open issues and questions.

  • We should review trcrn_bgcsw in icepack_shortwave for the indices. In particular, use of nbtrcr_sw seems like a bad implementation. This is the last index and does introduce a lot of risk if the order of the tracers changes. What is this tracer supposed to be. I know we might get away with this because it's only in skl mode, but still....

  • I have been looking at the "max" arguments in icepack_load_ocean_bio_array and am very confused about what should be passed and what's happening. Why are there two additional bio_index arrays for yet another indexing strategy. And why are those allocated to absolute full size even if tracers don't exist. And why is ocean_bio_all full size and why aren't we referencing the bio_index arrays in icepack_load_ocean_bio? Can we get rid of the bio_index arrays completely and reuse some of the other indexing we already have. This issue was brought up quickly in update icepack calls in CICE to add keywords CICE#378. Do we want to refactor this?

  • I almost feel like we need to redo all the tracer indexing. What I think might work is a datatype that defines the tracer names, their sizes maybe even in multiple dimensions, their start index in a given array, and maybe even some properties. We could allow users to instantiate a few different versions (physics, bgc, or all) as needed and would provide a unified way to reference them. We could implement set and get methods so you could query the start index of "bgc_N" for instance. Unfortunately, I don't think we can take this on right now nor whether it would work well or be even more painful. One thing I do know is that if we constantly have to query for indices, the code will probably slow down a bit and that's a bad thing. In lieu of that, I'd love to unify around a set of indices that we can clearly document. We have the "nt" indexing, the "nlt" indexing, the bio_index, and the depend/strata which is yet another set of indexing (or at least adds to it). Again, is there some way to unify or eliminate some of this indexing. Can someone explain why the bio_index arrays as well as the depend/strata stuff is needed.

  • We currently do not store nslyr, nilyr, nblyr, or ncat inside Icepack. These are passed in on each icepack call that needs them. Should these be stored inside icepack as well like other tracer indexes and sizes?

Copy link
Contributor

@eclare108213 eclare108213 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some of the tracers are required, and so I think they should not be initialized to 0. E.g. Tsfc is assumed to be the first tracer, so nt_Tsfc = 1. Likewise qice and qsno are required. Aren't there also indices for ice and snow volume? They don't seem to be listed in icepack_tracers.F90...

@eclare108213
Copy link
Contributor

Would love some feedback whether this is heading in the correct direction.

This is definitely heading in the right direction. I looked through all the code changes at this point, and entered the only thing I noticed that might need to be looked at again in the review above.

I'll have to look at the code more closely to understand the details, in order to answer your specific questions, but here are some initial thoughts.

  • We should review trcrn_bgcsw in icepack_shortwave for the indices. In particular, use of nbtrcr_sw seems like a bad implementation. This is the last index and does introduce a lot of risk if the order of the tracers changes. What is this tracer supposed to be. I know we might get away with this because it's only in skl mode, but still....

Can a slice be sent in, so that the indexing order doesn't matter? Or is the issue that more than one bgc tracer are used in the shortwave and sent together in one (bgc) array?

  • I have been looking at the "max" arguments in icepack_load_ocean_bio_array and am very confused about what should be passed and what's happening. Why are there two additional bio_index arrays for yet another indexing strategy.

The 'bio grid' includes two extra grid points, to handle boundary conditions at the top and bottom, and so entails some special indexing. This is awkward but is documented. I don't think the extra bio_index arrays are absolutely necessary, but they are convenient. Most likely, what happened is that there are too many bgc tracers to send slices for each through the interface, individually, and so the whole tracer array was sent and the bgc tracers copied out of it when needed. But the whole tracer array is too much and so a subset was indexed...

And why are those allocated to absolute full size even if tracers don't exist. And why is ocean_bio_all full size

My guess is it was just convenience at the time.

and why aren't we referencing the bio_index arrays in icepack_load_ocean_bio? Can we get rid of the bio_index arrays completely and reuse some of the other indexing we already have. This issue was brought up quickly in CICE-Consortium/CICE#378. Do we want to refactor this?

I think the answer is yes, we do want to fix this. The question is how and when. Would a reasonable compromise be to send the bgc tracers through, as a group? That would entail some bio indexing, but maybe it would be clearer?

  • I almost feel like we need to redo all the tracer indexing. What I think might work is a datatype that defines the tracer names, their sizes maybe even in multiple dimensions, their start index in a given array, and maybe even some properties. We could allow users to instantiate a few different versions (physics, bgc, or all) as needed and would provide a unified way to reference them. We could implement set and get methods so you could query the start index of "bgc_N" for instance. Unfortunately, I don't think we can take this on right now nor whether it would work well or be even more painful.

Something like this would be nice, but I tend to agree that it may be more than we can do right now. Isn't the tracer get/set stuff sort of like a datatype? We'd still have to query it to get what we need?

One thing I do know is that if we constantly have to query for indices, the code will probably slow down a bit and that's a bad thing. In lieu of that, I'd love to unify around a set of indices that we can clearly document. We have the "nt" indexing, the "nlt" indexing, the bio_index, and the depend/strata which is yet another set of indexing (or at least adds to it). Again, is there some way to unify or eliminate some of this indexing. Can someone explain why the bio_index arrays as well as the depend/strata stuff is needed.

I can explain this, and in fact this might be useful for the tutorial. I already have some slides that explain how the tracers work (we have depend/strata etc because we have tracers that are tracers on other tracers rather than on the bulk ice or snow). I suspect that some of the extra indexing can be unified but maybe not completely eliminated.

  • We currently do not store nslyr, nilyr, nblyr, or ncat inside Icepack. These are passed in on each icepack call that needs them. Should these be stored inside icepack as well like other tracer indexes and sizes?

It makes sense to me to let Icepack own those too, since they are so fundamental to Icepack's parameterizations. I think the original argument for passing them in was that they were set by the user (cpp or namelist), and Icepack doesn't have I/O.

@apcraig
Copy link
Contributor Author

apcraig commented Nov 25, 2019

Thanks @eclare108213.

Regarding defaults set in the declaration of icepack_tracers. Currently, things are undefined which I think is a little worse. As currently implemented, the driver has to initialize all values, so by adding these defaults, at least we make things a little more predictable if a user forgets to set some. If there are requirements like nt_Tsfc needs to be 1, then maybe we need to set that as the default and then NOT let users reset it. As it stands, users have to know to set it to 1. My hope is that it's just required, but not necessarily as the first tracer. It would be good to add some checks on the required tracers, to check that different tracers aren't given the same index, and so forth.

Regarding the tracers depend/strata stuff, my sense is that this is something that is based on the implementation of icepack. It's icepack that is defining these dependencies yet the user has to understand them and then initialize the tracer depend/strata arrays. Do I have that right? This seems a little backwards. I am now thinking we need a new interface. Something like

subroutine icepack_tracer_index_init(names,numbers,tracer_depend/strata/etc)

where names is a 1d array of character strings like ('Tsfc','qice','qsno',bgc_N1','bgc_N2',...) where those strings are limited to tracer names the code understands and would be documented. numbers is also a 1d array of the same size that defines each tracers number of tracers. Separately, we still need users to define the n_ parameters like n_algae (or we could have users define the n_ parameters first and not pass the numbers array if that could work). Anyway,from this new interface, the nt_ and nlt_ indexes can be set as well as the tracer_depend/strata arrays. Icepack should be the one defining those arrays as those may change over time as new tracers or interactions within Icepack change. It's dicey to have users have to define those tracer depend/strata arrays manually with little other checking after that. We could continue to support the current method, but try to migrate to the new implementation which provides better and more robust use.

Separately, I will try to continue to look at the bio_index arrays to see if I can understand them as well as possibly propose some refactoring.

Are bgc and bio the same thing? If so, I would like to move toward a uniform name if we can. If bgc is everything and bio is a subset, that's fine too. I guess I just need clarification.

@apcraig
Copy link
Contributor Author

apcraig commented Nov 25, 2019

Just a quick comment. I have confirmed that if I reset nblyr to 7 inside Icepack, but just pass in the current slices, I recover the results for the bgcsklNICE case as we had previously bit-for-bit. That means the current implementation was getting the wrong results for bgcsklNICE due to incorrect nblyr setting in the cpps. So that answer change is correct.

@phil-blain
Copy link
Member

I've not look in details at the code changes (I don't know this area of the code, though I can take a look).

I have more of a question :
If I understand well there are still some preprocessor macros in Icepack when it is compiled in standalone mode ? (altough when Icepack is used in CICE these are namelist parameters ?).
Why are they not also namelist parameters in Icepack ?

@eclare108213
Copy link
Contributor

Tsfc used to be required for the thermodynamics, but we've added an option that allows the code to run without thermodynamics, and so in that case it might not be required. I always had it as the first tracer, and I don't know whether that is a requirement any more. My guess is that it is not, because we use the nt_Tsfc index. Could try a run with it somewhere in the middle...

The idea with the depend/strata arrays is that users define which tracers they want, and then the code sets them up. I agree, this could be done within Icepack with set/get types of functionality. Part of the reason for having it in the host model is that there are different approaches, e.g. age can be a function of only the concentration, or it could be a function of the full vertical column i.e. volume, if someone wanted to track new ice growth in the column. This option isn't currently supported in CICE or Icepack (the code to do it is commented out, I think), and if someone wanted to do it, it could be defined as a new tracer. Anyhow, I (almost) always know the background "why"... and we don't have to keep things the way they are. Letting Icepack own all tracer properties makes sense to me.

BGC and bio are similar, but I consider bio a subset of BGC. E.g. BGC includes dust and black carbon and other things that aren't necessarily biological (but might be biologically relevant). To me, algae, DMS, etc are "bio" but aerosols in general are BGC. I'd prefer to label them all as BGC. I think "bio" is @njeffery's shorthand for biogeochemistry, but she can weigh in with her definition.

Thanks for the bgcsklNICE bug fix, @apcraig.

@phil-blain, the Icepack driver and CICE driver are not entirely up to date with each other. We try, but this is a definite downside to having two separate repositories with their own drivers and testing infrastructure. I can see how this could be confusing in the documentation, so we should probably make an issue to flag it for cleanup.

@apcraig
Copy link
Contributor Author

apcraig commented Nov 26, 2019

Just to clarify about the depend/strata situation. Those arrays are set in the driver for use in Icepack, correct? But the way tracers depend on each other is based on the Icepack implementation (any maybe options turned on within)?

@eclare108213
Copy link
Contributor

That's mostly right, but the transport (incremental remapping IR) also uses the depend/strata information, and in fact, the tracer structure was designed for IR and we just use it in Icepack parameterizations. There are different dependencies based on namelist inputs. Icepack parameterizations provide the physical reasoning for the dependencies; the transport is agnostic about that. So while IR is the motivation for the tracer structure (and the reason it's still set in the cice driver), perhaps those definitions do belong in Icepack and would be 'safer' there.

Copy link
Contributor

@eclare108213 eclare108213 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments on variable names or their references:

nbtrcr_in -> n_aero_in -- I think this is okay as long as aerosols are the only thing being passed. However, the 'b' sometimes refers to the bio vertical grid, as opposed to the physical grid. To make things even more complicated, there are 2 aerosol schemes and the original one (from NCAR) just had aerosols on the delta-Eddington radiation grid (2 snow layers, 2 ice layers). So if the aero usage here isn't that general, maybe the b should stay. Or use z. I'm not saying that what you've done is wrong, just to be careful with aero stuff. And yes this complexity drives me a little crazy.

! algal C is not yet distinct from algal N -> ! bgc C is not yet distinct from bgc N -- change the comment back. (line 644 in icepack_tracers.F90)

Should doc/source/user_guide/lg_interfaces.rst instead be ug_interfaces.rst? Oh, I see, lg refers to library.

@apcraig apcraig merged commit 843ec99 into CICE-Consortium:master Dec 8, 2019
Copy link
Contributor

@eclare108213 eclare108213 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @apcraig
You've done a huge amount of work on this. Looking through all the changes, only a couple of things in icepack_tracers.F90 concern me.

line 486: nfsd_in refers to categories, not layers (comment)

Categories and layers (ncat, nilyr, nslyr, nfsd, etc) have traditionally been considered 'domain size', but the domain_size module is part of the driver, not Icepack. It seems like those would be more appropriately placed in icepack_tracer_sizes. What is your reasoning for putting them in icepack_tracer_indices?

@eclare108213
Copy link
Contributor

and my apologies for not noticing that earlier

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants