Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CaseClass can look beyond /glade/scratch/mlevy #34

Merged
merged 6 commits into from
Nov 20, 2020

Conversation

mnlevy1981
Copy link
Contributor

Added three new variables to CaseClass.__init__() API:

  1. run_root: treats {run_root}/{casename}/run as $RUNDIR (looks for log files
    and history files there)
  2. archive_hist_root: Looks for archived history files in
    {archive_hist_root}/{casename}/{component}/hist
  3. campaign_root: Looks for time series output in
    {campaign_root}/{casename}/output/{component}/proc/tseries

The default values for all three of these are None, which implies "don't even
look for files in these directories." Ideally a future update will provide the
user some configuration method for setting the defaults. Given these defaults,
every notebook needed to be updated to include at least one of those three
variables when creating the CaseClass object.

I also created two notebooks showing how we can look at the 1 degree run with
this new set-up. Currently we have history files for years 95-104 on campaign
so plot_suite_1deg.ipynb looks at that decade of output while
plot_suite_maps_0095_1deg.ipynb shows plots for the first year of data.

We now have 14 years of 004 output in campaign, so I updated
plot_suite_004.ipynb to look at all 14 years instead of just the first 5. I did
not modify trend_maps.004.ipynb or run plot_suite_maps for any additional
years of 004.

Added three new variables to CaseClass.__init__() API:

1. run_root: treats {run_root}/{casename}/run as $RUNDIR (looks for log files
   and history files there)
2. archive_hist_root: Looks for archived history files in
   {archive_hist_root}/{casename}/{component}/hist
3. campaign_root: Looks for time series output in
   {campaign_root}/{casename}/output/{component}/proc/tseries

The default values for all three of these are None, which implies "don't even
look for files in these directories." Ideally a future update will provide the
user some configuration method for setting the defaults. Given these defaults,
every notebook needed to be updated to include at least one of those three
variables when creating the CaseClass object.

I also created two notebooks showing how we can look at the 1 degree run with
this new set-up. Currently we have history files for years 95-104 on campaign
so plot_suite_1deg.ipynb looks at that decade of output while
plot_suite_maps_0095_1deg.ipynb shows plots for the first year of data.

We now have 14 years of 004 output in campaign, so I updated
plot_suite_004.ipynb to look at all 14 years instead of just the first 5. I did
not modify trend_maps.004.ipynb or run plot_suite_maps for any additional
years of 004.
001 and 002 need run_root otherwise we miss the days from runs that crashed
earlier commit only had a partial version of plot_suite_003.ipynb, and after
some discussion with Keith I don't think we need to carry around more than 1
plot_suite_maps notebook per case... so I kept

plot_suite_maps_0001_003.ipynb
plot_suite_maps_0001_004.ipynb
plot_suite_maps_0095_1deg.ipynb

And got rid of the other years from 003 and 004.
Remove run_root, archive_hist_root, and campaign_root in favor of output_roots
(a list). For each output_root in output_roots, look for log files in
{output_root} and {output_root}/logs. Also look for netCDF files in
{output_root}, {output_root}/{component}/hist, and
{output_root}/{component}/proc/tseries/{freq}. There is a single list of all
netCDF files, and then get_history_files() and get_timeseries_files() only
pulls the desired format of file out of the list.
1. Reverted back to CaseClass objects having self._timeseries_files and
   self._history_files, though both are now returned via self._find_nc_files()
2. Moved component_and_freq dict onto CaseClass object as self._stream_metadata
   (and also defined it in a more pythonic manner)
3. self._get_single_year_timeseries_files() requires varname; new
   self.check_for_year_in_timeseries_files() method can be used to determine if
   any time series files exist for a given stream / year
4. Removed os.isdir() from xmlquery wrapper -- subprocess returns FileNotFound
   error if caseroot does not exist
5. Refactored gen_output_roots_from_caseroot to only include DOUT_S_ROOT if
   DOUT_S=TRUE in the case
6. Cleaned up Sanity Check notebook, extended plot_suites_004 through year 0017
7. Improved several docstrings
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant