Update mass conservation tables to include Ref & Dev in the same file #309

yantosca · 2024-03-26T20:42:20Z

Name and Institution (Required)

Name: Bob Yantosca
Institution: Harvard + GCST

Confirm you have reviewed the following documentation

Contributing guidelines

Describe the update

This PR updates the mass conservation table benchmark code as follows:

A new module gcpy/benchmark/modules/benchmark_mass_conc_table.py has been created. This contains the code from the make_benchmark_mass_conservation_tables in the benchmark_funcs.py module. The code has been refactored for clarity. It now accepts arguments from both Ref and Dev versions.
Total masses and statistics from Ref and Dev model versions are now printed in the same file.
TransportTracers benchmark script run_1yr_tt_benchmark.py now imports routine make_benchmark_mass_conservation_table from benchmark_mass_conc_table.py.
The old make make_benchmark_mass_conservation_table has now been removed

Expected changes

Prior to this change, the mass conservation table looked like this:

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
  Global Mass of Passive Tracer in 14.2.0-rc.1/GCClassic/TransportTracers  
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

 Date        Mass [Tg]
 ----------  ----------------
 2019-01-01   17.6561658812290
 2019-02-01   17.6561659006875
 2019-03-01   17.6561658888524
 2019-04-01   17.6561659034459
 2019-05-01   17.6561658979651
 2019-06-01   17.6561658832719
 2019-07-01   17.6561658830644
 2019-08-01   17.6561658772005
 2019-09-01   17.6561658896849
 2019-10-01   17.6561658980134
 2019-11-01   17.6561658915458
 2019-12-01   17.6561658810218
 
 Summary
 ------------------------------
 Max mass =   17.6561659034459 Tg
 Min mass =   17.6561658772005 Tg
 Abs diff =         26245.367 g
 Pct diff =      0.0000001486 %

With the updates in this PR, it now looks like this:

===============================================================================
Global mass of PassiveTracer

Ref = gcc-14.2.0-rc.0
Dev = gcc-14.2.0-rc.2
===============================================================================

 Date & Time                  Ref mass [Tg]             Dev mass [Tg]
 -----------------     --------------------      --------------------
 2019-01-01 00:00          17.6561658812290          17.6561658812290
 2019-02-01 00:00          17.6561659006875          17.6561659006875
 2019-03-01 00:00          17.6561658888524          17.6561658888524
 2019-04-01 00:00          17.6561659034459          17.6561659034459
 2019-05-01 00:00          17.6561658979651          17.6561658979651
 2019-06-01 00:00          17.6561658832719          17.6561658832719
 2019-07-01 00:00          17.6561658830644          17.6561658830644
 2019-08-01 00:00          17.6561658772005          17.6561658772005
 2019-09-01 00:00          17.6561658896849          17.6561658896849
 2019-10-01 00:00          17.6561658980134          17.6561658980134
 2019-11-01 00:00          17.6561658915458          17.6561658915458
 2019-12-01 00:00          17.6561658810218          17.6561658810218
 
 Summary                                Ref                       Dev
 -----------------     --------------------      --------------------
 Maximum mass [Tg]         17.6561659034459          17.6561659034459
 Minimum mass [Tg]         17.6561658772005          17.6561658772005
 Abs diff [g]           26245.3667687623238       26245.3667687623238
 % difference               0.0000001486470           0.0000001486470

Reference(s)

If this is a science update, please provide a literature citation.

Related Github Issue(s)

This PR partially addresses Combine Ref and Dev in the same file for TransportTracers mass conservation tables #308.

gcpy/benchmark/modules/benchmark_mass_cons_table.py - New script for printing mass conservation table output (from the PassiveTracer species). Ref and Dev versions are now printed in the same table. - Code has been refactored for more clarity and efficiency. We now open all files into a common dataset and instead of opening one file at a time. Also split off common operations into local functions. gcpy/benchmark/modules/benchmark_funcs.py - Removed make_benchmark_mass_conservation_table, this has now been ported to benchmark_mass_cons_table.py CHANGELOG.md - Updated accordingly Signed-off-by: Bob Yantosca <[email protected]>

gcpy/benchmark/modules/benchmark_mass_cons_table.py - Bug fix: Test if the area variable has a time dimension before trying to use .isel(time=0) - Add extra calls to verify_variable_type gcpy/benchmark/modules/run_1yr_tt_benchmark.py - No longer import make_benchmark_mass_conservation_table from benchmark_funcs.py; this has been removed - Import make_benchmark_mass_conservation_table from the benchmark_mass_cons_table.py module - Now pass Ref & Dev arguments to make_benchmark_mass_conservation_table

yantosca · 2024-03-26T21:11:23Z

@lizziel @msulprizio: Also note I expanded the date tag to date & time so that you can do mass conservation tests for shorter intervals (1 month, 1 week) with archived estarts for every 6 or 12 hours if you wish.

gcpy/benchmark/modules/benchmark_mass_cons_table.py - Add Pydoc comments for get_passive_tracer_varname - Update comments for the time dimension Signed-off-by: Bob Yantosca <[email protected]>

lizziel · 2024-03-27T15:24:51Z

This is a very useful update! Is it possible to have a column for difference? This will help see if the difference goes up and down, or if there is a steady bias. We also sometimes see a jump between the first and second entries due to the restart file.

lizziel · 2024-03-27T15:35:26Z

Along these lines, if it is easy to do I think we should have metrics at the bottom for mean, variance, and difference between start and stop. Something like this (I added columns for difference and % difference too).

 Summary                                Ref                       Dev                Difference    % Difference
 -----------------     --------------------      --------------------      --------------------   --------------------


 Maximum mass [Tg]         17.6561659034459          17.6561659034459
 Minimum mass [Tg]         17.6561658772005          17.6561658772005
 Abs diff [g]           26245.3667687623238       26245.3667687623238
 % difference               0.0000001486470           0.0000001486470

 Start mass [Tg]            value here                value here
 End mass [Tg]              value here                value here
 Abs diff [g]               value here                value here
 % difference               value here                value here
 
 Mean mass [Tg]             value here                value here
 Variance [Tg]              value here                value here

yantosca · 2024-03-27T17:51:19Z

@lizziel: I will try to implement your suggestions. It should not be that difficult to do.

gcpy/benchmark/modules/benchmark_mass_cons_tables - Add extra fields to statistics output: - Start mass - End mass - Abs diff - % diff - Mean mass - Variance - Also add extra columns for Abs Diff & % Diff for each of these CHANGELOG.md - Updated accordingly Signed-off-by: Bob Yantosca <[email protected]>

yantosca · 2024-03-27T20:11:59Z

@lizziel: In commit 5c8df13 I added the extra statistics output you requested. The table now looks like:

===============================================================================
Global mass of PassiveTracer

Ref = alpha.9
Dev = PR_2072
===============================================================================

 Date & Time                  Ref mass [Tg]             Dev mass [Tg]
 -----------------     --------------------      --------------------
 2019-01-01 00:00          17.6563096646018          17.6563096646018
 2019-01-02 00:00          17.6563097334271          17.6563097334271
 2019-01-03 00:00          17.6563096472464          17.6563096472464
 2019-01-04 00:00          17.6563097381031          17.6563097381031
 2019-01-05 00:00          17.6563096420086          17.6563096420086
 2019-01-06 00:00          17.6563096769969          17.6563096769969
 2019-01-07 00:00          17.6563097517587          17.6563097517587
 2019-01-08 00:00          17.6563096589453          17.6563096589453
 2019-01-09 00:00          17.6563096255438          17.6563096255438
 2019-01-10 00:00          17.6563096669641          17.6563096669641
 2019-01-11 00:00          17.6563095824931          17.6563095824931
 2019-01-12 00:00          17.6563096810990          17.6563096810990
 2019-01-13 00:00          17.6563097035783          17.6563097035783
 2019-01-14 00:00          17.6563096224065          17.6563096224065
 2019-01-15 00:00          17.6563097281873          17.6563097281873
 2019-01-16 00:00          17.6563096448919          17.6563096448919
 2019-01-17 00:00          17.6563096999110          17.6563096999110
 2019-01-18 00:00          17.6563096911278          17.6563096911278
 2019-01-19 00:00          17.6563096524387          17.6563096524387
 2019-01-20 00:00          17.6563096730235          17.6563096730235
 2019-01-21 00:00          17.6563096880015          17.6563096880015
 2019-01-22 00:00          17.6563096931819          17.6563096931819
 2019-01-23 00:00          17.6563096625323          17.6563096625323
 2019-01-24 00:00          17.6563096789355          17.6563096789355
 2019-01-25 00:00          17.6563096582845          17.6563096582845
 2019-01-26 00:00          17.6563096609817          17.6563096609817
 2019-01-27 00:00          17.6563096687468          17.6563096687468
 2019-01-28 00:00          17.6563096712458          17.6563096712458
 2019-01-29 00:00          17.6563096699832          17.6563096699832
 2019-01-30 00:00          17.6563096648479          17.6563096648479
 2019-01-31 00:00          17.6563096684741          17.6563096684741
 2019-02-01 00:00          17.6563096665982          17.6563096665982
 
 Summary                                Ref                       Dev      Abs Diff  % Diff
 -----------------     --------------------      -------------------- ------------- -------
 Maximum mass [Tg]         17.6563097517587          17.6563097517587      0.000000   0.000
 Minimum mass [Tg]         17.6563095824931          17.6563095824931      0.000000   0.000 
 Abs diff [g]          169265.6539376002911      169265.6539376002911      0.000000   0.000
 % difference               0.0000009586695           0.0000009586695      0.000000   0.000

 Start mass [Tg]           17.6563096646018          17.6563096646018      0.000000   0.000
 End mass [Tg]             17.6563096665982          17.6563096665982      0.000000   0.000
 Abs diff [g]            1996.3231068231835        1996.3231068231835      0.000000   0.000
 % difference               0.0000000113066           0.0000000113066      0.000000   0.000

 Mean mass [Tg]            17.6563096730177          17.6563096730177      0.000000   0.000
 Variance [Tg]              0.0000000000000           0.0000000000000      0.000000   0.000

gcpy/benchmark/modules/benchmark_mass_cons_table.py - Added "Abs Diff" & "% Diff" columns to the list of Ref & Dev total masses CHANGELOG.md - Updated accordingly Signed-off-by: Bob Yantosca <[email protected]>

yantosca · 2024-03-29T18:45:17Z

@lizziel: I added abs diff & % diff columns as such in 0c3bca7:

===============================================================================
Global mass of PassiveTracer

Ref = alpha.9
Dev = PR_2072
===============================================================================

 Date & Time                  Ref mass [Tg]             Dev mass [Tg]      Abs Diff  % Diff
 -----------------     --------------------      -------------------- ------------- -------
 2019-01-01 00:00          17.6563096646018          17.6563096646018      0.000000   0.000
 2019-01-02 00:00          17.6563097334271          17.6563097334271      0.000000   0.000
 2019-01-03 00:00          17.6563096472464          17.6563096472464      0.000000   0.000
 2019-01-04 00:00          17.6563097381031          17.6563097381031      0.000000   0.000
 2019-01-05 00:00          17.6563096420086          17.6563096420086      0.000000   0.000
 2019-01-06 00:00          17.6563096769969          17.6563096769969      0.000000   0.000
 2019-01-07 00:00          17.6563097517587          17.6563097517587      0.000000   0.000
 2019-01-08 00:00          17.6563096589453          17.6563096589453      0.000000   0.000
 2019-01-09 00:00          17.6563096255438          17.6563096255438      0.000000   0.000
 2019-01-10 00:00          17.6563096669641          17.6563096669641      0.000000   0.000
 2019-01-11 00:00          17.6563095824931          17.6563095824931      0.000000   0.000
 2019-01-12 00:00          17.6563096810990          17.6563096810990      0.000000   0.000
 2019-01-13 00:00          17.6563097035783          17.6563097035783      0.000000   0.000
 2019-01-14 00:00          17.6563096224065          17.6563096224065      0.000000   0.000
 2019-01-15 00:00          17.6563097281873          17.6563097281873      0.000000   0.000
 2019-01-16 00:00          17.6563096448919          17.6563096448919      0.000000   0.000
 2019-01-17 00:00          17.6563096999110          17.6563096999110      0.000000   0.000
 2019-01-18 00:00          17.6563096911278          17.6563096911278      0.000000   0.000
 2019-01-19 00:00          17.6563096524387          17.6563096524387      0.000000   0.000
 2019-01-20 00:00          17.6563096730235          17.6563096730235      0.000000   0.000
 2019-01-21 00:00          17.6563096880015          17.6563096880015      0.000000   0.000
 2019-01-22 00:00          17.6563096931819          17.6563096931819      0.000000   0.000
 2019-01-23 00:00          17.6563096625323          17.6563096625323      0.000000   0.000
 2019-01-24 00:00          17.6563096789355          17.6563096789355      0.000000   0.000
 2019-01-25 00:00          17.6563096582845          17.6563096582845      0.000000   0.000
 2019-01-26 00:00          17.6563096609817          17.6563096609817      0.000000   0.000
 2019-01-27 00:00          17.6563096687468          17.6563096687468      0.000000   0.000
 2019-01-28 00:00          17.6563096712458          17.6563096712458      0.000000   0.000
 2019-01-29 00:00          17.6563096699832          17.6563096699832      0.000000   0.000
 2019-01-30 00:00          17.6563096648479          17.6563096648479      0.000000   0.000
 2019-01-31 00:00          17.6563096684741          17.6563096684741      0.000000   0.000
 2019-02-01 00:00          17.6563096665982          17.6563096665982      0.000000   0.000
 
 Summary                                Ref                       Dev      Abs Diff  % Diff
 -----------------     --------------------      -------------------- ------------- -------
 Maximum mass [Tg]         17.6563097517587          17.6563097517587      0.000000   0.000
 Minimum mass [Tg]         17.6563095824931          17.6563095824931      0.000000   0.000 
 Abs diff [g]          169265.6539376002911      169265.6539376002911      0.000000   0.000
 % difference               0.0000009586695           0.0000009586695      0.000000   0.000

 Start mass [Tg]           17.6563096646018          17.6563096646018      0.000000   0.000
 End mass [Tg]             17.6563096665982          17.6563096665982      0.000000   0.000
 Abs diff [g]            1996.3231068231835        1996.3231068231835      0.000000   0.000
 % difference               0.0000000113066           0.0000000113066      0.000000   0.000

 Mean mass [Tg]            17.6563096730177          17.6563096730177      0.000000   0.000
 Variance [Tg]              0.0000000000000           0.0000000000000      0.000000   0.000

lizziel · 2024-03-29T19:04:38Z

Thanks @yantosca. We might need to increase precision on the diff column to capture the actual differences. Does it expand automatically if diffs are non-zero?

lizziel · 2024-03-29T19:05:48Z

I also wonder why the variance is zero. Is there a mistake there?

yantosca · 2024-03-29T19:18:40Z

@lizziel: I used np.var to compute the variance. Maybe I used the wrong argument to that. Let me check.

gcpy/benchmark/modules/benchmark_mass_cons_table.py - Now compute mean_mass and variance with dtype=np.float64 - Use 13.4e right-adjusted format for "Abs Diff" column - Use 8.3f right-adjusted format for "% Diff" column - Update underlines accordingly - Add an extra space between "Abs Diff" and "% Diff" column - Now display variance with 20.13e right-adjusted format, since it can potentially be very small Signed-off-by: Bob Yantosca <[email protected]>

yantosca · 2024-03-29T19:51:34Z

@lizziel: I've now made the abs diff in scientific notation, since those numbers may be very small. Also the variance was very small so that's why it was showing up as zero. Changing it to scientific notation fixes that. See commit 486185b.

===============================================================================
Global mass of PassiveTracer

Ref = alpha.9
Dev = PR_2072
===============================================================================

 Date & Time                  Ref mass [Tg]             Dev mass [Tg]      Abs Diff    % Diff
 -----------------     --------------------      -------------------- -------------  --------
 2019-01-01 00:00          17.6563096646018          17.6563096646018    0.0000e+00     0.000
 2019-01-02 00:00          17.6563097334271          17.6563097334271    0.0000e+00     0.000
 2019-01-03 00:00          17.6563096472464          17.6563096472464    0.0000e+00     0.000
 2019-01-04 00:00          17.6563097381031          17.6563097381031    0.0000e+00     0.000
 2019-01-05 00:00          17.6563096420086          17.6563096420086    0.0000e+00     0.000
 2019-01-06 00:00          17.6563096769969          17.6563096769969    0.0000e+00     0.000
 2019-01-07 00:00          17.6563097517587          17.6563097517587    0.0000e+00     0.000
 2019-01-08 00:00          17.6563096589453          17.6563096589453    0.0000e+00     0.000
 2019-01-09 00:00          17.6563096255438          17.6563096255438    0.0000e+00     0.000
 2019-01-10 00:00          17.6563096669641          17.6563096669641    0.0000e+00     0.000
 2019-01-11 00:00          17.6563095824931          17.6563095824931    0.0000e+00     0.000
 2019-01-12 00:00          17.6563096810990          17.6563096810990    0.0000e+00     0.000
 2019-01-13 00:00          17.6563097035783          17.6563097035783    0.0000e+00     0.000
 2019-01-14 00:00          17.6563096224065          17.6563096224065    0.0000e+00     0.000
 2019-01-15 00:00          17.6563097281873          17.6563097281873    0.0000e+00     0.000
 2019-01-16 00:00          17.6563096448919          17.6563096448919    0.0000e+00     0.000
 2019-01-17 00:00          17.6563096999110          17.6563096999110    0.0000e+00     0.000
 2019-01-18 00:00          17.6563096911278          17.6563096911278    0.0000e+00     0.000
 2019-01-19 00:00          17.6563096524387          17.6563096524387    0.0000e+00     0.000
 2019-01-20 00:00          17.6563096730235          17.6563096730235    0.0000e+00     0.000
 2019-01-21 00:00          17.6563096880015          17.6563096880015    0.0000e+00     0.000
 2019-01-22 00:00          17.6563096931819          17.6563096931819    0.0000e+00     0.000
 2019-01-23 00:00          17.6563096625323          17.6563096625323    0.0000e+00     0.000
 2019-01-24 00:00          17.6563096789355          17.6563096789355    0.0000e+00     0.000
 2019-01-25 00:00          17.6563096582845          17.6563096582845    0.0000e+00     0.000
 2019-01-26 00:00          17.6563096609817          17.6563096609817    0.0000e+00     0.000
 2019-01-27 00:00          17.6563096687468          17.6563096687468    0.0000e+00     0.000
 2019-01-28 00:00          17.6563096712458          17.6563096712458    0.0000e+00     0.000
 2019-01-29 00:00          17.6563096699832          17.6563096699832    0.0000e+00     0.000
 2019-01-30 00:00          17.6563096648479          17.6563096648479    0.0000e+00     0.000
 2019-01-31 00:00          17.6563096684741          17.6563096684741    0.0000e+00     0.000
 2019-02-01 00:00          17.6563096665982          17.6563096665982    0.0000e+00     0.000
 
 Summary                                Ref                       Dev      Abs Diff    % Diff
 -----------------     --------------------      -------------------- -------------  --------
 Maximum mass [Tg]         17.6563097517587          17.6563097517587    0.0000e+00     0.000
 Minimum mass [Tg]         17.6563095824931          17.6563095824931    0.0000e+00     0.000
 Abs diff [g]          169265.6539376002911      169265.6539376002911    0.0000e+00     0.000
 % difference               0.0000009586695           0.0000009586695    0.0000e+00     0.000

 Start mass [Tg]           17.6563096646018          17.6563096646018    0.0000e+00     0.000
 End mass [Tg]             17.6563096665982          17.6563096665982    0.0000e+00     0.000
 Abs diff [g]            1996.3231068231835        1996.3231068231835    0.0000e+00     0.000
 % difference               0.0000000113066           0.0000000113066    0.0000e+00     0.000

 Mean mass [Tg]            17.6563096730177          17.6563096730177    0.0000e+00     0.000
 Variance [Tg]          1.1452659234774e-15       1.1452659234774e-15    0.0000e+00     0.000

gcpy/benchmark/modules/benchmark_mass_cons_table.py - Increased the formatting from 20.13f to 20.15f for most entries in the 1st 2 columns. Signed-off-by: Bob Yantosca <[email protected]>

lizziel · 2024-04-10T21:09:04Z

I just tested this out with some recent 14.3.1 GCHP data courtesy of @yuanjianz. This is what I see.

Here are my thoughts on the format:

I think the mass in Tg should have 12 digits beyond the decimal, so equivalent to whole numbers of grams
Abs difference could be in whole grams (no decimal) and aligned right
Both % difference and variance in all places could be scientific notation with 4 digits past the decimal for consistency

Aside: if you are wondering why there is such a huge jump after the start.... I think it's because the run uses a 2019 restart file to start in 2022.

Also note that the times printed to the file are the file times and not the times in the filename. This is a limitation that could be addressed later on.

@lizziel

gcpy/benchmark/modules/benchmark_mass_cons_table.py - Implemented the following suggestiongs from @lizziel on PR #309: - Now use 12 decimal places for Ref & Dev columns (floating point) - Now use 4 decimal places for Abs Diff & Diff (exponential notation) - Absolute difference in grames is now reported as an integer - Lined up column labels accordingly Signed-off-by: Bob Yantosca <[email protected]>

yantosca · 2024-04-11T14:43:30Z

Thanks @lizziel for the great suggestions. I think the table is looking really nice now!

===============================================================================
Global mass of PassiveTracer

Ref = gcc-14.2.0-rc.2
Dev = gchp-14.2.0-rc.2
===============================================================================

 Date & Time               Ref mass [Tg]        Dev mass [Tg]      Abs Diff         % Diff
 -----------------     -----------------    ----------------- -------------  -------------
 2019-01-01 00:00        17.656165881229      17.656280192674    1.1431e-04     6.4743e-04
 2019-02-01 00:00        17.656165900687      17.656119120463   -4.6780e-05    -2.6495e-04
 2019-03-01 00:00        17.656165888852      17.656119334500   -4.6554e-05    -2.6367e-04
 2019-04-01 00:00        17.656165903446      17.656119237693   -4.6666e-05    -2.6430e-04
 2019-05-01 00:00        17.656165897965      17.656119239163   -4.6659e-05    -2.6426e-04
 2019-06-01 00:00        17.656165883272      17.656119120917   -4.6762e-05    -2.6485e-04
 2019-07-01 00:00        17.656165883064      17.656119294423   -4.6589e-05    -2.6387e-04
 2019-08-01 00:00        17.656165877200      17.656119153148   -4.6724e-05    -2.6463e-04
 2019-09-01 00:00        17.656165889685      17.656119076444   -4.6813e-05    -2.6514e-04
 2019-10-01 00:00        17.656165898013      17.656119174781   -4.6723e-05    -2.6463e-04
 2019-11-01 00:00        17.656165891546      17.656119148337   -4.6743e-05    -2.6474e-04
 2019-12-01 00:00        17.656165881022      17.656119245409   -4.6636e-05    -2.6413e-04
 
 Summary                             Ref                  Dev      Abs Diff         % Diff
 -----------------     -----------------    ----------------- -------------  -------------
 Maximum mass [Tg]       17.656165903446      17.656280192674    1.1429e-04     6.4730e-04
 Minimum mass [Tg]       17.656165877200      17.656119076444   -4.6801e-05    -2.6507e-04
 Abs diff [g]                      26245            161116229    1.6109e+08     6.1378e+05
 % difference             0.000000148647       0.000912523463    9.1237e-04     6.1379e+05

 Start mass [Tg]         17.656165881229      17.656280192674    1.1431e-04     6.4743e-04
 End mass [Tg]           17.656165881022      17.656119245409    1.1431e-04     6.4743e-04
 Abs diff [g]                       -207           -160947264   -1.6095e+08     7.7677e+07
 % difference            -0.000000001174      -0.000911558172   -9.1156e-04     7.7676e+07

 Mean mass [Tg]          17.656165889665      17.656132611496   -3.3278e-05    -1.8848e-04
 Variance [Tg]        6.998878437967e-17   1.980023970141e-09    1.9800e-09     2.8291e+09

gcpy/benchmark/modules/benchmark_utils.py - Added function get_datetimes_from_filenames to return an np.ndarray of np.datetime64 values. This is needed because often the initial restart file may have a different internal timestamp than the starting date of the simulation. gcpy/benchmark/modules/benchmark_mass_cons_table.py - Now use get_datetimes_from_filenames to define the ref_time and dev_time variables. CHANGELOG.md - Updated accordingly Signed-off-by: Bob Yantosca <[email protected]>

yantosca · 2024-04-11T15:29:09Z

@lizziel: We now get the datetimes from the restart file names in commit b889d2e,

gcpy/util.py - Add utility function "replace_whitespace" to replace whitespace characters with another character (underscore is default) gcpy/benchmark/modules/benchmark_mass_cons_table.py - Call replace_whitespace from util.py to replace spaces in version labels and the output file name. CHANGELOG.md - Updated accordingly Signed-off-by: Bob Yantosca <[email protected]>

yantosca · 2024-04-22T20:49:19Z

@lizziel: I added a function replace_whitespace to util.py that will replace any whitespace character with a supplied character (_ is default). Then I added spaces to the version labels:

data:
  ref:
    gcc:
      version: gcc 14.2.0 rc 0
      ...
    gchp:
      version: gchp 14.2.0 rc 0
  ...
  dev:
    gcc:
      version: gcc 14.2.0 rc 2
      ...
    gchp:
      version: gchp 14.2.0 rc 2
       ...

The code generated a file Passive_mass.gcc_14.2.0_rc_0_vs_gcc_14.2.0_rc_2.txt that looks like this:

===============================================================================
Global mass of PassiveTracer

Ref = gcc_14.2.0_rc_0
Dev = gcc_14.2.0_rc_2
===============================================================================

 Date & Time               Ref mass [Tg]        Dev mass [Tg]      Abs Diff         % Diff
 -----------------     -----------------    ----------------- -------------  -------------
 2019-01-01 00:00        17.656165881229      17.656165881229    0.0000e+00     0.0000e+00
 2019-02-01 00:00        17.656165900687      17.656165900687    0.0000e+00     0.0000e+00
 2019-03-01 00:00        17.656165888852      17.656165888852    0.0000e+00     0.0000e+00
 2019-04-01 00:00        17.656165903446      17.656165903446    0.0000e+00     0.0000e+00
 2019-05-01 00:00        17.656165897965      17.656165897965    0.0000e+00     0.0000e+00
 2019-06-01 00:00        17.656165883272      17.656165883272    0.0000e+00     0.0000e+00
 2019-07-01 00:00        17.656165883064      17.656165883064    0.0000e+00     0.0000e+00
 2019-08-01 00:00        17.656165877200      17.656165877200    0.0000e+00     0.0000e+00
 2019-09-01 00:00        17.656165889685      17.656165889685    0.0000e+00     0.0000e+00
 2019-10-01 00:00        17.656165898013      17.656165898013    0.0000e+00     0.0000e+00
 2019-11-01 00:00        17.656165891546      17.656165891546    0.0000e+00     0.0000e+00
 2019-12-01 00:00        17.656165881022      17.656165881022    0.0000e+00     0.0000e+00
 
 Summary                             Ref                  Dev      Abs Diff         % Diff
 -----------------     -----------------    ----------------- -------------  -------------
 Maximum mass [Tg]       17.656165903446      17.656165903446    0.0000e+00     0.0000e+00
 Minimum mass [Tg]       17.656165877200      17.656165877200    0.0000e+00     0.0000e+00
 Abs diff [g]                      26245                26245    0.0000e+00     0.0000e+00
 % difference             0.000000148647       0.000000148647    0.0000e+00     0.0000e+00

 Start mass [Tg]         17.656165881229      17.656165881229    0.0000e+00     0.0000e+00
 End mass [Tg]           17.656165881022      17.656165881022    0.0000e+00     0.0000e+00
 Abs diff [g]                       -207                 -207    0.0000e+00    -0.0000e+00
 % difference            -0.000000001174      -0.000000001174    0.0000e+00    -0.0000e+00

 Mean mass [Tg]          17.656165889665      17.656165889665    0.0000e+00     0.0000e+00
 Variance [Tg]        6.998878437967e-17   6.998878437967e-17    0.0000e+00     0.0000e+00

So this will make sure the generated file doesn't have spaces in it.

lizziel · 2024-04-22T21:21:33Z

This looks great. I think it is good to go except that I couldn't get it to work for GCHP vs GCC tables. I'll troubleshoot that with my data and let you know what I find.

yantosca · 2024-04-22T21:51:24Z

Thanks @lizziel. For what it's worth, I was able to get GCHP vs GCC output with my sample data:

===============================================================================
Global mass of PassiveTracer

Ref = gcc_14.2.0_rc_2
Dev = gchp_14.2.0_rc_2
===============================================================================

 Date & Time               Ref mass [Tg]        Dev mass [Tg]      Abs Diff         % Diff
 -----------------     -----------------    ----------------- -------------  -------------
 2019-01-01 00:00        17.656165881229      17.656280192674    1.1431e-04     6.4743e-04
 2019-02-01 00:00        17.656165900687      17.656119120463   -4.6780e-05    -2.6495e-04
 2019-03-01 00:00        17.656165888852      17.656119334500   -4.6554e-05    -2.6367e-04
 2019-04-01 00:00        17.656165903446      17.656119237693   -4.6666e-05    -2.6430e-04
 2019-05-01 00:00        17.656165897965      17.656119239163   -4.6659e-05    -2.6426e-04
 2019-06-01 00:00        17.656165883272      17.656119120917   -4.6762e-05    -2.6485e-04
 2019-07-01 00:00        17.656165883064      17.656119294423   -4.6589e-05    -2.6387e-04
 2019-08-01 00:00        17.656165877200      17.656119153148   -4.6724e-05    -2.6463e-04
 2019-09-01 00:00        17.656165889685      17.656119076444   -4.6813e-05    -2.6514e-04
 2019-10-01 00:00        17.656165898013      17.656119174781   -4.6723e-05    -2.6463e-04
 2019-11-01 00:00        17.656165891546      17.656119148337   -4.6743e-05    -2.6474e-04
 2019-12-01 00:00        17.656165881022      17.656119245409   -4.6636e-05    -2.6413e-04
 
 Summary                             Ref                  Dev      Abs Diff         % Diff
 -----------------     -----------------    ----------------- -------------  -------------
 Maximum mass [Tg]       17.656165903446      17.656280192674    1.1429e-04     6.4730e-04
 Minimum mass [Tg]       17.656165877200      17.656119076444   -4.6801e-05    -2.6507e-04
 Abs diff [g]                      26245            161116229    1.6109e+08     6.1378e+05
 % difference             0.000000148647       0.000912523463    9.1237e-04     6.1379e+05

 Start mass [Tg]         17.656165881229      17.656280192674    1.1431e-04     6.4743e-04
 End mass [Tg]           17.656165881022      17.656119245409    1.1431e-04     6.4743e-04
 Abs diff [g]                       -207           -160947264   -1.6095e+08     7.7677e+07
 % difference            -0.000000001174      -0.000911558172   -9.1156e-04     7.7676e+07

 Mean mass [Tg]          17.656165889665      17.656132611496   -3.3278e-05    -1.8848e-04
 Variance [Tg]        6.998878437967e-17   1.980023970141e-09    1.9800e-09     2.8291e+09

lizziel · 2024-04-23T14:08:52Z

I tried again to create the table comparing GC-Classic 4x5 and GCHP C180 output. I get this warning and no table is produced:

PerformanceWarning: \
Slicing is producing a large chunk. To accept the large
chunk and silence this warning, set the option
    >>> with dask.config.set(**{'array.slicing.split_large_chunks': False}):
    ...     array[indexer]

To avoid creating the large chunks, set the option
    >>> with dask.config.set(**{'array.slicing.split_large_chunks': True}):
    ...     array[indexer]
  value = value[(slice(None),) * axis + (subkey,)]

I am able to create the separate ref and dev tables using the GCPy dev branch so this must be something new in this branch.

yantosca · 2024-04-23T14:11:35Z

@lizziel: This must be an under-the-hood xarray/dask issue due to the large size of the data. I only tested with c24 and that works OK. I"ll see if there is a published workaround.

gcpy/benchmark/modules/benchmark_mass_cons_table.py - Import the dask.config class as "dask_config" - Add a call to dask_config in the same "with" blocks where we tell xarray to keep all variable/global attributes Signed-off-by: Bob Yantosca <[email protected]>

yantosca · 2024-04-23T14:26:48Z

@lizziel: I pushed a fix in commit df1e984. Please grab this update and let me know if it works. Thanks!

lizziel · 2024-04-23T14:55:11Z

Using the updated code results in the program being killed, even when I up the memory 160G.

yantosca · 2024-04-23T15:02:00Z

@lizziel found that the problem may be due to reading all files up-front, which takes too much memory. We can try looping over each file and then purging the memory afterwards.

gcpy/benchmark/modules/benchmark_mass_cons_table.py - Refactored code so that we read one file at a time in order to avoid memory issues when reading large files (e.g. c180 resolution). - Delete objects at the end of the loop over times to force garbage collection. - Remove references to dask_config, it's not needed. Signed-off-by: Bob Yantosca <[email protected]>

yantosca · 2024-04-23T15:43:08Z

@lizziel: I modified the code so that we now loop over individual files (and delete objects afterwards to force garbage collection) in commit af8ef64. Try it again and let me know if it works.

yantosca added 2 commits March 26, 2024 15:16

yantosca added category: Feature Request New feature or request topic: Benchmark Plots and Tables Issues pertaining to generating plots/tables from benchmark output labels Mar 26, 2024

yantosca added this to the 1.5.0 milestone Mar 26, 2024

yantosca assigned lizziel and yantosca Mar 26, 2024

yantosca requested a review from lizziel March 26, 2024 21:09

Update comments for clarity in benchmark_mass_cons_table.py

b2a9577

gcpy/benchmark/modules/benchmark_mass_cons_table.py - Add Pydoc comments for get_passive_tracer_varname - Update comments for the time dimension Signed-off-by: Bob Yantosca <[email protected]>

Add "Abs Diff" & "% Diff" columns in mass conservation table

0c3bca7

gcpy/benchmark/modules/benchmark_mass_cons_table.py - Added "Abs Diff" & "% Diff" columns to the list of Ref & Dev total masses CHANGELOG.md - Updated accordingly Signed-off-by: Bob Yantosca <[email protected]>

yantosca mentioned this pull request Mar 29, 2024

[FEATURE REQUEST] Transport tracer benchmark improvements #288

Open

Now display up to 15 digits in the mass conservation table

12107b6

gcpy/benchmark/modules/benchmark_mass_cons_table.py - Increased the formatting from 20.13f to 20.15f for most entries in the 1st 2 columns. Signed-off-by: Bob Yantosca <[email protected]>

yantosca merged commit 936ce78 into dev May 15, 2024

yantosca deleted the feature/update-mass-cons-table branch May 23, 2024 21:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update mass conservation tables to include Ref & Dev in the same file #309

Update mass conservation tables to include Ref & Dev in the same file #309

yantosca commented Mar 26, 2024

yantosca commented Mar 26, 2024

lizziel commented Mar 27, 2024

lizziel commented Mar 27, 2024 •

edited

Loading

yantosca commented Mar 27, 2024

yantosca commented Mar 27, 2024

yantosca commented Mar 29, 2024

lizziel commented Mar 29, 2024

lizziel commented Mar 29, 2024

yantosca commented Mar 29, 2024

yantosca commented Mar 29, 2024

lizziel commented Apr 10, 2024 •

edited

Loading

yantosca commented Apr 11, 2024

yantosca commented Apr 11, 2024

yantosca commented Apr 22, 2024

lizziel commented Apr 22, 2024

yantosca commented Apr 22, 2024

lizziel commented Apr 23, 2024

yantosca commented Apr 23, 2024

yantosca commented Apr 23, 2024

lizziel commented Apr 23, 2024

yantosca commented Apr 23, 2024

yantosca commented Apr 23, 2024

Update mass conservation tables to include Ref & Dev in the same file #309

Update mass conservation tables to include Ref & Dev in the same file #309

Conversation

yantosca commented Mar 26, 2024

Name and Institution (Required)

Confirm you have reviewed the following documentation

Describe the update

Expected changes

Reference(s)

Related Github Issue(s)

yantosca commented Mar 26, 2024

lizziel commented Mar 27, 2024

lizziel commented Mar 27, 2024 • edited Loading

yantosca commented Mar 27, 2024

yantosca commented Mar 27, 2024

yantosca commented Mar 29, 2024

lizziel commented Mar 29, 2024

lizziel commented Mar 29, 2024

yantosca commented Mar 29, 2024

yantosca commented Mar 29, 2024

lizziel commented Apr 10, 2024 • edited Loading

yantosca commented Apr 11, 2024

yantosca commented Apr 11, 2024

yantosca commented Apr 22, 2024

lizziel commented Apr 22, 2024

yantosca commented Apr 22, 2024

lizziel commented Apr 23, 2024

yantosca commented Apr 23, 2024

yantosca commented Apr 23, 2024

lizziel commented Apr 23, 2024

yantosca commented Apr 23, 2024

yantosca commented Apr 23, 2024

lizziel commented Mar 27, 2024 •

edited

Loading

lizziel commented Apr 10, 2024 •

edited

Loading