Skip to content

Commit

Permalink
Merge branch 'main' into sk/update_dyamond_summer
Browse files Browse the repository at this point in the history
  • Loading branch information
sriharshakandala committed Nov 21, 2024
2 parents 00c031a + c052a6a commit dd988b8
Show file tree
Hide file tree
Showing 19 changed files with 1,884 additions and 27 deletions.
62 changes: 48 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,38 +29,48 @@ artifacts.

## Artifacts available

### Atmosphere

- [Aerosol concentrations (monthly mean, decadal averaged from 1970 to 2030)](https://github.com/CliMA/ClimaArtifacts/tree/main/aerosol_concentrations)
- [Ozone concentrations (monthly mean, from 1950 to 2014)](https://github.com/CliMA/ClimaArtifacts/tree/main/ozone_concentrations)
- [Cloud properties (monthly mean, 2010)]((https://github.com/CliMA/ClimaArtifacts/tree/main/era5_cloud))
- [Temperature average and restart file for ClimaAtmos Held-Suarez perfect model calibration](https://github.com/CliMA/ClimaArtifacts/tree/main/atmos_held_suarez_obs)
- [Large-scale forcing from HadGEM2-A at cfSites for driving single column models in ClimaAtmos.jl](https://github.com/CliMA/ClimaArtifacts/tree/main/cfsite_gcm_forcing)
- [Initial conditions for summer DYAMOND simulations](https:////github.com/CliMA/ClimaArtifacts/tree/main/atmos_dyamond_summer)

### Land

- [CLM surface data for the year 2000](https://github.com/CliMA/ClimaArtifacts/tree/main/clm_data)
- [Soil parameters needed for Richards equation; global at 1km resolution from
S. Gupta et al 2022, 2024](https://github.com/CliMA/ClimaArtifacts/tree/main/soil_params_Gupta2020_2022)
- [GPCP precipitation data](https://github.com/CliMA/ClimaArtifacts/tree/main/precipitation_obs)
- [CERES radiative fluxes data](https://github.com/CliMA/ClimaArtifacts/tree/main/radiation_obs)
- [RMSE for CMIP models](https://github.com/CliMA/ClimaArtifacts/tree/main/cmip_model_rmse)
- [Temperature average and restart file for ClimaAtmos Held-Suarez perfect model calibration](https://github.com/CliMA/ClimaArtifacts/tree/main/atmos_held_suarez_obs)
- [ESM-SnowMIP data](https://github.com/CliMA/ClimaArtifacts/tree/main/snowmip)
- [Large-scale forcing from HadGEM2-A at cfSites for driving single column models in ClimaAtmos.jl](https://github.com/CliMA/ClimaArtifacts/tree/main/cfsite_gcm_forcing)
- [Measured sea ice concentration and sea surface
temperature](https://github.com/CliMA/ClimaArtifacts/tree/main/historical_sst_sic)
- [Global shortwave albedo from CLM model output for coupled and standalone land models](https://github.com/CliMA/ClimaArtifacts/tree/main/cesm2_albedo)
- [Earth orography at 30 and 60 arc-second resolutions](https:////github.com/CliMA/ClimaArtifacts/tree/main/earth_orography)
- [Earth bedrock depth at 30 and 60 arc-second resolutions](https:////github.com/CliMA/ClimaArtifacts/tree/main/bedrock_depth)
- [Monthly Mean CO2 from Mauna Loa](https://github.com/CliMA/ClimaArtifacts/tree/main/co2_dataset)
- [Foliage clumping index, derived from MODIS data for 2006](https:////github.com/CliMA/ClimaArtifacts/tree/main/modis_clumping_index)
- [Subset of ILAMB datasets](https:////github.com/CliMA/ClimaArtifacts/tree/main/ilamb_data)
- [Bonan 2019 Richards equation data](https:////github.com/CliMA/ClimaArtifacts/tree/main/bonan_richards_eqn)
- [TwoStream model implementation test data](https:////github.com/CliMA/ClimaArtifacts/tree/main/twostr_test)
- [Berkeley Earth Global Gridded Temperature Data](https:////github.com/CliMA/ClimaArtifacts/tree/main/surface_temperatures)
- [Selected FLUXNET tower data](https:////github.com/CliMA/ClimaArtifacts/tree/main/fluxnet_sites)
- [ESM-SnowMIP data](https://github.com/CliMA/ClimaArtifacts/tree/main/snowmip)
- [ERA5 Monthly Averages 2008](https:////github.com/CliMA/ClimaArtifacts/tree/main/era5_surface_fluxes_2008)
- [Initial conditions for summer DYAMOND simulations](https:////github.com/CliMA/ClimaArtifacts/tree/main/atmos_dyamond_summer)
- [Global shortwave albedo from CLM model output for coupled and standalone land models](https://github.com/CliMA/ClimaArtifacts/tree/main/cesm2_albedo)
- [Earth bedrock depth at 30 and 60 arc-second resolutions](https:////github.com/CliMA/ClimaArtifacts/tree/main/bedrock_depth)
- [ERA5 Land Forcing Data 2008](https:////github.com/CliMA/ClimaArtifacts/tree/main/era5_land_forcing_data2008)
- [ERA5 Monthly Averages on Pressure Levels 1979-2024](https:////github.com/CliMA/ClimaArtifacts/tree/main/era5_monthly_averages_pressure_levels_1979_2024)
- [TOPMODEL topographic index statistics](https:////github.com/CliMA/ClimaArtifacts/tree/main/topmodel)
- [ERA5 Monthly Averages on Single Levels 1979-2024](https:////github.com/CliMA/ClimaArtifacts/tree/main/era5_monthly_averages_single_level_1979_2024)

### Coupler/shared

- [Measured sea ice concentration and sea surface
temperature](https://github.com/CliMA/ClimaArtifacts/tree/main/historical_sst_sic)
- [GPCP precipitation data](https://github.com/CliMA/ClimaArtifacts/tree/main/precipitation_obs)
- [CERES radiative fluxes data](https://github.com/CliMA/ClimaArtifacts/tree/main/radiation_obs)
- [RMSE for CMIP models](https://github.com/CliMA/ClimaArtifacts/tree/main/cmip_model_rmse)
- [Earth orography at 30 and 60 arc-second resolutions](https:////github.com/CliMA/ClimaArtifacts/tree/main/earth_orography)
- [Monthly Mean CO2 from Mauna Loa](https://github.com/CliMA/ClimaArtifacts/tree/main/co2_dataset)

# The ultimate guide to ClimaArtifacts

Last update: 26 September 2024
Last update: 21 November 2024

#### What is an artifact?

Expand Down Expand Up @@ -252,6 +262,30 @@ that folder, where `AAAAAA` is the name of your artifact. It should print
`/groups/esm/ClimaArtifacts/artifacts/AAAAAA`, where `AAAAAA` is the folder you
just uploaded.

#### What is a _undownloadable_ artifact? Why do we need them?

`ClimaArtifacts` treats small and big files differently.

Small files are uploaded to network-accessible location. When a package using
these artifacts is registered, these files are also mirror by the Julia storage
servers. Users automatically obtain the artifact upon package instantiation (if
not lazy) or when first used (if lazy). When the artifact is obtained, its
integrity is also verified by computing a cryptographic hash and check it
against a known value.

Many of the above-described features that small files easy to use and
worry-free, become a problem for large files:
1. Mirroring large files on the Julia servers is not an acceptable use of shared
resources;
2. Automatically download large files greatly increases the instantiation time;
3. Computing cryptographic hashes for large files can take a long time.

In addition to this, we do not want to hide the location of the large files and
we want to let users pick the location. This is because home directories might
have limited storage space, and more generally users might want to decide where
to save large files. This is especially on clusters, where homes are small and
groups might decide to save the input files in folders shared across users.

#### How to upload on the Caltech Box?

Go to [Caltech Box](https://caltech.app.box.com/). Navigate to your favorite
Expand Down
22 changes: 11 additions & 11 deletions era5_monthly_averages_2008/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# ERA5 monthly averages
# ERA5 monthly averages 2008

This artifact processes data coming from [ERA5 monthly averaged reanalysis](https://cds.climate.copernicus.eu/datasets/reanalysis-era5-single-levels-monthly-means?tab=download) and contains
monthly averaged surface fluxes and runoff rates.
Expand Down Expand Up @@ -41,37 +41,37 @@ There processing of the downloaded dataset results in two output datasets:

Both of the output datasets contain the same variables and spatial coverage, but the variables are defined on different time dimensions.

## Spatial Coverage
### Spatial Coverage

- 1 degree latitude x 1 degree longitude grid
- -90N to 90N and 0E to 359E
- The latitudes are reversed during processing to be in increasing order

The shared variables, which are all stored as Float32s and defined on the latitude, longitude, and time dimensions, are:

## `mslhf`
### `mslhf`

This is the mean surface latent heat flux in units of W m**-2. No processing is done to this variable other than flipping the latitude dimension.

## `msshf`
### `msshf`

This is the mean surface sensible heat flux in units of W m**-2. No processing is done to this variable other than flipping the latitude dimension.

## `mssror`
### `mssror`

This is the mean sub-surface runoff rate in units of kg m**-2 s**-1. No processing is done to this variable other than flipping the latitude dimension.

## `msror`
### `msror`

This is the mean surface runoff rate in units of kg m**-2 s**-1. No processing is done to this variable other than flipping the latitude dimension.

## `msuwlwrf`
### `msuwlwrf`

This is the mean surface upward long-wave radiation flux in units of W m**-2.
This variable is created during processing by taking the difference of
mean surface downward long-wave radiation flux and mean surface net long-wave radiation flux, and then flipping the latitude dimension.

## `msuwswrf`
### `msuwswrf`

This is the mean surface upward short-wave radiation flux in units of W m**-2.
This variable is created during processing by taking the difference of
Expand All @@ -84,7 +84,7 @@ The two output files have different temporal coverage.
### `era5_monthly_surface_fluxes_200801-200812.nc`

This file contains Monthly averaged reanalysis, which is produced by averaging all daily data for each month. This results in 12 points on the
time dimension, where each point in the 15th of the month that the point represents. For example, the 6th index of `time` is 2008-06-15T00:00:00,
time dimension, where each point is the 15th of the month that the point represents. For example, the 6th index of `time` is 2008-06-15T00:00:00,
which represents the whole month of June in 2008.

### `era5_monthly_surface_fluxes_hourly_200801-200812.nc`
Expand All @@ -94,8 +94,8 @@ This file contains Monthly averages by hour of day, which constitutes the averag

## Citation

Hersbach, H., Bell, B., Berrisford, P., Biavati, G., Horányi, A., Muñoz Sabater, J., Nicolas, J., Peubey, C., Radu, R., Rozum, I., Schepers, D., Simmons, A., Soci, C., Dee, D., Thépaut, J-N. (2023): ERA5 monthly averaged data on single levels from 1940 to present. Copernicus Climate Change Service (C3S) Climate Data Store (CDS), DOI: 10.24381/cds.f17050d7 (Accessed on DD-MMM-YYYY)
Hersbach, H., Bell, B., Berrisford, P., Biavati, G., Horányi, A., Muñoz Sabater, J., Nicolas, J., Peubey, C., Radu, R., Rozum, I., Schepers, D., Simmons, A., Soci, C., Dee, D., Thépaut, J-N. (2023): ERA5 monthly averaged data on single levels from 1940 to present. Copernicus Climate Change Service (C3S) Climate Data Store (CDS), DOI: 10.24381/cds.f17050d7 (Accessed on 11-11-2024)

## License

See the [LICENSE](LICENSE.txt) file
See the [LICENSE](LICENSE.txt) file
2 changes: 1 addition & 1 deletion era5_monthly_averages_2008/create_artifact.jl
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ if !isfile(DOWNLOAD_FILE_NAME)
println("Enter your CDS Personal Access Token:")
cds_PAT = readline()
println("Downloading data with CDS API using PAT: $cds_PAT")
run(`python download_data.py $cds_PAT`)
run(`python download_data.py -k $cds_PAT`)
end
end

Expand Down
1 change: 0 additions & 1 deletion era5_monthly_averages_2008/subset_of_data.jl
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,6 @@ function create_new_ds_from_time_indx(
Float32.(reverse(var[:, :, time_indx], dims = 2)),
dimnames(var);
attrib = attrib,
# attrib = copy(var.attrib),
deflatelevel = 9,
)
end
Expand Down
93 changes: 93 additions & 0 deletions era5_monthly_averages_single_level_1979_2024/LICENSE.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
I. Licence to Use Copernicus Products

1. Definitions

1.1. ‘Licensor’ means the European Union, represented by the European Centre for Medium-Range Weather Forecasts (ECMWF).

1.2. ‘Licensee’ means all natural or legal persons who agree to the terms of this Licence.

1.3. ‘Licence’ means this license agreement between the Licensor and the Licensee as amended from time to time.

1.4. ‘Copernicus Services’ means:

1.4.1. the Copernicus Atmosphere Monitoring Service (CAMS), which is to provide information on air quality on a local, national, and European scale, and the chemical composition of the atmosphere on a global scale.

1.4.2. the Copernicus Climate Change Service (C3S), which is to provide information to increase the knowledge base to support policies on adaptation to and mitigation of climate change

1.5. ‘Copernicus Products’ means all products listed in the C3S or CAMS Service Product Specification or any other items available through an ECMWF Copernicus portal, except those items which are labelled/flagged as being subject to their own separate terms of use.

1.6. ‘Intellectual Property Rights’ refers to intellectual property rights of all kinds,

1.6.1. including: all patents; rights to inventions; copyright and related rights; moral rights; trademarks and service marks; trade names and domain names; rights in get-up; rights to goodwill or to sue for passing off or unfair competition; rights in designs; rights in computer software; database rights; rights in confidential information (including know-how and trade secrets); any other rights in the nature of intellectual property rights;

1.6.2. in each case whether registered or unregistered and including all applications (or rights to apply) for, and renewals or extensions of, such rights and all similar or equivalent rights or forms of protection which subsist or will subsist now or in the future in any part of the world together with all rights of action in relation to the infringement of any of the above.

1.7. ‘Copernicus Contractor’ refers to providers of Copernicus related goods and services to ECMWF, including information and data, to the Licensor and/or to the users.

1.8. ‘Copernicus Regulations’ refers to Regulation (EU) No 377/2014 of the European Parliament and of the Council of 3 April 2014 establishing the Copernicus Programme.

1.9. ‘ECMWF Agreement’ refers to the agreement between the European Commission and ECMWF dated 11 November 2014 on the implementation of CAMS and C3S.

2. Introduction

Copernicus is funded under the Copernicus Regulation and operated by ECMWF under the ECMWF Agreement. Access to all Copernicus (previously known as GMES or Global Monitoring for Environment and Security) Information and Data is regulated under Regulation (EU) No 1159/2013 of the European Parliament and of the Council of 12 July 2013 on the European Earth monitoring programme, under the ECMWF Agreement and under the European Commission’s Terms and Conditions. Access to all Copernicus information is regulated under Regulation (EU) No 1159/2013 and under the ECMWF Agreement.

3. Terms of the Licence

This Licence sets out the terms for use of Copernicus Products. By agreeing to these terms, the Licensee agrees to abide by all of the terms and conditions in this Licence for the use of Copernicus Products.

4. Licence Permission

4.1. This Licence is free of charge, worldwide, non-exclusive, royalty free and perpetual.

4.2. Access to Copernicus Products is given for any purpose in so far as it is lawful, whereas use may include, but is not limited to: reproduction; distribution; communication to the public; adaptation, modification and combination with other data and information; or any combination of the foregoing.

5. Attribution

5.1. All users of Copernicus Products must provide clear and visible attribution to the Copernicus programme. The Licensee will communicate to the public the source of the Copernicus Products by crediting the Copernicus Climate Change and Atmosphere Monitoring Services:

5.1.1. Where the Licensee communicates or distributes Copernicus Products to the public, the Licensee shall inform the recipients of the source by using the following or any similar notice:
• 'Generated using Copernicus Climate Change Service information [Year]' and/or
• 'Generated using Copernicus Atmosphere Monitoring Service information [Year]'.

5.1.2. Where the Licensee makes or contributes to a publication or distribution containing adapted or modified Copernicus Products, the Licensee shall provide the following or any similar notice:
• 'Contains modified Copernicus Climate Change Service information [Year]'; and/or
• 'Contains modified Copernicus Atmosphere Monitoring Service information [Year]'

5.1.3. Any such publication or distribution covered by clauses 5.1.1 and 5.1.2 shall state that neither the European Commission nor ECMWF is responsible for any use that may be made of the Copernicus information or data it contains.

6. Intellectual Property Rights

6.1. All Intellectual Property Rights in the Copernicus Products belong, and will continue to belong, to the European Union.

6.2. All Intellectual Property Rights of new items created as a result of modifying or adapting the Copernicus Products through the applications and workflows accessible on the ECMWF Copernicus portals (e.g. through the CDS Toolbox) will belong to the European Union.

6.3. All other new Intellectual Property Rights created as a result of modifying or adapting the Copernicus information will be owned by the creator.

7. Provision of Third Party Information and Data

This Licence only covers Copernicus Products. Access to third party products, information, and data related to Copernicus information to which the Licensee is directed or which can be directly accessed through any Copernicus portal will be subject to different licence terms.

8. Disclaimers

8.1. Neither the Licensor nor ECMWF warrant that Copernicus Products will be free from errors or omissions or that such errors or omissions can or will be rectified, or that the Licensee will have uninterrupted, continuous, or timely access to Copernicus Products.

8.2. The Licensor, as well as ECMWF, exclude all warranties, conditions, terms, undertakings, obligations whether express or implied by statute including but not limited to the implied warranties of satisfactory quality and fitness for a particular purpose or otherwise to the fullest extent permitted by law.

9. Liabilities

Neither the Licensor nor ECMWF will accept liability for any damage, loss whether direct, indirect or consequential resulting from the Licensee’s use of the Copernicus Products.

10. Termination of and Changes to this Licence

The Licensor may terminate this licence if the Licensee breaches its obligations under these terms. The Licensor may revise this Licence at any time and will notify the Licensee of any revisions.

11. Arbitration Clause and Governing Law

In the event of a dispute arising in connection with this License, the parties shall attempt to settle their differences in an amicable manner. If any dispute cannot be so settled, it shall be settled under the Rules of Arbitration of the International Chamber of Commerce by one arbitrator appointed in accordance with the said rules sitting in London, United Kingdom. The proceedings shall be in the English language. The right of appeal by either party to regular Courts on a question of law arising in the course of any arbitral proceedings or out of an award made in any arbitral proceedings is hereby agreed to be excluded.

It is the intention of the parties that this License shall comprehensively govern the legal relations between the parties to the Licence, without interference or contradiction by any unspecified law. However, where a matter is not specifically covered by these terms or a provision of the Licence terms is ambiguous or unclear, resolution shall be found by reference to the laws of England and Wales, including any relevant law of the European Union.

Nothing stated in this License shall be construed as a waiver of any privileges or immunities of the Licensor or of ECMWF.

Version 1.2 (November 2019)
Loading

0 comments on commit dd988b8

Please sign in to comment.