Skip to content
This repository has been archived by the owner on Aug 29, 2023. It is now read-only.

UC9 CLI Demo

Jump to bottom
Jānis Gailis edited this page Oct 4, 2016 · 26 revisions

Use Case 9

Problem Definition:

A climate scientist wishes to analyse potential correlations between Ozone and Cloud ECVs.

Required Toolbox Features:

  • Access to and ingestion of ESA CCI Ozone and Cloud data (Atmosphere Mole Content of Ozone and Cloud Cover)
  • Geometric adjustments (coregistration)
  • Spatial (point, polygon) and temporal subsetting
  • Visualisation of time series
  • Correlation analysis, scatter-plot of correlation statistics, saving of image and correlation statistics

1. Data product ingestion

Use ds list to list available products. You can filter them according to some name.

ect ds list -n ozone
3 data sources found
   1: esacci.OZONE.day.L3S.TC.GOME-2.Metop-A.MERGED.fv0100.r1
   2: esacci.OZONE.day.L3S.TC.GOME.ERS-2.MERGED.fv0100.r1
   3: esacci.OZONE.mon.L3.NP.multi-sensor.multi-platform.MERGED.fv0002.r1

ect ds list -n cloud
14 data sources found
   1: esacci.CLOUD.day.L3U.CLD_PRODUCTS.AVHRR.NOAA-15.AVHRR_NOAA.1-0.r1
   2: esacci.CLOUD.day.L3U.CLD_PRODUCTS.AVHRR.NOAA-16.AVHRR_NOAA.1-0.r1
   3: esacci.CLOUD.day.L3U.CLD_PRODUCTS.AVHRR.NOAA-17.AVHRR_NOAA.1-0.r1
   4: esacci.CLOUD.day.L3U.CLD_PRODUCTS.AVHRR.NOAA-18.AVHRR_NOAA.1-0.r1
   5: esacci.CLOUD.day.L3U.CLD_PRODUCTS.MODIS.Aqua.MODIS_AQUA.1-0.r1
   6: esacci.CLOUD.day.L3U.CLD_PRODUCTS.MODIS.Terra.MODIS_TERRA.1-0.r1
   7: esacci.CLOUD.mon.L3C.CLD_PRODUCTS.AVHRR.NOAA-15.AVHRR_NOAA.1-0.r1
   8: esacci.CLOUD.mon.L3C.CLD_PRODUCTS.AVHRR.NOAA-16.AVHRR_NOAA.1-0.r1
   9: esacci.CLOUD.mon.L3C.CLD_PRODUCTS.AVHRR.NOAA-17.AVHRR_NOAA.1-0.r1
  10: esacci.CLOUD.mon.L3C.CLD_PRODUCTS.AVHRR.NOAA-18.AVHRR_NOAA.1-0.r1
  11: esacci.CLOUD.mon.L3C.CLD_PRODUCTS.MODIS.Aqua.MODIS_AQUA.1-0.r1
  12: esacci.CLOUD.mon.L3C.CLD_PRODUCTS.MODIS.Terra.MODIS_TERRA.1-0.r1
  13: esacci.CLOUD.mon.L3S.CLD_PRODUCTS.AVHRR.multi-platform.AVHRR_MERGED.1-0.r1
  14: esacci.CLOUD.mon.L3S.CLD_PRODUCTS.MODIS.multi-platform.MODIS_MERGED.1-0.r1

Create a new workspace.

ect ws new

INFO:tornado.access:200 GET / (127.0.0.1) 0.50ms
INFO:tornado.access:200 GET /ws/new?description=&base_dir=%2Fhome%2Fccitbx%2FDesktop&save=False (127.0.0.1) 0.66ms
Workspace created.

Open the desired datasets, by providing their name and desired time-span.

ect res open cl07 esacci.CLOUD.mon.L3C.CLD_PRODUCTS.AVHRR.NOAA-15.AVHRR_NOAA.1-0.r1 2007-01-01 2007-12-30
ect res open oz07 esacci.OZONE.mon.L3.NP.multi-sensor.multi-platform.MERGED.fv0002.r1 2007-01-01 2007-12-30

2. Dataset variable selection

To select particular geophysical quantities to work with, use the 'select_var' operation together with ect res set command:

ect res set cc_tot select_var ds=cl07 var=cc_total
ect res set oz_tot select_var ds=oz07 var=O3_du_tot

We can plot the datasets and save the plots using the plot_map operation:

ect ws run plot_map ds=cc_tot var=cc_total file=fig1.png


ect ws run plot_map ds=oz_tot var=O3_du_tot file=fig2.png

3. Co-Register the datasets

The datasets now have different lat/lon definitions. This can be verified by using ect res print

ect res print cc_tot
ect res print oz_tot

ect op list --tag geom

will list all commands that have a tag that matches 'geom'. To find out more about a particular operation, use ect op info

ect op info coregister

To carry out coregistration, use ect res set again with appropriate operation parameters

ect res set cc_tot_res coregister ds_master=oz_tot ds_slave=cc_tot

ect ws run plot_map ds=cc_tot_res var=cc_total file=fig3.png

4. Spatial Filtering

To filter the datasets to contain only a particular region use the subset_spatial operation.

ect res set oz_africa subset_spatial ds=oz_tot lat_min=-40 lat_max=40 lon_min=-20 lon_max=60
ect res set cc_africa subset_spatial ds=cc_tot_res lat_min=-40 lat_max=40 lon_min=-20 lon_max=60

ect ws run plot_map ds=cc_africa var=cc_total file=fig4.png


ect ws run plot_map ds=cc_africa var=cc_total lat_min=-40 lat_max=40 lon_min=-20 lon_max=60 file=fig5.png

5. Temporal Filtering

To further filter the datasets to contain only a particular time-span, use subset_temporal operation

ect res set oz_africa_janoct subset_temporal ds=oz_africa time_min='2007-01-01' time_max='2007-10-30'
ect res set cc_africa_janoct subset_temporal ds=cc_africa time_min='2007-01-01' time_max='2007-10-30'

If on Linux, quotes enclosing datetime strings should be additionally escaped:

ect res set oz_africa_janoct subset_temporal ds=oz_africa time_min=\'2007-01-01\' time_max=\'2007-10-30\'
ect res set cc_africa_janoct subset_temporal ds=cc_africa time_min=\'2007-01-01\' time_max=\'2007-10-30\'

6. Extract time series

We'll extract spatial mean timeseries from both datasets using tseries_mean operation.

ect res set cc_africa_ts tseries_mean ds=cc_africa_janoct var=cc_total
ect res set oz_africa_ts tseries_mean ds=oz_africa_janoct var=O3_du_tot

This creates datasets that contain mean and std variables for both time-series.

7. Time Series Plot

To plot the time-series and save the plot plot_1D operation can be used together with ect ws run operation:

ect ws run plot_1D ds=cc_africa_ts var=cc_total file=fig6.png


ect ws run plot_1D ds=oz_africa_ts var=O3_du_tot file=fig7.png

8. Product-Moment Correlation (Pearson)

To carry out Pearson correlation on the mean time-series, pearson_correlation operation can be used.

ect op list --tag correlation
ect res set pearson pearson_correlation ds_y=cc_africa_ts ds_x=oz_africa_ts var_y=cc_total var_x=O3_du_tot file=pearson.txt

This will calculate the correlation coefficient along with the associated p_value for both mean time-series, as well as save the information in the given file. We can view the result using ect res print:

ect res print pearson

If both variables provided to the pearson_correlation operation have time/lat/lon dimensions and the lat/lon definition is the same, a pixel by pixel correlation will be carried out and result in the creation of two variables of the same lat/lon dimension - corr_coeff and p_value that can then be plotted on a map.

ect res set pearson_map pearson_correlation ds_y=cc_africa_janoct ds_x=oz_africa_janoct var_y=cc_total var_x=O3_du_tot
ect ws run plot_map ds=pearson_map var=corr_coef lat_min=-40 lat_max=40 lon_min=-20 lon_max=60 file=fig8.png

Clone this wiki locally