Software Requirements

Data Access Requirements

SR-01 Global data access function

The API shall offer a global function or method of a global object that returns a dataset representation when given a dataset name and a list of optional selectors. The selectors allow to subset the overall dataset in the data store.

Possible solutions:

The following example presents access to a local data store:

data_store = LocalDataStore('/home/norman/esa-cci-data')   
ozone_ds = data_store.load('ozone/data/total_columns/l3/merged/v0100/ESACCI-OZONE-L3S-TC-MERGED-DLR_1M-(?P<year>\d\d\d\d)0104-fv0100.nc', lambda d: int(d['year']) == 2012)
print(ozone_ds)    

It has the advantage that the LocalDataStore does not need to know anything about the contents of the source data tree. It has the disadvantage that users must know how the file tree is organised and how dataset files are named. A smarter LocalDataStore could prescan the tree and create an index. The index will contain the various different datasets available. Each dataset comprises a set of netCDF files or shapefiles where each file contributes to a unique time series. For each dataset the index would also provide the common file content schema, the spatial and temporal coverage and other information about the grid, the coordinate reference system used.