diff --git a/docs/_toc.yml b/docs/_toc.yml index ae13a255..1b5e738a 100644 --- a/docs/_toc.yml +++ b/docs/_toc.yml @@ -9,6 +9,11 @@ parts: - caption: GETTING STARTED chapters: - file: overview + - file: user-guide/start + sections: + - file: user-guide/fields + - file: user-guide/time + - file: user-guide/versioning - file: examples - file: hydrocron_tutorial - caption: API ENDPOINTS diff --git a/docs/examples.md b/docs/examples.md index cf87fa59..9d3795ee 100644 --- a/docs/examples.md +++ b/docs/examples.md @@ -2,7 +2,7 @@ ## Get time series GeoJSON for river reach -Search for a single river reach by reach ID. +Search for a single river reach by reach ID: [https://soto.podaac.earthdatacloud.nasa.gov/hydrocron/v1/timeseries?feature=Reach&feature_id=78340600051&output=geojson&start_time=2024-01-25T00:00:00Z&end_time=2024-03-29T00:00:00Z&fields=reach_id,time_str,wse,slope](https://soto.podaac.earthdatacloud.nasa.gov/hydrocron/v1/timeseries?feature=Reach&feature_id=78340600051&output=geojson&start_time=2024-01-25T00:00:00Z&end_time=2024-03-29T00:00:00Z&fields=reach_id,time_str,wse,slope) @@ -86,7 +86,7 @@ Will return GeoJSON: ## Get time series GeoJSON for river node -Search for a single river node by ID. +Search for a single river node by ID: [https://soto.podaac.earthdatacloud.nasa.gov/hydrocron/v1/timeseries?feature=Node&feature_id=12228200110861&start_time=2024-01-25T00:00:00Z&end_time=2024-03-30T00:00:00Z&output=geojson&fields=reach_id,node_id,time_str,wse](https://soto.podaac.earthdatacloud.nasa.gov/hydrocron/v1/timeseries?feature=Node&feature_id=12228200110861&start_time=2024-01-25T00:00:00Z&end_time=2024-03-30T00:00:00Z&output=geojson&fields=reach_id,node_id,time_str,wse) diff --git a/docs/time.md b/docs/time.md deleted file mode 100644 index 76486980..00000000 --- a/docs/time.md +++ /dev/null @@ -1,16 +0,0 @@ -# Handling Time - -SWOT source data is organized to include all of the features from the prior river and lake databases that the satellite crosses over during each pass of a continent. -If for any reason SWOT does not record an observation of a prior database feature during a pass, the source data will contain fillvalues for all observed fields, including the time of observation. - -To retain times where there was a satellite pass but no observation was made, Hydrocron queries on the *start time of the range of observations included in the pass over the continent during the cycle of interest*. For example, if it takes 10 seconds for the satellite to pass over North America, 3 different river reaches observed during that pass may have an observation time recorded at 2 seconds, 5 seconds, and 9 seconds. However, Hydrocron uses the range start time of 0 seconds (the beginning of the 10 second window for the pass over the continent), with a buffer of -30 seconds from the start_time, +30 seconds from the end_time specified in the query. - -## Example - -| reach_id | time | pass_start_time | wse | ... | -|-------------|---------------------|--------------------|----------|-----| -| 71224100223 | 2023-08-01T12:30:45 |2023-08-01T12:30:30 | 316.8713 | | -| 71224100234 | 2023-08-01T12:30:42 |2023-08-01T12:30:30 | 286.2983 | | -| 71224100283 | no_data |2023-08-01T12:30:30 | -999999999999.0000| | - -In this case, querying Hydrocron using a start_time of 2023-08-01T12:30:00 will return all three features, becasue it is the pass start time that is used in the query. The returned data will include the actual observation time, including the no_data value for the feature that was not observed. diff --git a/docs/timeseries.md b/docs/timeseries.md index 6c8f487b..7295dd04 100644 --- a/docs/timeseries.md +++ b/docs/timeseries.md @@ -83,10 +83,12 @@ Content-Type: text/csv ## Request Parameters +(feature)= ### feature : string, required: yes Type of feature being requested. Either: "Reach", "Node" or "PriorLake" +(feature_id)= ### feature_id : string, required: yes ID of the feature to retrieve @@ -132,86 +134,36 @@ Whether to return a 'compact' GeoJSON response. Either: "true" or "false" The default for header `Accept: application/geo+json` is to set compact to `true` if it is not provided. The default header for `application/json` is to set compact to `false` if it is not provided. See "Response" section for details on the returned compact response. -### fields : string, required: yes +(collection_name)= +### collection_name: string, required: no -The SWOT data fields to return in the request. +The name of the collection to return. Allows users to explicitly request data from a particular version the data. +Supported collection names include: -This is specified in the form of a comma separated list (without any spaces): `fields=reach_id,time_str,wse,slope` +Version C/2.0 (default): -Hydrocron includes additional fields beyond the source data shapefile attributes, including units fields on measurements, cycle and pass information, SWORD and PLD (prior river and lake database names), and collection versions. **NOTE: Units are always returned for fields that have corresponding units stored in Hydrocron, they do not need to be requested.** The complete list of input fields that are available through Hydrocron are below: +- SWOT_L2_HR_RiverSP_2.0, +- SWOT_L2_HR_RiverSP_reach_2.0, +- SWOT_L2_HR_RiverSP_node_2.0, +- SWOT_L2_HR_LakeSP_2.0, +- SWOT_L2_HR_LakeSP_prior_2.0, -**Reach data fields** +Version D: -```bash -'reach_id', 'time', 'time_tai', 'time_str', 'p_lat', 'p_lon', 'river_name', -'wse', 'wse_u', 'wse_r_u', 'wse_c', 'wse_c_u', -'slope', 'slope_u', 'slope_r_u', 'slope2', 'slope2_u', 'slope2_r_u', -'width', 'width_u', 'width_c', 'width_c_u', -'area_total', 'area_tot_u', 'area_detct', 'area_det_u', 'area_wse', -'d_x_area', 'd_x_area_u', -'layovr_val', 'node_dist', 'loc_offset', 'xtrk_dist', -'dschg_c', 'dschg_c_u', 'dschg_csf', 'dschg_c_q', -'dschg_gc', 'dschg_gc_u', 'dschg_gcsf', 'dschg_gc_q', -'dschg_m', 'dschg_m_u', 'dschg_msf', 'dschg_m_q', -'dschg_gm', 'dschg_gm_u', 'dschg_gmsf', 'dschg_gm_q', -'dschg_b', 'dschg_b_u', 'dschg_bsf', 'dschg_b_q', -'dschg_gb', 'dschg_gb_u', 'dschg_gbsf', 'dschg_gb_q', -'dschg_h', 'dschg_h_u', 'dschg_hsf', 'dschg_h_q', -'dschg_gh', 'dschg_gh_u', 'dschg_ghsf', 'dschg_gh_q', -'dschg_o', 'dschg_o_u', 'dschg_osf', 'dschg_o_q', -'dschg_go', 'dschg_go_u', 'dschg_gosf', 'dschg_go_q', -'dschg_s', 'dschg_s_u', 'dschg_ssf', 'dschg_s_q', -'dschg_gs', 'dschg_gs_u', 'dschg_gssf', 'dschg_gs_q', -'dschg_i', 'dschg_i_u', 'dschg_isf', 'dschg_i_q', -'dschg_gi', 'dschg_gi_u', 'dschg_gisf', 'dschg_gi_q', -'dschg_q_b', 'dschg_gq_b', -'reach_q', 'reach_q_b', -'dark_frac', 'ice_clim_f', 'ice_dyn_f', 'partial_f', 'n_good_nod', -'obs_frac_n', 'xovr_cal_q', 'geoid_hght', 'geoid_slop', -'solid_tide', 'load_tidef', 'load_tideg', 'pole_tide', -'dry_trop_c', 'wet_trop_c', 'iono_c', 'xovr_cal_c', -'n_reach_up', 'n_reach_dn', 'rch_id_up', 'rch_id_dn', -'p_wse', 'p_wse_var', 'p_width', 'p_wid_var', 'p_n_nodes', 'p_dist_out', -'p_length', 'p_maf', 'p_dam_id', 'p_n_ch_max', 'p_n_ch_mod', 'p_low_slp', -'cycle_id', 'pass_id', 'continent_id', 'range_start_time', 'range_end_time', -'crid', 'geometry', 'sword_version', 'collection_shortname', 'collection_version', -'granuleUR', 'ingest_time' -``` +- SWOT_L2_HR_RiverSP_D, +- SWOT_L2_HR_RiverSP_reach_D, +- SWOT_L2_HR_RiverSP_node_D, +- SWOT_L2_HR_LakeSP_D, +- SWOT_L2_HR_LakeSP_prior_D -**Node data fields** +(fields)= +### fields : string, required: yes -```bash -'reach_id', 'node_id', 'time', 'time_tai', 'time_str', -'lat', 'lon', 'lat_u', 'lon_u', 'river_name', -'wse', 'wse_u', 'wse_r_u', -'width', 'width_u', -'area_total', 'area_tot_u', 'area_detct', 'area_det_u', 'area_wse', -'layovr_val', 'node_dist', 'xtrk_dist', -'flow_angle', 'node_q', 'node_q_b', -'dark_frac', 'ice_clim_f', 'ice_dyn_f', 'partial_f', 'n_good_pix', -'xovr_cal_q', 'rdr_sig0', 'rdr_sig0_u', 'rdr_pol', -'geoid_hght', 'solid_tide', 'load_tidef', 'load_tideg', 'pole_tide', -'dry_trop_c', 'wet_trop_c', 'iono_c', 'xovr_cal_c', -'p_wse', 'p_wse_var', 'p_width', 'p_wid_var', 'p_dist_out', 'p_length', -'p_dam_id', 'p_n_ch_max', 'p_n_ch_mod', -'cycle_id', 'pass_id', 'continent_id', 'range_start_time', 'range_end_time', -'crid', 'geometry', 'sword_version', 'collection_shortname' -``` +The SWOT data fields to return in the request. -**Lake data fields** -```bash -'lake_id', 'reach_id', 'obs_id', 'overlap', 'n_overlap', -'time', 'time_tai', 'time_str', 'wse', 'wse_u', 'wse_r_u', 'wse_std', -'area_total', 'area_tot_u', 'area_detct', 'area_det_u', -'layovr_val', 'xtrk_dist', 'ds1_l', 'ds1_l_u', 'ds1_q', 'ds1_q_u', -'ds2_l', 'ds2_l_u', 'ds2_q', 'ds2_q_u', -'quality_f', 'dark_frac', 'ice_clim_f', 'ice_dyn_f', 'partial_f', -'xovr_cal_q', 'geoid_hght', 'solid_tide', 'load_tidef', 'load_tideg', 'pole_tide', -'dry_trop_c', 'wet_trop_c', 'iono_c', 'xovr_cal_c', 'lake_name', 'p_res_id', -'p_lon', 'p_lat', 'p_ref_wse', 'p_ref_area', 'p_date_t0', 'p_ds_t0', 'p_storage', -'cycle_id', 'pass_id', 'continent_id', 'range_start_time', 'range_end_time', -'crid', 'geometry', 'PLD_version', 'collection_shortname', 'crid' -``` +This is specified in the form of a comma separated list (without any spaces): `fields=reach_id,time_str,wse,slope` + +Hydrocron includes additional fields beyond the source data shapefile attributes, including units fields on measurements, cycle and pass information, SWORD and PLD (prior river and lake database names), and collection versions. **NOTE: Units are always returned for fields that have corresponding units stored in Hydrocron, they do not need to be requested.** The complete list of input fields that are available through Hydrocron are described in the [](fields-detail) section. ## Response Format diff --git a/docs/user-guide/fields.md b/docs/user-guide/fields.md new file mode 100644 index 00000000..5343152a --- /dev/null +++ b/docs/user-guide/fields.md @@ -0,0 +1,89 @@ +(fields-detail)= +# Fields + +Hydrocron will return every field that is available in the archived source data. For SWOT River and Lake data products, these fields are the attributes that are available in the shapefiles. + +In addition to the shapefile attributes, some additional fields are available through Hydrocron that are pulled from other locations such as the shapefile metadata (xml) and the filename. These include things like the cycle and pass numbers, CRID, granule name, continent ID, units, collection name, etc. + +The [](fields) parameter is required, and you must list every field that you want to return in a comma-separated list, with no spaces. Units are a special case in that they will be automatically returned for any fields you request that have units attached. You do not need to explicitly list the units fields separately. + +We strongly recommend returning and using the quality flags on the fields that have them to avoid degraded observations. + +For each feature type, the full list of currently supported fields is below. Full descriptions of what these fields are and how to use them are available in the SWOT Product Description Documents available on the PO.DAAC collection landing pages for [Rivers](https://podaac.jpl.nasa.gov/dataset/SWOT_L2_HR_RiverSP_2.0) and [Lakes](https://podaac.jpl.nasa.gov/dataset/SWOT_L2_HR_LakeSP_2.0) + +Occasionally new fields may be added to the SWOT data products. If there are fields you find in the SWOT shapefiles that are not returned from Hydrocron, please open an issue on the [Hydrocron GitHub repository](https://github.com/podaac/hydrocron). + +**Reach fields** + +```bash +'reach_id', 'time', 'time_tai', 'time_str', 'p_lat', 'p_lon', 'river_name', +'wse', 'wse_u', 'wse_r_u', 'wse_c', 'wse_c_u', +'slope', 'slope_u', 'slope_r_u', 'slope2', 'slope2_u', 'slope2_r_u', +'width', 'width_u', 'width_c', 'width_c_u', +'area_total', 'area_tot_u', 'area_detct', 'area_det_u', 'area_wse', +'d_x_area', 'd_x_area_u', +'layovr_val', 'node_dist', 'loc_offset', 'xtrk_dist', +'dschg_c', 'dschg_c_u', 'dschg_csf', 'dschg_c_q', +'dschg_gc', 'dschg_gc_u', 'dschg_gcsf', 'dschg_gc_q', +'dschg_m', 'dschg_m_u', 'dschg_msf', 'dschg_m_q', +'dschg_gm', 'dschg_gm_u', 'dschg_gmsf', 'dschg_gm_q', +'dschg_b', 'dschg_b_u', 'dschg_bsf', 'dschg_b_q', +'dschg_gb', 'dschg_gb_u', 'dschg_gbsf', 'dschg_gb_q', +'dschg_h', 'dschg_h_u', 'dschg_hsf', 'dschg_h_q', +'dschg_gh', 'dschg_gh_u', 'dschg_ghsf', 'dschg_gh_q', +'dschg_o', 'dschg_o_u', 'dschg_osf', 'dschg_o_q', +'dschg_go', 'dschg_go_u', 'dschg_gosf', 'dschg_go_q', +'dschg_s', 'dschg_s_u', 'dschg_ssf', 'dschg_s_q', +'dschg_gs', 'dschg_gs_u', 'dschg_gssf', 'dschg_gs_q', +'dschg_i', 'dschg_i_u', 'dschg_isf', 'dschg_i_q', +'dschg_gi', 'dschg_gi_u', 'dschg_gisf', 'dschg_gi_q', +'dschg_q_b', 'dschg_gq_b', +'reach_q', 'reach_q_b', +'dark_frac', 'ice_clim_f', 'ice_dyn_f', 'partial_f', 'n_good_nod', +'obs_frac_n', 'xovr_cal_q', 'geoid_hght', 'geoid_slop', +'solid_tide', 'load_tidef', 'load_tideg', 'pole_tide', +'dry_trop_c', 'wet_trop_c', 'iono_c', 'xovr_cal_c', +'n_reach_up', 'n_reach_dn', 'rch_id_up', 'rch_id_dn', +'p_wse', 'p_wse_var', 'p_width', 'p_wid_var', 'p_n_nodes', 'p_dist_out', +'p_length', 'p_maf', 'p_dam_id', 'p_n_ch_max', 'p_n_ch_mod', 'p_low_slp', +'cycle_id', 'pass_id', 'continent_id', 'range_start_time', 'range_end_time', +'crid', 'geometry', 'sword_version', 'collection_shortname', 'collection_version', +'granuleUR', 'ingest_time' +``` + +**Node fields** + +```bash +'reach_id', 'node_id', 'time', 'time_tai', 'time_str', +'lat', 'lon', 'lat_u', 'lon_u', 'river_name', +'wse', 'wse_u', 'wse_r_u', +'width', 'width_u', +'area_total', 'area_tot_u', 'area_detct', 'area_det_u', 'area_wse', +'layovr_val', 'node_dist', 'xtrk_dist', +'flow_angle', 'node_q', 'node_q_b', +'dark_frac', 'ice_clim_f', 'ice_dyn_f', 'partial_f', 'n_good_pix', +'xovr_cal_q', 'rdr_sig0', 'rdr_sig0_u', 'rdr_pol', +'geoid_hght', 'solid_tide', 'load_tidef', 'load_tideg', 'pole_tide', +'dry_trop_c', 'wet_trop_c', 'iono_c', 'xovr_cal_c', +'p_wse', 'p_wse_var', 'p_width', 'p_wid_var', 'p_dist_out', 'p_length', +'p_dam_id', 'p_n_ch_max', 'p_n_ch_mod', +'cycle_id', 'pass_id', 'continent_id', 'range_start_time', 'range_end_time', +'crid', 'geometry', 'sword_version', 'collection_shortname', 'collection_version', +'granuleUR', 'ingest_time' +``` + +**Lake fields** +```bash +'lake_id', 'reach_id', 'obs_id', 'overlap', 'n_overlap', +'time', 'time_tai', 'time_str', 'wse', 'wse_u', 'wse_r_u', 'wse_std', +'area_total', 'area_tot_u', 'area_detct', 'area_det_u', +'layovr_val', 'xtrk_dist', 'ds1_l', 'ds1_l_u', 'ds1_q', 'ds1_q_u', +'ds2_l', 'ds2_l_u', 'ds2_q', 'ds2_q_u', +'quality_f', 'dark_frac', 'ice_clim_f', 'ice_dyn_f', 'partial_f', +'xovr_cal_q', 'geoid_hght', 'solid_tide', 'load_tidef', 'load_tideg', 'pole_tide', +'dry_trop_c', 'wet_trop_c', 'iono_c', 'xovr_cal_c', 'lake_name', 'p_res_id', +'p_lon', 'p_lat', 'p_ref_wse', 'p_ref_area', 'p_date_t0', 'p_ds_t0', 'p_storage', +'cycle_id', 'pass_id', 'continent_id', 'range_start_time', 'range_end_time', +'crid', 'geometry', 'PLD_version', 'collection_shortname', 'collection_version', +'granuleUR', 'ingest_time' +``` diff --git a/docs/user-guide/start.md b/docs/user-guide/start.md new file mode 100644 index 00000000..0f1bf7ec --- /dev/null +++ b/docs/user-guide/start.md @@ -0,0 +1,3 @@ +# User Guide + +In this user guide, you will find detailed descriptions of the behavior of Hydrocron and examples of how to use it. diff --git a/docs/user-guide/time.md b/docs/user-guide/time.md new file mode 100644 index 00000000..2b6ed74d --- /dev/null +++ b/docs/user-guide/time.md @@ -0,0 +1,16 @@ +# Observed Time vs Range Time + +SWOT source data is organized to include all of the features from the prior river and lake databases that the satellite crosses over during each pass of a continent. +If for any reason SWOT does not record an observation of a prior database feature during a pass, the source data will contain fillvalues for all observed fields, including the time of observation. + +To retain times where there was a satellite pass but no observation was made, Hydrocron queries on the *start time of the range of observations included in the pass over the continent during the cycle of interest*. For example, if it takes 10 seconds for the satellite to pass over North America, 3 different river reaches observed during that pass may have an observation time recorded at 2 seconds, 5 seconds, and 9 seconds. However, Hydrocron uses the range start time of 0 seconds (the beginning of the 10 second window for the pass over the continent) as the start time, and the range end time of 9 seconds as the end time when querying for data. + +## Example + +| reach_id | time | range_start_time | range_start_time | wse | ... | +|-------------|---------------------|---------------------|---------------------|---------------|-----| +| 71224100223 | 2023-08-01T12:30:45 |2023-08-01T12:30:30 |2023-08-01T12:40:30 | 316.8713 | | +| 71224100223 | no_data |2023-09-01T12:30:30 |2023-09-01T12:40:30 | -99999999.0000| | +| 71224100223 | 2023-10-01T12:30:42 |2023-10-01T12:30:30 |2023-10-01T12:40:30 | 286.2983 | | + +In this simplified example, querying Hydrocron using a start_time of 2023-08-01T12:30:00 and an end_time of 2023-10-01T13:00:00 will return all three features, becasue it is the pass start time that is used in the query. The returned data will include the actual observation time, including the no_data value for the feature that was not observed. diff --git a/docs/user-guide/versioning.md b/docs/user-guide/versioning.md new file mode 100644 index 00000000..e764d917 --- /dev/null +++ b/docs/user-guide/versioning.md @@ -0,0 +1,54 @@ +# SWOT Data Versioning + +SWOT data periodically undergoes reprocessing, where data products are recreated from the original observations with updated processing algorithms. This results in a new data collection version. + +In version 1.6.0, Hydrocron will support multiple versions of SWOT data collections. To specify which collection you want to retrieve data from, use the new [](collection_name) request parameter. + +SWOT collection names supported by Hydrocron inclcude: + +Version C/2.0 (default): + +- SWOT_L2_HR_RiverSP_2.0 +- SWOT_L2_HR_RiverSP_reach_2.0 +- SWOT_L2_HR_RiverSP_node_2.0 +- SWOT_L2_HR_LakeSP_2.0 +- SWOT_L2_HR_LakeSP_prior_2.0 + +Version D (coming early 2025): + +- SWOT_L2_HR_RiverSP_D +- SWOT_L2_HR_RiverSP_reach_D +- SWOT_L2_HR_RiverSP_node_D +- SWOT_L2_HR_LakeSP_D +- SWOT_L2_HR_LakeSP_prior_D + +## Parent Collections and Sub Collections + +SWOT hydrology data products are organized into parent and child collections. The parent collections indicate the general feature type (river or lake): + +- SWOT_L2_HR_RiverSP_2.0 +- SWOT_L2_HR_LakeSP_2.0 + +and the sub collections indicate the specific feature or subtype of data (reach or node for rivers, prior, observed, or unassigned for lakes) + +- SWOT_L2_HR_RiverSP_reach_2.0 +- SWOT_L2_HR_RiverSP_node_2.0 +- SWOT_L2_HR_LakeSP_prior_2.0 + +The SWOT Product Description Documents for SWOT Rivers and Lakes contain more information about the differences between subtypes of data. Note that currently Hydrocron does not support the lake subtypes of 'observed' and 'unassigned'. + +## Collection name and feature type + +Specifying sub collections in requests to Hydrocron is not necessary to distinguish the type of data to return, as you also specify whether you are looking for Reach, Node, or PriorLake data with the [](feature) parameter. + +Note that if you mix feature types in the [](feature) and [](collection_name) parameters, the [](collection_name) parameter takes precedence. If you request [](feature_id) for a feature type that is different that what you request in [](collection_name), you will get an error. + +For example, here we request feature=Reach with a feature_id= a reach feature, but collection_name=SWOT_L2_HR_LakeSP_2.0. In this case you will get an error because Hydrocron will try to query the Lakes table for the reach_id. + +[https://soto.podaac.earthdatacloud.nasa.gov/hydrocron/v1/timeseries?feature=Reach&feature_id=78340600051&output=geojson&start_time=2024-01-25T00:00:00Z&end_time=2024-03-29T00:00:00Z&collection_name=SWOT_L2_HR_LakeSP_2.0&fields=reach_id,time_str,wse,slope](https://soto.podaac.earthdatacloud.nasa.gov/hydrocron/v1/timeseries?feature=Reach&feature_id=78340600051&output=geojson&start_time=2024-01-25T00:00:00Z&end_time=2024-03-29T00:00:00Z&collection_name=SWOT_L2_HR_LakeSP_2.0&fields=reach_id,time_str,wse,slope) + +## CRID + +The CRID indicates the version of the processing software that was used to generate the data. When data is first delivered to PO.DAAC and ingested into the archive, the CRID will be the forward-streaming version. If that data is later reprocessed, the CRID will be updated to indicate the reprocessing version. Hydrocron only keeps one version of a given observation at a time. If the data has been reprocessed, it will overwrite the earlier forward-streaming version with the reprocessed version. + +Note that you cannot query by CRID, but you can return it as a field in the request and see which CRID was used to process the data that is being returned.