Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update get_onc_ctd and get_onc_ferry workers to work with ONC API v3 #234

Merged
merged 13 commits into from
Feb 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 63 additions & 6 deletions config/nowcast.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -245,7 +245,7 @@ temperature salinity:


observations:
# Dataset that maps high resolution lon/lat grid on to NEMO j/i indices
# Dataset that maps high-resolution lon/lat grid on to NEMO j/i indices
lon/lat to NEMO ji map: /SalishSeaCast/grid/grid_from_lat_lon_mask999.nc
# ONC Strait of Georgia nodes real-time CTD data
ctd data:
Expand All @@ -259,12 +259,69 @@ observations:
dest dir: /results/observations/ONC/CTD/
# Template for ONC CTD T&S data file path
# **Must be quoted to project {} characters**
filepath template: '{station}/{station}_CTD_15m_{yyyymmdd}.nc'
filepath template: "{station}/{station}_CTD_15m_{yyyymmdd}.nc"

# ONC Strait of Georgia ferry platforms real-time data
ferry data:
ferries: {}
# see PR#109 for TWDP ferry example of this section and corresponding config tests
ferries:
# Tsawwassen - Duke Point route
TWDP:
route name: Tsawwassen - Duke Point
ONC station description: Mobile Platforms, British Columbia Ferries, Tsawwassen - Duke Point
location:
# ONC scalardata.getByStation API query parameters:
#
# ONC station identifiers to use for nav data, in priority order
stations:
- TWDP.N1
- TWDP.N2
device category: NAV
sensors:
- longitude
- latitude
terminals:
# Terminal names from salishsea_tools.places.PLACES
- Tsawwassen
- Duke Pt.
devices:
# ONC device category
TSG:
# device sensor names
sensors:
# ERDDAP sensor name: ONC sensor name
temperature: temperature
conductivity: conductivity
salinity: salinity
OXYSENSOR:
sensors:
o2_saturation: oxygen_saturation
o2_concentration_corrected: oxygen_corrected
o2_temperature: temperature
TURBCHLFL:
sensors:
cdom_fluorescence: cdom_fluorescence
chlorophyll: chlorophyll
turbidity: turbidity
CO2SENSOR:
sensors:
co2_partial_pressure: partial_pressure
co2_concentration_linearized: co2
TEMPHUMID:
sensors:
air_temperature: air_temperature
relative_humidity: rel_humidity
BARPRESS:
sensors:
barometric_pressure: barometric_pressure
PYRANOMETER:
sensors:
solar_radiation: solar_radiation
PYRGEOMETER:
sensors:
longwave_radiation: downward_radiation
# Template for ONC ferry data file path
# **Must be quoted to project {} characters**
filepath template: "{ferry_platform}/{ferry_platform}_TSG_O2_TURBCHLFL_CO2_METEO_1m_{yyyymmdd}.nc"
# Destination directory for ONC ferry data netCDF files
dest dir: /results/observations/ONC/ferries/

Expand All @@ -274,7 +331,7 @@ observations:
csv dir: /opp/observations/AISDATA/
# Destination directory for VFPA HADCP data netCDF files
dest dir: /opp/observations/AISDATA/netcdf/
# Template for VFPA HADCP data file path
# Template for the VFPA HADCP data file path
# **Must be quoted to project {} characters**
filepath template: 'VFPA_2ND_NARROWS_HADCP_2s_{yyyymm}.nc'

Expand Down Expand Up @@ -429,7 +486,7 @@ erddap:
# out of service since 22-Dec-2019; repair ETA unknown
# - ubcONCUSDDLCTD15mV1
TWDP-ferry:
- ubcONCTWDP1mV1
- ubcONCTWDP1mV18-01
nowcast-green:
- ubcSSg3DBiologyFields1hV21-11
- ubcSSg3DLightFields1hV21-11
Expand Down
17 changes: 8 additions & 9 deletions nowcast/workers/get_onc_ctd.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,7 @@


def main():
"""Set up and run the worker.

For command-line usage see:
"""For command-line usage see:

:command:`python -m nowcast.workers.get_onc_ctd -h`
"""
Expand All @@ -61,6 +59,7 @@ def main():
help="UTC date to get ONC node CTD data for.",
)
worker.run(get_onc_ctd, success, failure)
return worker


def success(parsed_args):
Expand All @@ -80,15 +79,15 @@ def failure(parsed_args):
def get_onc_ctd(parsed_args, config, *args):
ymd = parsed_args.data_date.format("YYYY-MM-DD")
logger.info(f"requesting ONC {parsed_args.onc_station} CTD T&S data for {ymd}")
TOKEN = os.environ["ONC_USER_TOKEN"]
onc_data = data_tools.get_onc_data(
"scalardata",
"getByStation",
TOKEN,
station=parsed_args.onc_station,
deviceCategory="CTD",
sensors="salinity,temperature",
"getByLocation",
os.environ["ONC_USER_TOKEN"],
locationCode=parsed_args.onc_station,
deviceCategoryCode="CTD",
sensorCategoryCodes="salinity,temperature",
dateFrom=data_tools.onc_datetime(f"{ymd} 00:00", "utc"),
dateTo=data_tools.onc_datetime(f"{ymd} 23:59", "utc"),
)
try:
ctd_data = data_tools.onc_json_to_dataset(onc_data)
Expand Down
Loading