Fixes for TDS 5.0

[dopplershift]

• Doesn't create an NCSS access url for a point collection
• Warnings on parsing current NCSS dataset.xml

WARNING:root:No parser found for element TimeUnit
WARNING:root:No parser found for element AltitudeUnits
WARNING:root:attribute type uint not understood.
WARNING:root:attribute type uint not understood.
WARNING:root:attribute type uint not understood.
WARNING:root:attribute type uint not understood.
WARNING:root:attribute type uint not understood.
WARNING:root:attribute type uint not understood.
WARNING:root:attribute type uint not understood.
WARNING:root:attribute type uint not understood.
WARNING:root:attribute type uint not understood.

[dopplershift]

Need to prioritize this given that TDS 5 is now the stable release and Unidata production servers are now using it.

For instance, today running this:

from siphon.catalog import TDSCatalog
cat = TDSCatalog('https://thredds.ucar.edu/thredds/catalog/grib/NCEP/RAP/CONUS_13km/catalog.xml')
ds = cat.latest
ncss = ds.subset()

gives:

Cannot convert "[337]" to int. Keeping type as str.
Cannot convert "[451]" to int. Keeping type as str.
Cannot convert "[]" to int. Keeping type as str.
Cannot convert "[22]" to int. Keeping type as str.
Cannot convert "[1]" to int. Keeping type as str.
Cannot convert "[1]" to int. Keeping type as str.
Cannot convert "[1]" to int. Keeping type as str.
Cannot convert "[6]" to int. Keeping type as str.
Cannot convert "[21]" to int. Keeping type as str.
Cannot convert "[1]" to int. Keeping type as str.
Cannot convert "[1]" to int. Keeping type as str.
Cannot convert "[1]" to int. Keeping type as str.
Cannot convert "[37]" to int. Keeping type as str.
Cannot convert "[2]" to int. Keeping type as str.

(cc @haileyajohnson)

[zoj613]

Need to prioritize this given that TDS 5 is now the stable release and Unidata production servers are now using it.

For instance, today running this:

from siphon.catalog import TDSCatalog
cat = TDSCatalog('https://thredds.ucar.edu/thredds/catalog/grib/NCEP/RAP/CONUS_13km/catalog.xml')
ds = cat.latest
ncss = ds.subset()

gives:

Cannot convert "[337]" to int. Keeping type as str.
Cannot convert "[451]" to int. Keeping type as str.
Cannot convert "[]" to int. Keeping type as str.
Cannot convert "[22]" to int. Keeping type as str.
Cannot convert "[1]" to int. Keeping type as str.
Cannot convert "[1]" to int. Keeping type as str.
Cannot convert "[1]" to int. Keeping type as str.
Cannot convert "[6]" to int. Keeping type as str.
Cannot convert "[21]" to int. Keeping type as str.
Cannot convert "[1]" to int. Keeping type as str.
Cannot convert "[1]" to int. Keeping type as str.
Cannot convert "[1]" to int. Keeping type as str.
Cannot convert "[37]" to int. Keeping type as str.
Cannot convert "[2]" to int. Keeping type as str.

(cc @haileyajohnson)

IM glad im not the only one who is getting this and it is causing my unit tests to fail because the resulting lat/lon coordinates are strings and strings cant be compared to int/float values. Is there a way around this unexpected behaviour?

[dopplershift]

I'm sure it's a minor parsing problems with dataset.xml. In your unit tests you're examining the parsed dataset.xml?

I'm unlikely to get to this in the short term. Depending on your urgency, you may want to consider submitting a pull request fixing it.

[JimiC]

The problem occurs because the shape attribute is represented as a stringified array (i.e. '[451]').
The root cause starts in

siphon/src/siphon/ncss_dataset.py

Line 340 in 198cbb4

typed_vals = self._types.handle_typed_values(axis['shape'], 'shape', 'int')

Example: <axis name="latitude" shape="[721]" type="float" axisType="Lat"> <attribute xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2" name="units" value="degrees_north"/> <values spacing="regularPoint" npts="721" start="90.0" end="-90.0" resolution="-0.25"/> </axis> from https://thredds.ucar.edu/thredds/ncss/grid/grib/NCEP/GFS/Global_0p25deg/Best/dataset.xml

P.S. A nice improvement at this line would be to parse the type attribute of axis to determine the value type, like typed_vals = self._types.handle_typed_values(axis['shape'], 'shape', axis['type']). You may also want to use a fallback in case axis['type'] does not exist.

Then on

siphon/src/siphon/ncss_dataset.py

Line 78 in 198cbb4

val = [int(v) for v in re.split('[ ,]', val) if v]

and

siphon/src/siphon/ncss_dataset.py

Line 83 in 198cbb4

val = [float(v) for v in re.split('[ ,]', val) if v]

the re.split fails producing the warning message.

If replaced with re.split("[ ,]", re.sub(r"^\[|\]$", "", val)) problem is solved.

Because I'm not familiar with the code-base, I won't attempt a PR since the maintainers may have a better idea how to tackle this issue.

I'm just pointing out were it fails.

[dopplershift]

@JimiC Thanks for the potential adjustment. Unfortunately, the type for the axis tag refers to the values within the axis, not the shape, which should always be one or more integers.

Also, not sure that spliting on "[" or "]" is really what we want. I think #852 has a good fix.