issue when using scale_factor and add_offset in zarr

[gdkrmr]

Please correct me if this is not an issue:

However, if the scale_factor and add_offset attributes are of a different data type from the variable (containing the packed data) then the unpacked data should match the type of these attributes, which must both be of type float or both be of type double .

From the CFConventions 8.1 Packed Data.

This is an issue with Zarr specifically: Attributes are defined in JSON and JSON can only define 64 bit floats. scale_factor and add_offset should be the same data type as the unpacked data. This means, at least in a strict sense, that CF-compliant Zarr cannot define 32 bit floats when using scale_factor and/or add_offset.

[taylor13]

Are you saying that in producing a Zarr file one has no control over the datatype of attributes? If so, this would seem to be a general, unfortunate limitation, which Zarr might want to remedy. (And I see you have raised the issue with them, so let's wait to hear what they have to say.)

[ethanrd]

There are a number of places where the netCDF and Zarr data models don't match exactly, typed attributes is one example. The netCDF library developers have started and are working with the Zarr community on an NCZarr extension (see discussion #186) to improve the mapping between the two data models. There is also work on a Zarr version 3 specification that, as I understand it, will include some of the netCDF features that are missing in Zarr version 2.

[gdkrmr]

which Zarr might want to remedy.

Agreed, this is a limitation of Zarr

(see discussion #zarr-developers/zarr-specs#186)

Seems like the Zarr developers are aware of the issue. Thanks for pointing me there.

Should the issue stay open, maybe with an "implementation" tag or similar?

[JonathanGregory]

I have changed the label of this issue from defect to question because the discussion shows it's not an issue about the text of the CF convention.

[JonathanGregory]

I'm closing this issue because I believe the question was answered and there is no action for CF. Thanks for asking the question, @gdkrmr.