You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
Incorrect _FillValue in NetCDF output when saving MTG data that has been loaded with other calibration than 'count'. Pixels outside the disk get assigned zeros, while netcdf attributes report _FillValue to be 65535.
Expected behavior
I would expect the _FillValue would be set so that the parts outside the disk would contain the fill value in the netcdf output. In the example case zeros are assigned to pixels outside the fulldisk, when the netcdf reports _FillValue to be 65535US.
Also setting _FillValue returns xarray exception "ValueError: failed to prevent overwriting existing key _FillValue in attrs on variable 'vis_06'. This is probably an encoding field used by xarray to describe how a variable is serialized. To proceed, remove this key from the variable's attributes manually.". I would expect overriding the _FillValue this way to work.
Actual results
Text output of actual results or error messages including full tracebacks if applicable.
Environment Info:
OS: OSX
Satpy Version: 0.51.0
PyResample Version: 1.30.0
Readers and writers dependencies (when relevant):
Readers
=======
abi_l1b: ok
abi_l1b_scmi: ok
abi_l2_nc: ok
acspo: ok
agri_fy4a_l1: ok
agri_fy4b_l1: ok
ahi_hrit: ok
ahi_hsd: ok
ahi_l1b_gridded_bin: ok
ahi_l2_nc: ok
ami_l1b: ok
amsr2_l1b: ok
amsr2_l2: ok
amsr2_l2_gaasp: ok
amsub_l1c_aapp: ok
ascat_l2_soilmoisture_bufr: cannot find module 'satpy.readers.ascat_l2_soilmoisture_bufr' (('Missing eccodes-python and/or eccodes C-library installation. Use conda to install eccodes.\n Error: ', ModuleNotFoundError("No module named 'eccodes'")))
atms_l1b_nc: ok
atms_sdr_hdf5: ok
avhrr_l1b_aapp: ok
avhrr_l1b_eps: cannot find module 'satpy.readers.eps_l1b' (No module named 'defusedxml')
avhrr_l1b_gaclac: cannot find module 'satpy.readers.avhrr_l1b_gaclac' (No module named 'pygac')
avhrr_l1b_hrpt: ok
avhrr_l1c_eum_gac_fdr_nc: ok
caliop_l2_cloud: cannot find module 'satpy.readers.caliop_l2_cloud' (No module named 'pyhdf')
clavrx: cannot find module 'satpy.readers.clavrx' (No module named 'pyhdf')
cmsaf-claas2_l2_nc: ok
electrol_hrit: ok
epic_l1b_h5: ok
fci_l1c_nc: ok
fci_l2_bufr: cannot find module 'satpy.readers.eum_l2_bufr' (Missing eccodes-python and/or eccodes C-library installation. Use conda to install eccodes)
fci_l2_nc: ok
fy3a_mersi1_l1b: ok
fy3b_mersi1_l1b: ok
fy3c_mersi1_l1b: ok
generic_image: cannot find module 'satpy.readers.generic_image' (No module named 'rioxarray')
geocat: ok
gerb_l2_hr_h5: ok
ghi_l1: ok
ghrsst_l2: ok
gld360_ualf2: ok
glm_l2: ok
gms5-vissr_l1b: cannot find module 'satpy.readers.gms.gms5_vissr_l1b' (No module named 'numba')
goci2_l2_nc: ok
goes-imager_hrit: ok
goes-imager_nc: ok
gpm_imerg: ok
grib: cannot find module 'satpy.readers.grib' (No module named 'pygrib')
hsaf_grib: cannot find module 'satpy.readers.hsaf_grib' (No module named 'pygrib')
hsaf_h5: ok
hy2_scat_l2b_h5: ok
iasi_l2: ok
iasi_l2_cdr_nc: ok
iasi_l2_so2_bufr: cannot find module 'satpy.readers.iasi_l2_so2_bufr' (('Missing eccodes-python and/or eccodes C-library installation. Use conda to install eccodes.\n Error: ', ModuleNotFoundError("No module named 'eccodes'")))
ici_l1b_nc: ok
insat3d_img_l1b_h5: cannot find module 'satpy.readers.insat3d_img_l1b_h5' (No module named 'datatree' It can be installed with the xarray-datatree package.)
jami_hrit: ok
li_l2_nc: ok
maia: ok
meris_nc_sen3: ok
mersi2_l1b: ok
mersi3_l1b: ok
mersi_ll_l1b: ok
mersi_rm_l1b: ok
mhs_l1c_aapp: ok
mimicTPW2_comp: ok
mirs: ok
modis_l1b: cannot find module 'satpy.readers.modis_l1b' (No module named 'pyhdf')
modis_l2: cannot find module 'satpy.readers.modis_l2' (No module named 'pyhdf')
modis_l3: cannot find module 'satpy.readers.modis_l3' (No module named 'pyhdf')
msi_safe: cannot find module 'satpy.readers.msi_safe' (No module named 'defusedxml')
msi_safe_l2a: cannot find module 'satpy.readers.msi_safe' (No module named 'defusedxml')
msu_gsa_l1b: ok
mtsat2-imager_hrit: ok
mviri_l1b_fiduceo_nc: ok
mwi_l1b_nc: ok
mws_l1b_nc: ok
nucaps: ok
nwcsaf-geo: ok
nwcsaf-msg2013-hdf5: ok
nwcsaf-pps_nc: ok
oceancolorcci_l3_nc: ok
olci_l1b: ok
olci_l2: ok
omps_edr: ok
osisaf_nc: ok
safe_sar_l2_ocn: ok
sar-c_safe: cannot find module 'satpy.readers.sar_c_safe' (No module named 'defusedxml')
satpy_cf_nc: ok
scatsat1_l2b: cannot find module 'satpy.readers.scatsat1_l2b' (cannot import name 'Dataset' from 'satpy.dataset' (/Users/ANVIR/micromamba/envs/satpy/lib/python3.12/site-packages/satpy/dataset/init.py))
seadas_l2: cannot find module 'satpy.readers.seadas_l2' (No module named 'pyhdf')
seviri_l1b_hrit: ok
seviri_l1b_icare: cannot find module 'satpy.readers.seviri_l1b_icare' (No module named 'pyhdf')
seviri_l1b_native: ok
seviri_l1b_nc: ok
seviri_l2_bufr: cannot find module 'satpy.readers.eum_l2_bufr' (Missing eccodes-python and/or eccodes C-library installation. Use conda to install eccodes)
seviri_l2_grib: cannot find module 'satpy.readers.seviri_l2_grib' (Missing eccodes-python and/or eccodes C-library installation. Use conda to install eccodes)
sgli_l1b: ok
slstr_l1b: ok
smos_l2_wind: ok
tropomi_l2: ok
vii_l1b_nc: ok
vii_l2_nc: ok
viirs_compact: ok
viirs_edr: ok
viirs_edr_active_fires: ok
viirs_edr_flood: cannot find module 'satpy.readers.viirs_edr_flood' (No module named 'pyhdf')
viirs_l1b: ok
viirs_l2: ok
viirs_sdr: ok
viirs_vgac_l1c_nc: ok
virr_l1b: ok
Writers
awips_tiled: ok
cf: ok
geotiff: ok
mitiff: ok
ninjogeotiff: ok
ninjotiff: cannot find module 'satpy.writers.ninjotiff' (No module named 'pyninjotiff')
simple_image: ok
Extras
cartopy: No module named 'cartopy'
geoviews: No module named 'geoviews'
Describe the bug
Incorrect _FillValue in NetCDF output when saving MTG data that has been loaded with other calibration than 'count'. Pixels outside the disk get assigned zeros, while netcdf attributes report _FillValue to be 65535.
To Reproduce
Expected behavior
I would expect the _FillValue would be set so that the parts outside the disk would contain the fill value in the netcdf output. In the example case zeros are assigned to pixels outside the fulldisk, when the netcdf reports _FillValue to be 65535US.
Also setting _FillValue returns xarray exception "ValueError: failed to prevent overwriting existing key _FillValue in attrs on variable 'vis_06'. This is probably an encoding field used by xarray to describe how a variable is serialized. To proceed, remove this key from the variable's attributes manually.". I would expect overriding the _FillValue this way to work.
Actual results
Text output of actual results or error messages including full tracebacks if applicable.
Environment Info:
Readers
=======
abi_l1b: ok
abi_l1b_scmi: ok
abi_l2_nc: ok
acspo: ok
agri_fy4a_l1: ok
agri_fy4b_l1: ok
ahi_hrit: ok
ahi_hsd: ok
ahi_l1b_gridded_bin: ok
ahi_l2_nc: ok
ami_l1b: ok
amsr2_l1b: ok
amsr2_l2: ok
amsr2_l2_gaasp: ok
amsub_l1c_aapp: ok
ascat_l2_soilmoisture_bufr: cannot find module 'satpy.readers.ascat_l2_soilmoisture_bufr' (('Missing eccodes-python and/or eccodes C-library installation. Use conda to install eccodes.\n Error: ', ModuleNotFoundError("No module named 'eccodes'")))
atms_l1b_nc: ok
atms_sdr_hdf5: ok
avhrr_l1b_aapp: ok
avhrr_l1b_eps: cannot find module 'satpy.readers.eps_l1b' (No module named 'defusedxml')
avhrr_l1b_gaclac: cannot find module 'satpy.readers.avhrr_l1b_gaclac' (No module named 'pygac')
avhrr_l1b_hrpt: ok
avhrr_l1c_eum_gac_fdr_nc: ok
caliop_l2_cloud: cannot find module 'satpy.readers.caliop_l2_cloud' (No module named 'pyhdf')
clavrx: cannot find module 'satpy.readers.clavrx' (No module named 'pyhdf')
cmsaf-claas2_l2_nc: ok
electrol_hrit: ok
epic_l1b_h5: ok
fci_l1c_nc: ok
fci_l2_bufr: cannot find module 'satpy.readers.eum_l2_bufr' (Missing eccodes-python and/or eccodes C-library installation. Use conda to install eccodes)
fci_l2_nc: ok
fy3a_mersi1_l1b: ok
fy3b_mersi1_l1b: ok
fy3c_mersi1_l1b: ok
generic_image: cannot find module 'satpy.readers.generic_image' (No module named 'rioxarray')
geocat: ok
gerb_l2_hr_h5: ok
ghi_l1: ok
ghrsst_l2: ok
gld360_ualf2: ok
glm_l2: ok
gms5-vissr_l1b: cannot find module 'satpy.readers.gms.gms5_vissr_l1b' (No module named 'numba')
goci2_l2_nc: ok
goes-imager_hrit: ok
goes-imager_nc: ok
gpm_imerg: ok
grib: cannot find module 'satpy.readers.grib' (No module named 'pygrib')
hsaf_grib: cannot find module 'satpy.readers.hsaf_grib' (No module named 'pygrib')
hsaf_h5: ok
hy2_scat_l2b_h5: ok
iasi_l2: ok
iasi_l2_cdr_nc: ok
iasi_l2_so2_bufr: cannot find module 'satpy.readers.iasi_l2_so2_bufr' (('Missing eccodes-python and/or eccodes C-library installation. Use conda to install eccodes.\n Error: ', ModuleNotFoundError("No module named 'eccodes'")))
ici_l1b_nc: ok
insat3d_img_l1b_h5: cannot find module 'satpy.readers.insat3d_img_l1b_h5' (No module named 'datatree' It can be installed with the xarray-datatree package.)
jami_hrit: ok
li_l2_nc: ok
maia: ok
meris_nc_sen3: ok
mersi2_l1b: ok
mersi3_l1b: ok
mersi_ll_l1b: ok
mersi_rm_l1b: ok
mhs_l1c_aapp: ok
mimicTPW2_comp: ok
mirs: ok
modis_l1b: cannot find module 'satpy.readers.modis_l1b' (No module named 'pyhdf')
modis_l2: cannot find module 'satpy.readers.modis_l2' (No module named 'pyhdf')
modis_l3: cannot find module 'satpy.readers.modis_l3' (No module named 'pyhdf')
msi_safe: cannot find module 'satpy.readers.msi_safe' (No module named 'defusedxml')
msi_safe_l2a: cannot find module 'satpy.readers.msi_safe' (No module named 'defusedxml')
msu_gsa_l1b: ok
mtsat2-imager_hrit: ok
mviri_l1b_fiduceo_nc: ok
mwi_l1b_nc: ok
mws_l1b_nc: ok
nucaps: ok
nwcsaf-geo: ok
nwcsaf-msg2013-hdf5: ok
nwcsaf-pps_nc: ok
oceancolorcci_l3_nc: ok
olci_l1b: ok
olci_l2: ok
omps_edr: ok
osisaf_nc: ok
safe_sar_l2_ocn: ok
sar-c_safe: cannot find module 'satpy.readers.sar_c_safe' (No module named 'defusedxml')
satpy_cf_nc: ok
scatsat1_l2b: cannot find module 'satpy.readers.scatsat1_l2b' (cannot import name 'Dataset' from 'satpy.dataset' (/Users/ANVIR/micromamba/envs/satpy/lib/python3.12/site-packages/satpy/dataset/init.py))
seadas_l2: cannot find module 'satpy.readers.seadas_l2' (No module named 'pyhdf')
seviri_l1b_hrit: ok
seviri_l1b_icare: cannot find module 'satpy.readers.seviri_l1b_icare' (No module named 'pyhdf')
seviri_l1b_native: ok
seviri_l1b_nc: ok
seviri_l2_bufr: cannot find module 'satpy.readers.eum_l2_bufr' (Missing eccodes-python and/or eccodes C-library installation. Use conda to install eccodes)
seviri_l2_grib: cannot find module 'satpy.readers.seviri_l2_grib' (Missing eccodes-python and/or eccodes C-library installation. Use conda to install eccodes)
sgli_l1b: ok
slstr_l1b: ok
smos_l2_wind: ok
tropomi_l2: ok
vii_l1b_nc: ok
vii_l2_nc: ok
viirs_compact: ok
viirs_edr: ok
viirs_edr_active_fires: ok
viirs_edr_flood: cannot find module 'satpy.readers.viirs_edr_flood' (No module named 'pyhdf')
viirs_l1b: ok
viirs_l2: ok
viirs_sdr: ok
viirs_vgac_l1c_nc: ok
virr_l1b: ok
Writers
awips_tiled: ok
cf: ok
geotiff: ok
mitiff: ok
ninjogeotiff: ok
ninjotiff: cannot find module 'satpy.writers.ninjotiff' (No module named 'pyninjotiff')
simple_image: ok
Extras
cartopy: No module named 'cartopy'
geoviews: No module named 'geoviews'
Additional context
Discussion thread on Pytroll slack: https://pytroll.slack.com/archives/C0LNH7LMB/p1729147424510329
The text was updated successfully, but these errors were encountered: