Skip to content
This repository has been archived by the owner on Oct 8, 2019. It is now read-only.

when downloading new MODIS files, check that each HDF file is valid #341

Open
robinkraft opened this issue Feb 13, 2013 · 0 comments
Open
Labels

Comments

@robinkraft
Copy link
Contributor

Inspired by #299.

A checker function in the updater script should open each HDF file after it's downloaded and thereby catch any obvious issues before they showed up in a Cascalog query. Only after it was successfully opened should the upload to S3 commence. On error, the problem file should be noted and its filename included in the notification email.

fname = "/mnt/files/MOD13A1.A2012305.h09v09.005.2012322025347.hdf"
f = gdal.Open(fname)
ERROR 4: `/mnt/files/MOD13A1.A2012305.h09v09.005.2012322025347.hdf' not recognised as a supported file format.

if type(f) == NoneType:
    raise Exception("Invalid file: %s" % fname)
else:
    startUpload(fname)
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

1 participant