Update tutorial notebooks with direct data and model downloads (when possible)

[elboyran]

See Leon's nice tutorial on RFB case #707 on how to download directly from Zenodo. See also the text in italics below.

tutorials

├── README.md
├── conversion_onnx
│   ├── ...
├── overview.ipynb
└── explainers
├── KernelSHAP
│   ├── kernelshap_geometric_shapes.ipynb
│   ├── kernelshap_mnist.ipynb
│   ├── kernelshap_tabular_penguin.ipynb
│   └── kernelshap_tabular_weather.ipynb
├── LIME
│   ├── lime_images.ipynb
│   ├── lime_tabular_penguin.ipynb
│   ├── lime_tabular_weather.ipynb
│   ├── lime_text.ipynb
│   ├── lime_timeseries_coffee.ipynb
│   └── lime_timeseries_weather.ipynb
└── RISE
├── rise_imagenet.ipynb
├── rise_mnist.ipynb
├── rise_text.ipynb
└── rise_timeseries_weather.ipynb
└── rise_timeseries_frb.ipynb (already model downloaded, not by pooch)

Zenodo links per tutorial:

Generic

• overview.ipynb : data available only locally | text ONNX model| TS regression ONNX model| binary MNIST onnx model The model variant used in tutorial is with TF int he name, otherwise the one from zenodo gives an TF error...

KernelSHAP

• kernelshap_geometric_shapes.ipynb: shape data | onnx model
• kernelshap_mnist.ipynb: data available only locally | onnx model
• kernelshap_tabular_penguin.ipynb: data already downloaded from SNS | onnx model class.
• kernelshap_tabular_weather.ipynb: data already downloaded from Zenodo using pandas, try pooch | onnx model regr.

LIME

• lime_images.ipynb: example image is local | onnx model
• lime_tabular_penguin.ipynb: see kernelshap_tabular_penguin.ipynb above
• lime_tabular_weather.ipynb see kernelshap_tabular_weather.ipynb above
• lime_text.ipynb: the word vectors are only local | onnx model
• lime_timeseries_coffee.ipynb: might be easier to load locally the data and labels compared to from the source | onnx model
• lime_timeseries_weather.ipynb data generated in the notebook or already downloaded from Zenodo or read locally | onnx model

RISE

• rise_imagenet.ipynb: example image is local | model loaded from TF
• rise_mnist.ipynb: see kernelshap_mnist.ipynb above
• rise_text.ipynb: see lime_text.ipynb above
• rise_timeseries_weather.ipynb: see lime_timeseries_weather.ipynb above

[loostrum]

Perhaps we can use the pooch library to simplify this

[elboyran]

@loostrum, do you mind if I make (in the ever growing #776) the RFB model being downloaded under models folder, not in the same folder as the tutorial?
I refer to the # Download model from zenodo cell at the beginning of rise_rimeseries_frb.ipynb.

[elboyran]

@loostrum, do you mind if I make (in the ever growing #776) the RFB model being downloaded under models folder, not in the same folder as the tutorial? I refer to the # Download model from zenodo cell at the beginning of rise_rimeseries_frb.ipynb.

done by faf3c4f

[loostrum]

That won't work if dianna is installed system-wide though, what is the objection to downloading it to the local folder?

[elboyran]

What do you mean by "installed system-wide"? Why wouldn't it work, if we still use the root_dir?
As for the reasoning - it looks a bit confusing that all other tutorial models are under models' folder and this one ends up along with all RISE tutorials.

[loostrum]

Effectively I mean that it fails when the user that installed dianna is not the same user that runs the notebooks. As a concrete example we pre-installed dianna at the AI4S2S workshop. The participants would not have write acces to the root_dir.
In general I'm not sure it's a good idea to download things to the install folder, I'm not sure if those downloaded files are removed when someone later uninstalls dianna (which they shouldn't, of course ;-) )

[elboyran]

Perhaps we can use the pooch library to simplify this
Didn't understand how to get the hash.

[elboyran]

For the overview notebook, it's been addressed in #792

[loostrum]

Perhaps we can use the pooch library to simplify this
Didn't understand how to get the hash.

The hash is a hash of the file, e.g. a MD5 of SHA256 hash. It's probably easiest to get with pooch itself, see this part of the documentation of pooch.retrieve:

You can bypass these checks by passing ``known_hash=None``. If this is
done, the SHA256 hash of the downloaded file will be logged to the screen.
It is highly recommended that you copy and paste this hash as *known_hash*
so that future downloads are guaranteed to be the exact same file. This is
crucial for reproducible computations.

[loostrum]

@elboyran Nice that it's fixed in the overview notebook! Shall I pick it up for the other notebooks?

[elboyran]

@loostrum , please, go ahead!