Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Included callable layer selector support for IntegratedGradients. #894

Merged
merged 4 commits into from
Mar 17, 2023

Conversation

RobertSamoilescu
Copy link
Collaborator

This PR includes callable layer selector support for IntegratedGradients. Previously, the reloading functionality was using the index of the layer within the model.layers list. Such an approach fails when the model is constructed base on nested layers. With the new functionality, the user can pass a function which select the desired model.

@codecov
Copy link

codecov bot commented Mar 16, 2023

Codecov Report

Merging #894 (7c2de13) into master (b0ae50d) will increase coverage by 0.01%.
The diff coverage is 84.84%.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #894      +/-   ##
==========================================
+ Coverage   85.32%   85.34%   +0.01%     
==========================================
  Files          74       74              
  Lines        8779     8801      +22     
==========================================
+ Hits         7491     7511      +20     
- Misses       1288     1290       +2     
Flag Coverage Δ
macos-latest-3.10 85.29% <84.84%> (+0.01%) ⬆️
ubuntu-latest-3.10 85.29% <84.84%> (+0.10%) ⬆️
ubuntu-latest-3.7 85.20% <84.84%> (+0.01%) ⬆️
ubuntu-latest-3.8 85.23% <84.84%> (+0.01%) ⬆️
ubuntu-latest-3.9 85.13% <84.84%> (+0.01%) ⬆️
windows-latest-3.9 82.98% <84.84%> (+0.11%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
alibi/exceptions.py 94.44% <66.66%> (-5.56%) ⬇️
alibi/explainers/integrated_gradients.py 88.81% <84.21%> (+0.10%) ⬆️
alibi/saving.py 83.75% <90.90%> (+0.41%) ⬆️

Copy link
Contributor

@jklaise jklaise left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice one! Left a few comments for some improvements.

alibi/explainers/integrated_gradients.py Show resolved Hide resolved
Comment on lines 844 to 848
try:
layer_num = model.layers.index(layer)
layer_meta = model.layers.index(layer)
except ValueError:
logger.info("Layer not in the list of model.layers")
layer_num = None
layer_meta = None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should use this case to also disallow serialization as currently it leads to a bad outcome where the explainer serializes successfully but is actually unusable/cannot be deserialized. I think this involves modifying the save logic depending on the value of layer_meta.

layer_num: Optional[int] = 0
else:
self.layer = None
layer_meta: Optional[Union[int, Callable[[tf.keras.Model], tf.keras.layers.Layer]]] = 0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this is a good time to define layer_meta better instead of using sentinel values like 0 and None which aren't very descriptive?

Perhaps a string Enum could be used here with 2 states corresonding to these two states + a callable corresponding to the 3rd state? Then the config would save either a string (if one of the first 2 states is true) or the lambda callable otherwise to layer_meta.

alibi/tests/test_saving.py Show resolved Hide resolved
Copy link
Contributor

@jklaise jklaise left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice one, looks great!

@jklaise jklaise merged commit 71027cb into SeldonIO:master Mar 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants