is it possible to explain a black box model with Shapley networks?

[unnir]

Is it possible to explain a black box model with Shapley networks?
Suppose, I have a trained CNN model, and I want to get feature attributions for a single image from IMAGENET data set.

[RuiWang1998]

Hi,

Thanks for your question. ShapNet has been designed to be an intrinsic explanation model, which means that it explains itself and only itself by design.

However, you could learn a ShapNet that mimic the behavior of your model of choice, which could work in theory, but we have no idea how well.

Best.

[unnir]

Thank you for your answer! Also, I think the paper 👍