Support for GradientBoostingClassifier using TreeExplainer

[SaadAhmed96]

I am trying to use the GradientBoostingClassifier using the sklearn library. I wanted to known if its already supported or am I doing any mistake in implementation?

[slundberg]

It is because only the log odds prior is supported right now for the GradientBoostingClassifier: https://github.com/slundberg/shap/blob/b9e71b7d71e04d835d5ce6a000a5b7ea211c7d21/shap/explainers/tree.py#L472-L477

In order to support the PriorProbabilityEstimator another elif would need to be added that correctly sets the base_offset (the starting point the tree begin boosting from), and the units of the values in the leaves of the tree. Then a unit test would need to be added that checks that the sum of the SHAP values equals the model output as expected (this catches almost all types of errors). I'll add a help-wanted tag in case anyone wants to work on a PR for this :)

[SaadAhmed96]

Okay got it.
Thanks

[r-ichi]

I'm not receiving any errors when trying to use sklearn's GB implementation, is it now correctly supported by any means? :/

Anyway, the package is great! thanks for all the effort!

[crankyelephant]

Hello!
I have a slightly different issue with applying TreeExplainer SHAP to scikit-learn's GradientBoostingClassifier. Generating SHAP values runs ok, but the expected_value of the Explainer doesn't really make sense. It returns something like -8, whereas it should be from [0,1].

Here is a piece of code I'm using:

model = GradientBoostingClassifier(criterion='friedman_mse', init=None,
              learning_rate=0.1, loss='deviance', max_depth=9,
              max_features=None, max_leaf_nodes=None,
              min_impurity_decrease=0.0, min_impurity_split=None,
              min_samples_leaf=1, min_samples_split=2,
              min_weight_fraction_leaf=0.0, n_estimators=600,
              n_iter_no_change=None, presort='auto', random_state=None,
              subsample=1.0, tol=0.0001, validation_fraction=0.1,
              verbose=0, warm_start=False)
model.fit(X_train, y_train)
explainer = shap.TreeExplainer(model)
shap_values = explainer.shap_values(X_test)

Running
explainer.expected_value
returns -7.8781 and force plots give nonsensical values for output value.

[desertnaut]

@crankyelephant it is not that straightforward; see the SO thread How to interpret base_value of GBT classifier when using SHAP?

[desertnaut]

@slundberg should we close the issue now?

[subsheep]

Indeed no errors when using sklearn's GradientBoostingClassifier with TreeExplainer.
Looking at a comment on a related issue (#232 (comment)), it seems that GB is now supported.
Should we close this issue then @slundberg, or I wonder if the support is still under development?

[RoyiAvital]

I can see that GradientBoostingClassifier is only supported for binary classification.
Is there a plan to extend it?

[github-actions]

This issue has been inactive for two years, so it's been automatically marked as 'stale'.

We value your input! If this issue is still relevant, please leave a comment below. This will remove the 'stale' label and keep it open.

If there's no activity in the next 90 days the issue will be closed.