Partial dependence errors with column with string and NaN values

[jeremyliweishih]

Partial dependence errors when a column contains both string values and NaN. There seems to be some failure in generating the grid as np.unique()` cannot run on mixed values.

import pandas as pd
from evalml import AutoMLSearch
from evalml.model_understanding import partial_dependence

df = pd.read_csv('1625078186889-mushroom_subset.csv')
y_train = df['class']
X_train = df.drop('class', axis=1)

aml = AutoMLSearch(X_train, y_train, 'binary')
aml.search()

pipeline = aml.best_pipeline

holdout = pd.read_csv('mushroom_holdout.csv')
partial_dependence(pipeline, holdout, 3)

Notebook and datasets:
string_nan_ex.zip

[freddyaboulton]

I'm not sure if there's much we can do here until we compute the grid for partial dependence ourselves for categoricals, which is what I think #2502 is tracking or we can wait for sklearn to let us provide the grid ourselves: scikit-learn/scikit-learn#20890

[freddyaboulton]

Chatted with @dsherry and the team at OH. We agree that in order to fix this bug we need to be able to compute our own grid for partial dependence values. That is what #2502 is tracking.

FYI @chukarsten since you werent at OH. This might affect the current sprint scope.