Groupwise permutation importance?

[bgreenwell]

I've been asked several times about assigning importance to groups of features. Does it make sense to randomly permute all the features in a group to obtain such a metric?

[bradleyboehmke]

Are they looking to pre-specify this group or have the algorithm auto identify groups?

[bgreenwell]

Prespecified. As in all features fall into a particular group/bucket.

[bgreenwell]

Example for a model with five features: x1, x2, x3, x4, and x5

feature_groups <- list(
  "group1" = c("x1", "x2", "x5"),  # psychological feature
  "group2" = c("x3", "x4")         # physiological features
)
vi_permute(model, feature_names = feature_groups, ...)

[bgreenwell]

# This would work without having to change any other code
permute_columns <- function(x, columns = NULL) {
  x[, columns] <- x[sample(nrow(x)), columns]
  x
}

[bgreenwell]

I want to implement this, but is there any literature on this?!?!

[ptaconet]

maybe this paper ? https://www.sciencedirect.com/science/article/pii/S0167947315000997

[ptaconet]

or this one ? https://www.researchgate.net/publication/351095658_Grouped_Feature_Importance_and_Combined_Features_Effect_Plot

[bgreenwell]

Yes @ptaconet, this is exactly what I’ve been looking for, thanks for posting it!! My only struggle was whether it makes more sense to permute features in a group independently or jointly, but it looks like this paper makes the case for the latter, at least upon skimming.

I’ve neglected some open source projects lately due to other obligations, but I’ll bump this to the top of the list when I revisits the package in the next month or two! PRs welcome as well!

[Wenxin-Wan]

very looking forward to the update on this grouped implementation - indeed very relevant for many researchers and the https://www.researchgate.net/publication/351095658_Grouped_Feature_Importance_and_Combined_Features_Effect_Plot represents a nice reference for it.