Improve SAR performance #914
Conversation
Co-authored-by: Overv <[email protected]>
zegerius
requested review from
gramhagen,
miguelgfierro and
yueguoguo
as code owners
|
|
| seen = temp_df[[self.col_user_id, self.col_item_id]].values | ||
| self.seen_items = np.ones(self.user_affinity.shape) | ||
| self.seen_items[seen[:, 0], seen[:, 1]] = -np.inf | ||
|
|
There was a problem hiding this comment.
what if we remove seen and self.seen_items altogether and leverage the user_affinity since that is already captured, see code change below
| # remove items in the train set so recommended items are always novel | ||
| if remove_seen: | ||
| logger.info("Removing seen items") | ||
| test_scores = test_scores * self.seen_items[user_ids, :] |
There was a problem hiding this comment.
here we can add the reduced user_affinity back in after multiplying with -inf, zeros will stay zero and anything we consider the user has seen will have a non-zero entry which will turn to -inf.
test_scores += self.user_affinity[user_ids, :] * -np.inf
|
Thank you @gramhagen for that very elegant improvement. I have incorporate your suggested changes. All tests pass. Also ran a quick performance comparison and I don't see any difference compared to our initial solution (at least not more than one std dev): |
* Improve score() performance Co-authored-by: Overv <[email protected]> * Simplify remove_seen
Co-authored-by: @Overv
Description
During production implementation of the SAR recommender algorithm, we were facing severe performance issues. This was the result of calculating recommendations for all users in the entire set, despite only requiring recommendations for one user.
The approach that was taken during the initial implementation allowed for easy removal of previously 'seen items'.
We changed the size of the affinity matrix that is used for the calculations, by only keeping the rows for the users in the test set, as follows:
https://github.com/microsoft/recommenders/blob/4c875df2d0b6b3ddad70a49a4353c5de19c28e61/reco_utils/recommender/sar/sar_singlenode.py#L307
To be able to do the same thing with the seen-items matrix, we needed to reshape it to reflect the affinity matrix, as follows:
https://github.com/microsoft/recommenders/blob/4c875df2d0b6b3ddad70a49a4353c5de19c28e61/reco_utils/recommender/sar/sar_singlenode.py#L252-L254
To follow the conventions set in the repo, we also clean up the
seen-variable during model fitting.https://github.com/microsoft/recommenders/blob/4c875df2d0b6b3ddad70a49a4353c5de19c28e61/reco_utils/recommender/sar/sar_singlenode.py#L260-L261
This then allows us to do the following, when the
remove_seenarg isTrue:https://github.com/microsoft/recommenders/blob/4c875df2d0b6b3ddad70a49a4353c5de19c28e61/reco_utils/recommender/sar/sar_singlenode.py#L314-L316
The resulting matrix then has all previously seen items set to
-np.inf. The result gets cleaned up, as follows (not part of our PR):https://github.com/microsoft/recommenders/blob/4c875df2d0b6b3ddad70a49a4353c5de19c28e61/reco_utils/recommender/sar/sar_singlenode.py#L457-L458
Performance
The performance improvement seen is significant, as summarized in the following table:
This analysis was done on our production dataset of approx. 500K rows.
We have not investigated the difference in training time.
Related Issues
#907
Checklist: