Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Freeze pretrained weights #467

Closed
phiweger opened this issue Jul 7, 2021 · 4 comments
Closed

Freeze pretrained weights #467

phiweger opened this issue Jul 7, 2021 · 4 comments

Comments

@phiweger
Copy link

phiweger commented Jul 7, 2021

Following up on #341 : Is it possible to freeze the pretrained weights, so that they are not changed during training?

@benfred
Copy link
Owner

benfred commented Sep 9, 2021

Is the use case here incremental retraining ? I'd like to add support for that at some point - though I was thinking of a different API than freezing items you don't want to update

@phiweger
Copy link
Author

phiweger commented Sep 9, 2021

@benfred yes, I'd like to train say item embeddings using some other method (like a variational autoencoder) and then use these latent features in the recommender, but not modify them during training.

@slowwavesleep
Copy link

slowwavesleep commented Dec 28, 2021

It seems that I came up with same idea as @phiweger.

I think adding something like if not self.freeze_item_factors before this line would do the trick. Which would make Alternating Least Squares not alternating any more, I guess? But still it would be nice to be able to use an existing library for this. For example, to be able to easily evaluate the effect of using existing item factors.

I do realize though, that it would take quite a bit more more work than that to implement this feature.

@benfred
Copy link
Owner

benfred commented Jan 25, 2022

There is a PR here which should make this possible #527 (as well as let you incrementally retrain models on subsets of users/items).

As an example given a user_items sparse matrix containing interactions, and a existing_item_factors dense matrix containing precalculated item factors:

# Train an ALS model with pre-existing item factors, but calculating user factors
model = AlternatingLeastSquares()
model.item_factors = existing_item_factors
userids = np.arange(user_items.shape[0])
model.partial_fit_users(userids, user_items)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants