Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

evaluation module crashes #149

Closed
kolomietsdv opened this issue Aug 20, 2018 · 2 comments
Closed

evaluation module crashes #149

kolomietsdv opened this issue Aug 20, 2018 · 2 comments

Comments

@kolomietsdv
Copy link

Hello!
First of all- thank you for your work!
I trying to use FaissAlternatingLeastSquares model and evaluate it checking map and precision metrics from your evaluate module.
While it works good with cpu model it always crashes the jupyter kernels in gpu mode spilling an error
"Faiss assertion 'k <= 1024' failed in void faiss::gpu::IVFFlat::query(faiss::gpu::Tensor<float, 2, true>&, int, int, faiss::gpu::Tensor<float, 2, true>&, faiss"
I use all default parameters and k=23, (rank=32, alpha=4, regularization=4, 10 iterations)
Can you check the possibility of measuring metrics on FaissALS, Im not good and programming, so I couldnt figured out the problem...
Thank you.

@XiangYongzhou
Copy link

Hi there,
Please look at the code in approximate_als.py
line 383 - line 395
implicit is filtering liked item from recommendation result.

 user = self._user_factor(userid, user_items, recalculate_user)

        # calculate the top N items, removing the users own liked items from
        # the results
        liked = set(user_items[userid].indices)
        if filter_items:
            liked.update(filter_items)
        count = N + len(liked)

if count size is greater than 1024. It will exceeds faiss GPU limitation.
I work around the issue by using CPU version if the size is greater than 1024.
You can also control the size of input query as well.

benfred added a commit that referenced this issue Aug 28, 2018
Querying a GPU Faiss index and asking for more than 1024 results caused an
assert that kills the program. This most frequently happens when trying
to recommend items for a user that has more than 1K liked items.
(see #149 )

Fix by falling back to the exact CPU version in this case.
@benfred
Copy link
Owner

benfred commented Aug 28, 2018

@XiangYongzhou Nice catch!

I've fixed up the faiss/gpu code here e4817ab to fall back to using the exact CPU version if we are asking for more than 1K items. This should fix the problem here .

This will be in the next release, but you can pull from master here to get this fix in the meantime

@benfred benfred closed this as completed Aug 28, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants