-
Notifications
You must be signed in to change notification settings - Fork 612
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
evaluation module crashes #149
Comments
Hi there, user = self._user_factor(userid, user_items, recalculate_user)
# calculate the top N items, removing the users own liked items from
# the results
liked = set(user_items[userid].indices)
if filter_items:
liked.update(filter_items)
count = N + len(liked) if count size is greater than 1024. It will exceeds faiss GPU limitation. |
Querying a GPU Faiss index and asking for more than 1024 results caused an assert that kills the program. This most frequently happens when trying to recommend items for a user that has more than 1K liked items. (see #149 ) Fix by falling back to the exact CPU version in this case.
@XiangYongzhou Nice catch! I've fixed up the faiss/gpu code here e4817ab to fall back to using the exact CPU version if we are asking for more than 1K items. This should fix the problem here . This will be in the next release, but you can pull from master here to get this fix in the meantime |
Hello!
First of all- thank you for your work!
I trying to use FaissAlternatingLeastSquares model and evaluate it checking map and precision metrics from your evaluate module.
While it works good with cpu model it always crashes the jupyter kernels in gpu mode spilling an error
"Faiss assertion 'k <= 1024' failed in void faiss::gpu::IVFFlat::query(faiss::gpu::Tensor<float, 2, true>&, int, int, faiss::gpu::Tensor<float, 2, true>&, faiss"
I use all default parameters and k=23, (rank=32, alpha=4, regularization=4, 10 iterations)
Can you check the possibility of measuring metrics on FaissALS, Im not good and programming, so I couldnt figured out the problem...
Thank you.
The text was updated successfully, but these errors were encountered: