Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix normalize_embedding using numba #2347

Merged
merged 3 commits into from
Mar 22, 2022
Merged

Conversation

tstadel
Copy link
Member

@tstadel tstadel commented Mar 22, 2022

Currently when passing a single vector to DocumentStore.normalize_embedding an error is thrown if numba is installed. normalize_embedding expects 1-dimensional or 2-dimentional vectors, selecting an appropriate code path for each. However numba needs to compile the whole function. Thus the 2-dim path is also being compiled with 1-dim args resulting in a TypingError as ascontigousarray() only exists for ndarrays and not for floats:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/tstad/git/haystack/haystack/document_stores/weaviate.py", line 477, in write_documents
    self.normalize_embedding(vector)
  File "/home/tstad/miniconda3/envs/haystack-dev/lib/python3.7/site-packages/numba/core/dispatcher.py", line 468, in _compile_for_args
    error_rewrite(e, 'typing')
  File "/home/tstad/miniconda3/envs/haystack-dev/lib/python3.7/site-packages/numba/core/dispatcher.py", line 409, in error_rewrite
    raise e.with_traceback(None)
numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
No implementation of function Function(<built-in function ascontiguousarray>) found for signature:
 
 >>> ascontiguousarray(float32)
 

Impact:
normalized_embedding is used by FAISSDocumentStore, Milvus1DocumentStore, PineconeDocumentStore and WeaviateDocumentStore when using cosine similarity.

Proposed changes:

  • split "dynamic-typed" normalize_embedding into "static-typed" normalize_embedding_1D and normalize_embedding_2D, so numba can deal with it.

Status (please check what you already did):

  • First draft (up for discussions & feedback)
  • Final code

Fixes the bad one in #2346

Copy link
Member

@julian-risch julian-risch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 👍 If there were more vectors than just two in emb in _normalize_embedding_2D() then we could even consider @njit(parallel=True, fastmath=True) to get an additional speed boost: https://numba.pydata.org/numba-doc/latest/user/performance-tips.html It's definitely not a bottleneck though. 😄 Maybe a nice Hacky Friday project to find performance bottlenecks and speed them up...

@tstadel tstadel merged commit 851fe1c into master Mar 22, 2022
@tstadel tstadel deleted the fix_normalize_embedding_numba branch March 22, 2022 22:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants