Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add test for vector sync #629

Merged
merged 3 commits into from
Aug 9, 2023
Merged

Add test for vector sync #629

merged 3 commits into from
Aug 9, 2023

Conversation

thejumpman2323
Copy link
Contributor

@thejumpman2323 thejumpman2323 commented Aug 5, 2023

Description

This pr intends to add tests for copy vectors which implicitly test vector database sync with CDC

Resolves:
#632
#388

Related Issues

Checklist

  • Is this code covered by new or existing unit tests or integration tests?
  • Did you run make test successfully?
  • Do new classes, functions, methods and parameters all have docstrings?
  • Were existing docstrings updated, if necessary?
  • Was external documentation updated, if necessary?

Additional Notes or Comments

@thejumpman2323 thejumpman2323 requested review from nenb and blythed August 5, 2023 20:08
@thejumpman2323 thejumpman2323 self-assigned this Aug 5, 2023
for d in data:
dict_d = d.to_dict()
vector = dict_d['vector']
if isinstance(vector, torch.Tensor):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@blythed Lancedb do not support torch tensors, hence I had to convert explicitly
let me know if the above seems alrite to you

@codecov-commenter
Copy link

codecov-commenter commented Aug 5, 2023

Codecov Report

Patch coverage: 86.30% and project coverage change: +0.61% 🎉

Comparison is base (ad0b706) 79.97% compared to head (f0cd5df) 80.59%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #629      +/-   ##
==========================================
+ Coverage   79.97%   80.59%   +0.61%     
==========================================
  Files          80       80              
  Lines        4865     4922      +57     
==========================================
+ Hits         3891     3967      +76     
+ Misses        974      955      -19     
Files Changed Coverage Δ
superduperdb/db/mongodb/cdc.py 83.37% <82.75%> (+7.05%) ⬆️
superduperdb/vector_search/inmemory.py 98.03% <100.00%> (ø)
superduperdb/vector_search/lancedb_client.py 88.04% <100.00%> (+4.09%) ⬆️

... and 3 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@thejumpman2323 thejumpman2323 merged commit 6bc0078 into superduper-io:main Aug 9, 2023
@thejumpman2323 thejumpman2323 deleted the tests/extend_cdc_tests branch August 9, 2023 08:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants