Add vector search service #1248

thejumpman2323 · 2023-11-06T20:28:36Z

Description

fix #1242

Related Issues

Checklist

Is this code covered by new or existing unit tests or integration tests?
Did you run make test successfully?
Do new classes, functions, methods and parameters all have docstrings?
Were existing docstrings updated, if necessary?
Was external documentation updated, if necessary?

Additional Notes or Comments

superduperdb/base/config.py

superduperdb/base/build.py

blythed · 2023-11-08T07:53:43Z

superduperdb/vector_search/server/app.py

+# --------------- Create vector search serve -----------------
+def serve():
+    # TODO: configurable
+    uvicorn.run(app, host="0.0.0.0", port=8000, reload=False)


Can we add a configuration?

what to add in Config: class

vector_search_service as a block ?

lance://localhost:8000 or inmemory://localhost:8000 (this at the bottom level)

This would make it consistent with the other things, and open to add new configurations.

superduperdb/base/config.py

superduperdb/misc/server.py

superduperdb/base/config.py

superduperdb/misc/server.py

test/integration/test_vector_search_server.py

jieguangzhou

Maybe we could add the test cases for mode == 'production' and mode != 'production' in the future

LGTM, a few changes, do not affect the merging

superduperdb/components/listener.py

superduperdb/misc/server.py

codecov-commenter · 2023-11-10T13:53:13Z

Codecov Report

Attention: 104 lines in your changes are missing coverage. Please review.

Comparison is base (34830a7) 80.33% compared to head (47cc51c) 80.61%.
Report is 28 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1248      +/-   ##
==========================================
+ Coverage   80.33%   80.61%   +0.27%     
==========================================
  Files          95      103       +8     
  Lines        6602     6919     +317     
==========================================
+ Hits         5304     5578     +274     
- Misses       1298     1341      +43

Files	Coverage Δ
superduperdb/__main__.py	`71.42% <100.00%> (ø)`
superduperdb/backends/mongodb/cdc/base.py	`77.77% <100.00%> (ø)`
superduperdb/backends/mongodb/cdc/listener.py	`82.78% <100.00%> (ø)`
superduperdb/base/config.py	`100.00% <100.00%> (ø)`
superduperdb/base/datalayer.py	`71.73% <100.00%> (+3.56%)`	⬆️
superduperdb/components/model.py	`93.66% <100.00%> (+11.00%)`	⬆️
superduperdb/ext/openai/model.py	`95.40% <ø> (-0.05%)`	⬇️
superduperdb/ext/torch/model.py	`76.50% <ø> (-0.31%)`	⬇️
superduperdb/misc/superduper.py	`76.25% <ø> (ø)`
superduperdb/vector_search/in_memory.py	`98.24% <100.00%> (+12.28%)`	⬆️
... and 13 more

... and 11 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

thejumpman2323 self-assigned this Nov 6, 2023

thejumpman2323 requested a review from blythed November 6, 2023 20:28

blythed reviewed Nov 7, 2023

View reviewed changes

superduperdb/base/config.py Outdated Show resolved Hide resolved

blythed reviewed Nov 7, 2023

View reviewed changes

superduperdb/base/build.py Outdated Show resolved Hide resolved