Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Support index building from host dataset for IVF-FLAT #1178

Open
tfeher opened this issue Jan 25, 2023 · 1 comment
Open

[FEA] Support index building from host dataset for IVF-FLAT #1178

tfeher opened this issue Jan 25, 2023 · 1 comment
Labels
feature request New feature or request

Comments

@tfeher
Copy link
Contributor

tfeher commented Jan 25, 2023

Is your feature request related to a problem? Please describe.
Currently only device datasets are allowed for extending the index (which is part of the build process). The goal is to accept input data from the host, so that we have more free device memory for building the index.

Describe the solution you'd like
Similar to IVF-PQ, one could process the input data batch wise, and copy that to GPU memory in smaller batches.

@tfeher tfeher added the feature request New feature or request label Jan 25, 2023
@wphicks
Copy link
Contributor

wphicks commented Jan 8, 2024

Just ran into this issue with the knowhere integration, as well. Would be a good thing to fix for consistency across ANN methods.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants