Skip to content

Commit

Permalink
new: add gpu support for fastembed, add fastembed providers (#612)
Browse files Browse the repository at this point in the history
* new: add gpu support for fastembed, add fastembed providers

* new: update fastembed, add fastembed-gpu, add type alias for onnx providers

* new: update readme

* fix: fix poetry.lock
  • Loading branch information
joein authored May 3, 2024
1 parent 6df7324 commit 3bdb6f6
Show file tree
Hide file tree
Showing 5 changed files with 485 additions and 381 deletions.
17 changes: 17 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,23 @@ search_result = client.query(
print(search_result)
```

FastEmbed can also utilise GPU for faster embeddings. To enable GPU support, install
```bash
pip install 'qdrant-client[fastembed-gpu]'
```

```python
from qdrant_client import QdrantClient

# Initialize the client
client = QdrantClient(":memory:") # or QdrantClient(path="path/to/db")
client.set_model(client.DEFAULT_EMBEDDING_MODEL, providers=["CUDAExecutionProvider", "CPUExecutionProvider"])
```

> Note: `fastembed-gpu` and `fastembed` are mutually exclusive. You can only install one of them.
>
> If you previously installed `fastembed`, you might need to start from a fresh environment to install `fastembed-gpu`.
## Connect to Qdrant server

To connect to Qdrant server, simply specify host and port:
Expand Down
Loading

0 comments on commit 3bdb6f6

Please sign in to comment.