superboogaV2 requirements adjustment for issue #4307

[mbuc]

• I have read the Contributing guidelines.

Workaround for issue loading superboogav2 extension

Issue

When activating the superboogav2 extension, after installing the needed packages via extensions/requirements.txt, the following error message and stack trace occurs in the console when launching ooba with the extension enabled:

(expand) Failed to load the extension "superboogav2" due to error in chromadb --> pydantic

06:43:27-618399 INFO     Loading the extension "superboogav2"
06:43:27-623398 ERROR    Failed to load the extension "superboogav2".
Traceback (most recent call last):
  File "G:\AI\LLMs\text-generation-webui\modules\extensions.py", line 37, in load_extensions
    exec(f"import extensions.{name}.script")
  File "<string>", line 1, in <module>
  File "G:\AI\LLMs\text-generation-webui\extensions\superboogav2\script.py", line 20, in <module>
    from .chromadb import make_collector
  File "G:\AI\LLMs\text-generation-webui\extensions\superboogav2\chromadb.py", line 2, in <module>
    import chromadb
  File "G:\AI\LLMs\text-generation-webui\installer_files\env\Lib\site-packages\chromadb\__init__.py", line 1, in <module>
    import chromadb.config
  File "G:\AI\LLMs\text-generation-webui\installer_files\env\Lib\site-packages\chromadb\config.py", line 1, in <module>
    from pydantic import BaseSettings
  File "G:\AI\LLMs\text-generation-webui\installer_files\env\Lib\site-packages\pydantic\__init__.py", line 363, in __getattr__
    return _getattr_migration(attr_name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\AI\LLMs\text-generation-webui\installer_files\env\Lib\site-packages\pydantic\_migration.py", line 296, in wrapper
    raise PydanticImportError(
pydantic.errors.PydanticImportError: `BaseSettings` has been moved to the `pydantic-settings` package. See https://docs.pydantic.dev/2.5/migration/#basesettings-has-moved-to-pydantic-settings for more details.

For further information visit https://errors.pydantic.dev/2.5/u/import-error

Root cause:

The package pydantic updated a while back as can be seen here. However it's the dependency chromadb that is causing the thrown error. The version set to be installed (chromadb==0.3.18) is incompatible with more recent versions of pydantic, as discussed in their associated issue below.

Separate incidents:

• The open issue mentioned: SuperboogaV2-chromadb/pydantic BUG #4307

• Recent comment on the original PR discussion for superboogav2: Supercharging superbooga #3272 (comment)

Contained solution

This comment on #4307 suggests locking pydantic to version 1.10.12, which works to get the extension to load. Others' comments and reactions and my personal testing (launching the extension and minimal use) seem to resolve the issue.

This PR simply locks the package requirement to 1.10.12. Note that this is the same version that the base requirements.txt for ooba was locked to (see next note).

Notes

Context within this repo, text-generation-webui

• Previously, the base requirements.txt for ooba was locked to this same version ever since 66d5cab
  • This was the case until Bump to latest gradio (3.47) #4258 when requirements changed a lot and pydantic was removed.
  • It seems to not have been mentioned or modified since that time.
  • Note, a small file from the openai extension is also using the same deprecated functionality that raises the exception in the chromadb package:
    

    text-generation-webui/extensions/openai/typing.py

    Line 5 in 4b25acf

    from pydantic import BaseModel, Field

Thoughts

Not sure why chromadb is locked to 0.3.18 in the original commit. Any idea @HideLord ? Is it just because superbooga 1 did so?

• chromadb did eventually patch to work with pydantic v2 - closing out their similar bug report which is what led me to SuperboogaV2-chromadb/pydantic BUG #4307.
  • So if chromadb is just locked there because it was what was used at the time, it would be worth fully testing the current version with the extension, then removing this separate and possibly conflicting dependency from this file altogether.

But wait! The latest gradio dependency does use a 2.0+ version of chromadb in their requirements.txt! However, I'm not sure about this repo's current dependency (gradio==3.50.*) or any repercussions - there are no issues so far in my testing, but it is certainly worth noting.

• This makes this better as a temporary patch for users of the extension - there seems to be good reason to update the extensions to not use outdated chromadb, and to follow whatever gradio uses. Therefore I will not be disappointed if this PR is rejected - instead this will hopefully serve as a good reference for now.

[oobabooga]

Why not just update chromadb? Could you try seeing if it works with the latest version?

On a side note, are you on Windows or Linux? Last time I checked, chromadb used to not install successfully on Windows. I don't know if that has changed.

[sandyis]

Why not just update chromadb? Could you try seeing if it works with the latest version?

On a side note, are you on Windows or Linux? Last time I checked, chromadb used to not install successfully on Windows. I don't know if that has changed.

@oobabooga i tried doing that basically in the extensions/superboogav2/requirements.txt removed ==0.3.18 that did install the chromadb chromadb-0.4.22

1. but that created below error
2. then tried following SuperboogaV2-chromadb/pydantic BUG #4307, that totally broke TGW
3. now doing a clean install
4. below logs form running pip install -r extensions/superboogav2/requirements.txt --upgrade after removing ==0.3.18

Successfully installed Mako-1.3.0 alembic-1.13.1 asgiref-3.7.2 backoff-2.2.1 bcrypt-4.1.2 blis-0.7.11 build-1.0.3 catalogue-2.0.10 chroma-hnswlib-0.7.3 chromadb-0.4.22 cloudpathlib-0.16.0 colorlog-6.8.0 confection-0.1.4 cymem-2.0.8 deprecated-1.2.14 flatbuffers-23.5.26 googleapis-common-protos-1.62.0 graphviz-0.20.1 greenlet-3.0.3 httptools-0.6.1 icecream-2.1.3 importlib-metadata-6.11.0 kubernetes-28.1.0 langcodes-3.3.0 lxml-4.9.2 mmh3-4.0.1 monotonic-1.6 murmurhash-1.0.10 onnxruntime-1.16.3 opentelemetry-api-1.22.0 opentelemetry-exporter-otlp-proto-common-1.22.0 opentelemetry-exporter-otlp-proto-grpc-1.22.0 opentelemetry-instrumentation-0.43b0 opentelemetry-instrumentation-asgi-0.43b0 opentelemetry-instrumentation-fastapi-0.43b0 opentelemetry-proto-1.22.0 opentelemetry-sdk-1.22.0 opentelemetry-semantic-conventions-0.43b0 opentelemetry-util-http-0.43b0 optuna-3.5.0 overrides-7.4.0 pandas-2.0.3 posthog-2.4.2 preshed-3.0.9 pulsar-client-3.4.0 pypika-0.48.9 pyproject_hooks-1.0.0 pytextrank-3.2.5 python-dotenv-1.0.0 sentence_transformers-2.2.2 smart-open-6.4.0 spacy-3.7.2 spacy-legacy-3.0.12 spacy-loggers-1.0.5 sqlalchemy-2.0.25 srsly-2.4.8 tenacity-8.2.3 thinc-8.2.2 typer-0.9.0 urllib3-1.26.18 uvloop-0.19.0 wasabi-1.1.2 watchfiles-0.21.0 weasel-0.3.4 websocket-client-1.7.0 wrapt-1.16.0 zipp-3.17.0

Traceback (most recent call last): File "/home/test/text-generation-webui-main/modules/extensions.py", line 46, in load_extensions extension.setup() File "/home/test/text-generation-webui-main/extensions/superboogav2/script.py", line 35, in setup collector = make_collector() ^^^^^^^^^^^^^^^^ File "/home/test/text-generation-webui-main/extensions/superboogav2/chromadb.py", line 376, in make_collector return ChromaCollector(SentenceTransformerEmbedder()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/test/text-generation-webui-main/extensions/superboogav2/chromadb.py", line 101, in __init__ self.collection = self.chroma_client.create_collection(name="context", embedding_function=self.embedder.embed) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/test/text-generation-webui-main/installer_files/env/lib/python3.11/site-packages/chromadb/api/client.py", line 198, in create_collection return self._server.create_collection( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/test/text-generation-webui-main/installer_files/env/lib/python3.11/site-packages/chromadb/telemetry/opentelemetry/__init__.py", line 127, in wrapper return f(*args, **kwargs) ^^^^^^^^^^^^^^^^^^ File "/home/test/text-generation-webui-main/installer_files/env/lib/python3.11/site-packages/chromadb/api/segment.py", line 190, in create_collection return Collection( ^^^^^^^^^^^ File "/home/test/text-generation-webui-main/installer_files/env/lib/python3.11/site-packages/chromadb/api/models/Collection.py", line 87, in __init__ validate_embedding_function(embedding_function) File "/home/test/text-generation-webui-main/installer_files/env/lib/python3.11/site-packages/chromadb/api/types.py", line 196, in validate_embedding_function raise ValueError( ValueError: Expected EmbeddingFunction.__call__ to have the following signature: odict_keys(['self', 'input']), got odict_keys(['self', 'args', 'kwargs']) Please see https://docs.trychroma.com/embeddings for details of the EmbeddingFunction interface. Please note the recent change to the EmbeddingFunction interface: https://docs.trychroma.com/migration#migration-to-0416---november-7-2023

[TheDarkTrumpet]

Why not just update chromadb? Could you try seeing if it works with the latest version?

On a side note, are you on Windows or Linux? Last time I checked, chromadb used to not install successfully on Windows. I don't know if that has changed.

My personal feeling on this is we should get the requirements patch for a known good version of chroma to work, then test out a separate patch that upgrades chromadb to the newest version. Since superboogav2 was likely tested or in use for awhile, it should be assumed that it works as intended as originally developed. Fixing the dependency, then upgrading is safer than upgrading and messing over people.

We have have a good deal of duplicates sitting around for issues reporting this. I really feel like we should get these cleaned up, I don't mind progressing further on the Chroma testing front for the new version.

[SpekFox]

pip uninstall chromadb -y && pip cache purge && pip install chromadb==0.3.18

[oobabooga]

Closing in favor of #5656