Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(forge/llm): Create BaseOpenAIProvider -> deduplicate GroqProvider & OpenAIProvider #7178

Conversation

Pwuts
Copy link
Member

@Pwuts Pwuts commented May 31, 2024

It seems to be a trend for new LLM providers to mimic the OpenAI API. The same goes for Llamafile. We can leverage this and create a common base class that implements most of the functionality for providers with OpenAI-like interfaces. This has the additional benefit of increasing maintainability and making it easier to add support for more such providers in the future.

This PR also paves the way for a clean integration of #7091.

Changes

  • Add _BaseOpenAIProvider, BaseOpenAIChatProvider, and BaseOpenAIEmbeddingProvider
  • (Re)move as much code as possible from GroqProvider and OpenAIProvider

Also:

  • Rename get_available_models() to get_available_chat_models() on BaseChatModelProvider
  • Add get_available_models() to BaseModelProvider
  • Add get_available_embedding_models() to BaseEmbeddingModelProvider
  • Move common fix_failed_parse_tries config attribute into base ModelProviderConfiguration

…deduplicate `GroqProvider` & `OpenAIProvider`

- Add `_BaseOpenAIProvider`, `BaseOpenAIChatProvider`, and `BaseOpenAIEmbeddingProvider`
- (Re)move as much code as possible from `GroqProvider` and `OpenAIProvider`

Also:
- Rename `get_available_models()` to `get_available_chat_models()` on `BaseChatModelProvider`
- Add `get_available_models()` to `BaseModelProvider`
- Add `get_available_embedding_models()` to `BaseEmbeddingModelProvider`
- Move common `fix_failed_parse_tries` config attribute into base `ModelProviderConfiguration`
@Pwuts Pwuts requested a review from a team as a code owner May 31, 2024 03:09
@Pwuts Pwuts requested review from Torantulino and Bentlybro and removed request for a team May 31, 2024 03:09
Copy link

netlify bot commented May 31, 2024

Deploy Preview for auto-gpt-docs canceled.

Name Link
🔨 Latest commit cd9a5d7
🔍 Latest deploy log https://app.netlify.com/sites/auto-gpt-docs/deploys/6659429be16f86000832747b

@Pwuts Pwuts requested review from kcze and a team May 31, 2024 03:10
Copy link

codecov bot commented May 31, 2024

Codecov Report

Attention: Patch coverage is 34.66667% with 147 lines in your changes are missing coverage. Please review.

Project coverage is 42.70%. Comparing base (738c8ff) to head (cd9a5d7).

Files Patch % Lines
forge/forge/llm/providers/_openai_base.py 23.12% 122 Missing and 1 partial ⚠️
forge/forge/llm/providers/openai.py 40.62% 19 Missing ⚠️
forge/forge/llm/providers/multi.py 60.00% 2 Missing ⚠️
forge/forge/llm/providers/schema.py 84.61% 2 Missing ⚠️
forge/forge/llm/providers/anthropic.py 83.33% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7178      +/-   ##
==========================================
+ Coverage   33.75%   42.70%   +8.95%     
==========================================
  Files          22       83      +61     
  Lines        1893     4826    +2933     
  Branches      330      657     +327     
==========================================
+ Hits          639     2061    +1422     
- Misses       1240     2673    +1433     
- Partials       14       92      +78     
Flag Coverage Δ
Linux 42.70% <34.66%> (+8.95%) ⬆️
Windows 42.74% <34.66%> (+1.74%) ⬆️
agbenchmark ?
autogpt-agent 36.02% <100.00%> (?)
forge 45.11% <34.37%> (?)
macOS 42.70% <34.66%> (+8.95%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Pwuts Pwuts changed the title refactor(forge/llm): Create BaseOpenAI(Chat|Embedding)Provider and deduplicate GroqProvider & OpenAIProvider refactor(forge/llm): Create BaseOpenAIProvider and deduplicate GroqProvider & OpenAIProvider May 31, 2024
@Pwuts Pwuts changed the title refactor(forge/llm): Create BaseOpenAIProvider and deduplicate GroqProvider & OpenAIProvider refactor(forge/llm): Create BaseOpenAIProvider -> deduplicate GroqProvider & OpenAIProvider May 31, 2024
@Pwuts Pwuts enabled auto-merge (squash) May 31, 2024 03:33
@Pwuts Pwuts disabled auto-merge May 31, 2024 03:33
@Pwuts Pwuts enabled auto-merge (squash) May 31, 2024 03:35
@Pwuts Pwuts merged commit 4e76768 into master Jun 2, 2024
29 checks passed
@Pwuts Pwuts deleted the reinier/open-1100-abstract-openaiprovider-groqprovider-to-a-common branch June 2, 2024 23:29
@Pwuts Pwuts mentioned this pull request Jun 3, 2024
9 tasks
@Pwuts Pwuts linked an issue Jun 9, 2024 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Support using other/local LLMs
2 participants