feat: support for OpenAI/Anthropic `tools` APIs #300

ErikBjare · 2024-12-03T08:12:58Z

Rebased and testing #280 by @jrmi

Important

Adds support for OpenAI and Anthropic tools APIs with configurable tool parsing methods and updates related tests and dependencies.

Behavior:
- Adds tool_format parameter to chat() and step() in chat.py to support different tool parsing methods ('markdown', 'xml', 'tool').
- Updates CLI in cli.py to include --tool-format option for specifying tool parsing method.
- Modifies reply() in llm/__init__.py to handle tools based on tool_format.
Tools:
- Updates ToolSpec and ToolUse in tools/base.py to support new tool_format.
- Adds examples() functions in various tool files to generate examples based on tool_format.
- Updates tool execution logic in tools/*.py to handle tool_format.
Tests:
- Adds tests in test_tool_use.py and test_prompt_tools.py to verify tool output formatting.
- Updates existing tests in test_cli.py to cover new --tool-format option.
Misc:
- Adds json-repair dependency in pyproject.toml for JSON handling.
- Minor refactoring and code cleanup in several files.

^{This description was created by}^{for 1647814. It will automatically update as commits are pushed.}

ellipsis-dev

👍 Looks good to me! Reviewed everything up to 67f8f1c in 1 minute and 19 seconds

More details

Looked at 2517 lines of code in 33 files
Skipped 1 files when reviewing.
Skipped posting 1 drafted comments based on config settings.

1. pyproject.toml:58

Draft comment:
Ensure 'json-repair' is included in the dependencies to avoid runtime errors.
Reason this comment was not posted:
Comment did not seem useful.

Workflow ID: wflow_zuIZ3VgodsYA1onL

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

ellipsis-dev

👍 Looks good to me! Incremental review on 988020e in 46 seconds

More details

Looked at 249 lines of code in 6 files
Skipped 0 files when reviewing.
Skipped posting 5 drafted comments based on config settings.

1. gptme/llm/llm_anthropic.py:15

Draft comment:
The import PromptCachingBetaToolParam is unused and can be removed to clean up the code.
Reason this comment was not posted:
Confidence changes required: 10%
The import statement for PromptCachingBetaToolParam is not used in the file gptme/llm/llm_anthropic.py. It should be removed to clean up the code.

2. gptme/llm/llm_anthropic.py:105

Draft comment:
Avoid using print statements for logging. Consider using logger.debug or another appropriate logging level instead.
Reason this comment was not posted:
Confidence changes required: 50%
The print statement in the stream function is not necessary for production code and should be removed or replaced with a logging statement.

3. gptme/server/api.py:24

Draft comment:
Reorder imports to follow PEP8 guidelines. Group get_model with other imports from gptme.llm.
Reason this comment was not posted:
Confidence changes required: 10%
The import order in gptme/server/api.py is not following PEP8 guidelines. The import of get_model should be grouped with other imports from gptme.llm.

4. gptme/server/api.py:178

Draft comment:
Avoid using print statements for logging. Consider using logger.debug or another appropriate logging level instead.
Reason this comment was not posted:
Confidence changes required: 50%
The print statement in the generate function is not necessary for production code and should be removed or replaced with a logging statement.

5. gptme/tools/python.py:18

Draft comment:
The import ConfirmFunc is unused and can be removed to clean up the code.
Reason this comment was not posted:
Confidence changes required: 10%
The import statement for ConfirmFunc in gptme/tools/python.py is not used in the file. It should be removed to clean up the code.

Workflow ID: wflow_lUpuTWWpnegXDqJY

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

codecov-commenter · 2024-12-03T09:32:54Z

Codecov Report

Attention: Patch coverage is 92.73743% with 26 lines in your changes missing coverage. Please review.

Project coverage is 73.93%. Comparing base (6a3813b) to head (1647814).

✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
gptme/tools/save.py	74.19%	8 Missing ⚠️
gptme/tools/patch.py	60.00%	6 Missing ⚠️
gptme/llm/llm_openai.py	90.38%	5 Missing ⚠️
gptme/tools/base.py	98.03%	2 Missing ⚠️
gptme/tools/tmux.py	80.00%	2 Missing ⚠️
gptme/llm/__init__.py	93.33%	1 Missing ⚠️
gptme/llm/llm_anthropic.py	98.24%	1 Missing ⚠️
gptme/tools/shell.py	90.90%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #300      +/-   ##
==========================================
+ Coverage   72.52%   73.93%   +1.40%     
==========================================
  Files          65       65              
  Lines        4386     4604     +218     
==========================================
+ Hits         3181     3404     +223     
+ Misses       1205     1200       -5

Flag	Coverage Δ
anthropic/claude-3-haiku-20240307	`71.98% <80.72%> (+0.66%)`	⬆️
openai/gpt-4o-mini	`71.69% <79.60%> (+0.54%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

ellipsis-dev

👍 Looks good to me! Incremental review on 4ed70f1 in 28 seconds

More details

Looked at 82 lines of code in 2 files
Skipped 0 files when reviewing.
Skipped posting 1 drafted comments based on config settings.

1. gptme/chat.py:54

Draft comment:
The return type annotation for the chat function should be None instead of -> None:. Remove the colon after the closing parenthesis.
Reason this comment was not posted:
Confidence changes required: 50%
The return type annotation for the chat function was missing a None return type. This is a minor issue but should be corrected for clarity and consistency.

Workflow ID: wflow_okw7h830aOwz2fDV

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

ellipsis-dev

👍 Looks good to me! Incremental review on 11b74f1 in 39 seconds

More details

Looked at 122 lines of code in 3 files
Skipped 0 files when reviewing.
Skipped posting 2 drafted comments based on config settings.

1. gptme/llm/llm_anthropic.py:62

Draft comment:
Ensure that the anthropic module is available in the environment where this function is executed, as it is imported conditionally within the function.
Reason this comment was not posted:
Confidence changes required: 50%
The import statements for anthropic and openai modules are conditionally imported within functions. This is a good practice to avoid unnecessary imports when the module is not used, but it can lead to issues if the module is not available when the function is called. It's important to ensure that these modules are available in the environment where the functions are executed.

2. gptme/llm/llm_openai.py:179

Draft comment:
Ensure that the openai module is available in the environment where this function is executed, as it is imported conditionally within the function.
Reason this comment was not posted:
Confidence changes required: 50%
The import statements for anthropic and openai modules are conditionally imported within functions. This is a good practice to avoid unnecessary imports when the module is not used, but it can lead to issues if the module is not available when the function is called. It's important to ensure that these modules are available in the environment where the functions are executed.

Workflow ID: wflow_jPvuTWg7yzNsUm3X

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

- Enhance anthropic streaming to handle different chunk types - Rename global anthropic to _anthropic - Fix imports and formatting in various files Part of the tools API refactoring work.

ellipsis-dev

👍 Looks good to me! Incremental review on 1647814 in 29 seconds

More details

Looked at 34 lines of code in 1 files
Skipped 0 files when reviewing.
Skipped posting 1 drafted comments based on config settings.

1. gptme/llm/llm_openai.py:172

Draft comment:
Consider moving the imports of ChoiceDeltaToolCall and ChoiceDeltaToolCallFunction back to the top of the file to avoid repeated imports during function calls.
Reason this comment was not posted:
Confidence changes required: 50%
The import statements for ChoiceDeltaToolCall and ChoiceDeltaToolCallFunction were moved inside the stream function. This change is unnecessary and can lead to performance issues due to repeated imports during function calls. It's better to keep imports at the top of the file unless there's a specific reason to import them inside a function.

Workflow ID: wflow_lqrbgLavSVZ9iVY0

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

ErikBjare · 2024-12-03T10:49:18Z

This is passing now!

My limited local testing suggests this is good enough to merge. Any surprises will be handled in follow-up PRs.

Thanks a lot for the huge effort @jrmi! 🥇

ellipsis-dev bot reviewed Dec 3, 2024

View reviewed changes

ErikBjare and others added 9 commits December 3, 2024 11:37

feat: started working on support for OpenAI's tools API option

85dd4ae

fix: fixed tool calls

da78364

fix: basic tools API support for anthropic

f4fcf82

fix: fixed typing in llm_openai.py

b399d9e

feat: add tool support for all tools

b05524d

Fix python tool description length

957d7a1

refactor(tools-api): improve anthropic streaming and minor fixes

d501196

- Enhance anthropic streaming to handle different chunk types - Rename global anthropic to _anthropic - Fix imports and formatting in various files Part of the tools API refactoring work.

test: fixed test_tool_format_option test

8c83c49

fix: fixed lazy import of openai/anthropic to speed up start times

9ee19a0

ErikBjare force-pushed the dev/tools-api-by-jrmi branch from 11b74f1 to 9ee19a0 Compare December 3, 2024 10:39

ErikBjare changed the title ~~feat: started working on support for OpenAI's tools API option~~ feat: support for OpenAI/Anthropic tools APIs Dec 3, 2024

fix: fixed import in llm_openai.py

1647814

ellipsis-dev bot reviewed Dec 3, 2024

View reviewed changes

ErikBjare merged commit d83f723 into master Dec 3, 2024
7 checks passed

This was referenced Dec 3, 2024

Remaining issues with tools API #302

Open

Draft: Add tools api support #280

Closed

feat: implemented use of OpenAI's tools API #219

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: support for OpenAI/Anthropic `tools` APIs #300

feat: support for OpenAI/Anthropic `tools` APIs #300

ErikBjare commented Dec 3, 2024 •

edited

Loading

ellipsis-dev bot left a comment

ellipsis-dev bot left a comment

codecov-commenter commented Dec 3, 2024 •

edited

Loading

ellipsis-dev bot left a comment

ellipsis-dev bot left a comment

ellipsis-dev bot left a comment

ErikBjare commented Dec 3, 2024

feat: support for OpenAI/Anthropic tools APIs #300

feat: support for OpenAI/Anthropic tools APIs #300

Conversation

ErikBjare commented Dec 3, 2024 • edited Loading

ellipsis-dev bot left a comment

Choose a reason for hiding this comment

ellipsis-dev bot left a comment

Choose a reason for hiding this comment

codecov-commenter commented Dec 3, 2024 • edited Loading

Codecov Report

ellipsis-dev bot left a comment

Choose a reason for hiding this comment

ellipsis-dev bot left a comment

Choose a reason for hiding this comment

ellipsis-dev bot left a comment

Choose a reason for hiding this comment

ErikBjare commented Dec 3, 2024

feat: support for OpenAI/Anthropic `tools` APIs #300

feat: support for OpenAI/Anthropic `tools` APIs #300

ErikBjare commented Dec 3, 2024 •

edited

Loading

codecov-commenter commented Dec 3, 2024 •

edited

Loading