cohere[patch]: Add cohere tools agent #19602

harry-cohere · 2024-03-26T17:27:40Z

Description: Adds a cohere tools agent and related notebook.

vercel · 2024-03-26T17:36:52Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
langchain	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Mar 28, 2024 1:31am

libs/partners/cohere/docs/cohere_agent.ipynb

efriis · 2024-03-26T20:08:25Z

libs/partners/cohere/langchain_cohere/chat_models.py

-            "token_count": response.token_count,
+            "is_search_required": response.is_search_required,
+            "generation_id": response.generation_id,
+            "tool_calls": response.tool_calls,


let's factor this into the same format as other models

tool_calls = [ { "id": "some id string", "function": { "name": ..., "arguments": "json serialized dict or args {}" } } ]

Without doing this, this will also mess with serialization

This is implemented, thank you!

efriis · 2024-03-26T20:09:15Z

libs/partners/cohere/langchain_cohere/cohere_agent.py

+        )
+
+
+class CohereToolsAgentOutputParser(


Matching the format above will allow ChatCohere to be used with OpenAIToolsAgentOutputParser!

so we can delete this

That sounds good, however, we'd like the flexibility (and naming) in the partner package. We're only 'exporting' create_cohere_tools_agent, and if there are plans to refactor some of the tools interfaces in langchain we could later update this. Would that work for you?

Sounds good. Added an integration test to confirm interface matches at least, and it's failing :(

Could you get that passing, then it looks good

sure thing!

I think I fixed a couple of bugs, but I also wanted to double check that the behaviour is correct as I changed the tests because they looked wrong to me:

In the non-stream test I updated it to use invoke instead of stream

What's the expected kwargs.tool_calls value when tools are bound to the request but the model indicates to not call any tool? The tests previously had this as "tool_calls is not present", but I think it should be "tool_calls is an empty sequence". Happy to change this though!

efriis · 2024-03-27T20:53:21Z

libs/partners/cohere/tests/integration_tests/test_chat_models.py

@harry-cohere these are the tests that should work on the chat model

efriis · 2024-03-28T00:04:00Z

libs/partners/cohere/tests/integration_tests/test_chat_models.py

@@ -121,7 +121,7 @@ class Person(BaseModel):
    }

    # where it doesn't call the tool
-    strm = tool_llm.stream("What is 2+2?")
+    strm = llm.stream("What is 2+2?")


This was right before! Does cohere's tool calling always force a tool call, even if the query isn't relevant to the provided tool?

The default behaviour is to indicate that the provided tools are irrelevant. In these cases an empty tool_calls is returned. I can fix the implementation so that it doesn't populate kwargs.tool_calls in this case!

efriis · 2024-03-28T00:04:11Z

libs/partners/cohere/tests/integration_tests/test_chat_models.py

@@ -80,7 +80,7 @@ class Person(BaseModel):
    tool_llm = llm.bind_tools([Person])

    # where it calls the tool
-    result = tool_llm.stream("Erick, 27 years old")
+    result = tool_llm.invoke("Erick, 27 years old")


efriis · 2024-03-28T00:14:21Z

libs/partners/cohere/tests/integration_tests/test_chat_models.py

+def test_streaming_tool_call_no_tool_calls() -> None:
+    llm = ChatCohere(temperature=0)
+
+    class Person(BaseModel):
+        name: str
+        age: int
+
+    tool_llm = llm.bind_tools([Person])
+
+    # where it doesn't call the tool
+    strm = tool_llm.stream("What is 2+2?")
+    acc: Any = None
+    for chunk in strm:
+        assert isinstance(chunk, AIMessageChunk)
+        acc = chunk if acc is None else acc + chunk
+    assert acc.content != ""
+    assert "tool_calls" not in acc.additional_kwargs


@harry-cohere this is the test that is failing currently.

If it's a model-level problem, I'm happy to skip it.

But currently, this call returns an AIMessage with

content = ""

tool calls = []

So basically an empty response :(

DMd you! In short, "don't call any tools because they're all irrelevant" is expressed as empty content and an empty tool_calls array. Happy to change to whatever suits the framework! :)

Sounds good. I'll mark this test as "skip" for now.

As a heads up, this will make cohere models perform poorly in agents. There's lots of cases, where e.g. you might give an agent a tool like calculator, but still want the agent to be able to respond to queries like "what is the capital of france"

**Description**: Adds a cohere tools agent and related notebook. --------- Co-authored-by: BeatrixCohere <[email protected]> Co-authored-by: Erick Friis <[email protected]>

BeatrixCohere added 3 commits March 26, 2024 07:51

Add tools agent

27a24c1

Upadte

ada3f8c

Lint and format

f2ea3cd

efriis added the partner label Mar 26, 2024

efriis self-assigned this Mar 26, 2024

harry-cohere mentioned this pull request Mar 26, 2024

langchain_cohere: Add cohere tools agent #19555

Closed

change types to avoid langchain import

802a003

harry-cohere added 2 commits March 26, 2024 17:39

fix lint

e52ea14

Bump version number

88e3402

harry-cohere commented Mar 26, 2024

View reviewed changes

libs/partners/cohere/docs/cohere_agent.ipynb Outdated Show resolved Hide resolved

fix tests

c9d31ea

harry-cohere marked this pull request as ready for review March 26, 2024 17:48

dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. Ɑ: agent Related to agents module 🔌: cohere Primarily related to Cohere integrations 🤖:enhancement A large net-new component, integration, or chain. Use sparingly. The largest features labels Mar 26, 2024

harry-cohere changed the title ~~langchain_cohere: Add cohere tools agent #19555~~ langchain_cohere: Add cohere tools agent Mar 26, 2024

efriis reviewed Mar 26, 2024

View reviewed changes

harry-cohere added 3 commits March 26, 2024 20:18

Merge branch 'master' into beatrix/AddToolsAgent

7172c50

format Cohere tool_calls

f72b999

lint

7b3a0e5

dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Mar 26, 2024

harry-cohere added 6 commits March 26, 2024 22:22

better conversion to cohere tool

d487a89

fix lint

81a1092

fix agent parsing

d47cb7e

fix lint

95d1490

Remove directly_answer

64cef07

lint

20a7d8a

bump version number

6b9afaf

vercel bot deployed to Preview March 27, 2024 17:50 View deployment

fix tool_call conversion to json schema format

0bec1cf

vercel bot deployed to Preview March 27, 2024 19:39 View deployment

efriis added 2 commits March 27, 2024 13:50

x

e3c7c5b

invoke test

a6cd384

efriis reviewed Mar 27, 2024

View reviewed changes

vercel bot deployed to Preview March 27, 2024 21:05 View deployment

harry-cohere added 2 commits March 27, 2024 23:40

Add conversion for BaseModel -> Cohere tool

99b3a49

change/fix tests, improve types

ef4a1f5

efriis reviewed Mar 28, 2024

View reviewed changes

vercel bot deployed to Preview March 28, 2024 00:05 View deployment

efriis added 2 commits March 27, 2024 17:06

save doc for future test

276da7e

split integration test to separate

871334c

dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:XL This PR changes 500-999 lines, ignoring generated files. labels Mar 28, 2024

efriis reviewed Mar 28, 2024

View reviewed changes

don't omit tool_calls when empty

97afa7c

dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Mar 28, 2024

fix lint

579cd43

vercel bot deployed to Preview March 28, 2024 00:35 View deployment

xfail agent case

0a08927

vercel bot deployed to Preview March 28, 2024 01:31 View deployment

efriis changed the title ~~langchain_cohere: Add cohere tools agent~~ cohere[patch]: Add cohere tools agent Mar 28, 2024

efriis merged commit 3685f8c into langchain-ai:master Mar 28, 2024
20 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cohere[patch]: Add cohere tools agent #19602

cohere[patch]: Add cohere tools agent #19602

harry-cohere commented Mar 26, 2024

vercel bot commented Mar 26, 2024 •

edited

Loading

efriis Mar 26, 2024

efriis Mar 26, 2024

harry-cohere Mar 27, 2024

efriis Mar 26, 2024

efriis Mar 26, 2024

harry-cohere Mar 27, 2024

efriis Mar 27, 2024

harry-cohere Mar 27, 2024

harry-cohere Mar 27, 2024

efriis Mar 27, 2024

efriis Mar 28, 2024

harry-cohere Mar 28, 2024

efriis Mar 28, 2024

efriis Mar 28, 2024

harry-cohere Mar 28, 2024

efriis Mar 28, 2024

cohere[patch]: Add cohere tools agent #19602

cohere[patch]: Add cohere tools agent #19602

Conversation

harry-cohere commented Mar 26, 2024

vercel bot commented Mar 26, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vercel bot commented Mar 26, 2024 •

edited

Loading