Skip to content

Commit

Permalink
docs: ollama doc update (toolcalling, install, notebook examples) (la…
Browse files Browse the repository at this point in the history 
…ngchain-ai#25549)

The new `langchain-ollama` package seems pretty well implemented, but I
noticed the docs were still outdated so I decided to fix em up a bit.

- Llama3.1 was release on 23rd of July;
https://ai.meta.com/blog/meta-llama-3-1/
- Ollama supports tool calling since 25th of July;
https://ollama.com/blog/tool-support
- LangChain Ollama partner package was released 1st of august;
https://pypi.org/project/langchain-ollama/

**Problem**: Docs note langchain-community instead of langchain-ollama

**Solution**: Update docs to
https://python.langchain.com/v0.2/docs/integrations/chat/ollama/


**Problem**: OllamaFunctions is deprecated, as noted on
[Integrations](https://python.langchain.com/v0.2/docs/integrations/chat/ollama_functions/):
This was an experimental wrapper that attempts to bolt-on tool calling
support to models that do not natively support it. The [primary Ollama
integration](https://python.langchain.com/v0.2/docs/integrations/chat/ollama/) now
supports tool calling, and should be used instead.

**Solution**: Delete old notebook from repo, update the existing one
with @tool decorator + pydantic examples to the notebook


**Problem**: Llama3.1 was released while llama3-groq-tool-call fine-tune
Is noted in notebooks.

**Solution**: update docs + notebooks to llama3.1 (which has improved
tool calling support)


**Problem**: Install instructions are incomplete, there is no
information to download a model and/or run the Ollama server

**Solution**: Add simple instructions to start the ollama service and
pull model (for toolcalling)

---------

Co-authored-by: Chester Curme <[email protected]>
  • Loading branch information
@BobMerkus @ccurme
BobMerkus and ccurme authored Aug 20, 2024
1 parent 12e490e commit 8e3e532
Show file tree
Hide file tree
Showing 5 changed files with 72 additions and 381 deletions.
70 changes: 29 additions & 41 deletions docs/docs/integrations/chat/ollama.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -110,15 +110,15 @@
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": 9,
"id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae",
"metadata": {},
"outputs": [],
"source": [
"from langchain_ollama import ChatOllama\n",
"\n",
"llm = ChatOllama(\n",
" model=\"llama3\",\n",
" model=\"llama3.1\",\n",
" temperature=0,\n",
" # other params...\n",
")"
Expand All @@ -134,7 +134,7 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": 10,
"id": "62e0dbc3",
"metadata": {
"tags": []
Expand All @@ -143,10 +143,10 @@
{
"data": {
"text/plain": [
"AIMessage(content='Je adore le programmation.\\n\\n(Note: \"programmation\" is not commonly used in French, but I translated it as \"le programmation\" to maintain the same grammatical structure and meaning as the original English sentence.)', response_metadata={'model': 'llama3', 'created_at': '2024-07-22T17:43:54.731273Z', 'message': {'role': 'assistant', 'content': ''}, 'done_reason': 'stop', 'done': True, 'total_duration': 11094839375, 'load_duration': 10121854667, 'prompt_eval_count': 36, 'prompt_eval_duration': 146569000, 'eval_count': 46, 'eval_duration': 816593000}, id='run-befccbdc-e1f9-42a9-85cf-e69b926d6b8b-0', usage_metadata={'input_tokens': 36, 'output_tokens': 46, 'total_tokens': 82})"
"AIMessage(content='The translation of \"I love programming\" from English to French is:\\n\\n\"J\\'adore programmer.\"', response_metadata={'model': 'llama3.1', 'created_at': '2024-08-19T16:05:32.81965Z', 'message': {'role': 'assistant', 'content': ''}, 'done_reason': 'stop', 'done': True, 'total_duration': 2167842917, 'load_duration': 54222584, 'prompt_eval_count': 35, 'prompt_eval_duration': 893007000, 'eval_count': 22, 'eval_duration': 1218962000}, id='run-0863daa2-43bf-4a43-86cc-611b23eae466-0', usage_metadata={'input_tokens': 35, 'output_tokens': 22, 'total_tokens': 57})"
]
},
"execution_count": 4,
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
Expand All @@ -167,17 +167,17 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": 11,
"id": "d86145b3-bfef-46e8-b227-4dda5c9c2705",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Je adore le programmation.\n",
"The translation of \"I love programming\" from English to French is:\n",
"\n",
"(Note: \"programmation\" is not commonly used in French, but I translated it as \"le programmation\" to maintain the same grammatical structure and meaning as the original English sentence.)\n"
"\"J'adore programmer.\"\n"
]
}
],
Expand All @@ -197,17 +197,17 @@
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": 12,
"id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"AIMessage(content='Ich liebe Programmieren!\\n\\n(Note: \"Ich liebe\" means \"I love\", \"Programmieren\" is the verb for \"programming\")', response_metadata={'model': 'llama3', 'created_at': '2024-07-04T04:22:33.864132Z', 'message': {'role': 'assistant', 'content': ''}, 'done_reason': 'stop', 'done': True, 'total_duration': 1310800083, 'load_duration': 1782000, 'prompt_eval_count': 16, 'prompt_eval_duration': 250199000, 'eval_count': 29, 'eval_duration': 1057192000}, id='run-cbadbe59-2de2-4ec0-a18a-b3220226c3d2-0')"
"AIMessage(content='Das Programmieren ist mir ein Leidenschaft! (That\\'s \"Programming is my passion!\" in German.) Would you like me to translate anything else?', response_metadata={'model': 'llama3.1', 'created_at': '2024-08-19T16:05:34.893548Z', 'message': {'role': 'assistant', 'content': ''}, 'done_reason': 'stop', 'done': True, 'total_duration': 2045997333, 'load_duration': 22584792, 'prompt_eval_count': 30, 'prompt_eval_duration': 213210000, 'eval_count': 32, 'eval_duration': 1808541000}, id='run-d18e1c6b-50e0-4b1d-b23a-973fa058edad-0', usage_metadata={'input_tokens': 30, 'output_tokens': 32, 'total_tokens': 62})"
]
},
"execution_count": 9,
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
Expand Down Expand Up @@ -242,56 +242,56 @@
"source": [
"## Tool calling\n",
"\n",
"We can use [tool calling](https://blog.langchain.dev/improving-core-tool-interfaces-and-docs-in-langchain/) with an LLM [that has been fine-tuned for tool use](https://ollama.com/library/llama3-groq-tool-use): \n",
"We can use [tool calling](https://blog.langchain.dev/improving-core-tool-interfaces-and-docs-in-langchain/) with an LLM [that has been fine-tuned for tool use](https://ollama.com/library/llama3.1): \n",
"\n",
"```\n",
"ollama pull llama3-groq-tool-use\n",
"ollama pull llama3.1\n",
"```\n",
"\n",
"We can just pass normal Python functions directly as tools."
"Details on creating custom tools are available in [this guide](/docs/how_to/custom_tools/). Below, we demonstrate how to create a tool using the `@tool` decorator on a normal python function."
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "5250bceb-1029-41ff-b447-983518704d88",
"execution_count": 13,
"id": "f767015f",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[{'name': 'validate_user',\n",
" 'args': {'addresses': ['123 Fake St, Boston MA',\n",
" '234 Pretend Boulevard, Houston TX'],\n",
" 'user_id': 123},\n",
" 'id': 'fe2148d3-95fb-48e9-845a-4bfecc1f1f96',\n",
" 'args': {'addresses': '[\"123 Fake St, Boston, MA\", \"234 Pretend Boulevard, Houston, TX\"]',\n",
" 'user_id': '123'},\n",
" 'id': '40fe3de0-500c-4b91-9616-5932a929e640',\n",
" 'type': 'tool_call'}]"
]
},
"execution_count": 10,
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from typing import List\n",
"\n",
"from langchain_core.tools import tool\n",
"from langchain_ollama import ChatOllama\n",
"from typing_extensions import TypedDict\n",
"\n",
"\n",
"def validate_user(user_id: int, addresses: List) -> bool:\n",
"@tool\n",
"def validate_user(user_id: int, addresses: List[str]) -> bool:\n",
" \"\"\"Validate user using historical addresses.\n",
"\n",
" Args:\n",
" user_id: (int) the user ID.\n",
" addresses: Previous addresses.\n",
" user_id (int): the user ID.\n",
" addresses (List[str]): Previous addresses as a list of strings.\n",
" \"\"\"\n",
" return True\n",
"\n",
"\n",
"llm = ChatOllama(\n",
" model=\"llama3-groq-tool-use\",\n",
" model=\"llama3.1\",\n",
" temperature=0,\n",
").bind_tools([validate_user])\n",
"\n",
Expand All @@ -303,18 +303,6 @@
"result.tool_calls"
]
},
{
"cell_type": "markdown",
"id": "2bb034ff-218f-4865-afea-3f5e57d3bdee",
"metadata": {},
"source": [
"We look at the LangSmith trace to see that the tool call was performed: \n",
"\n",
"https://smith.langchain.com/public/4169348a-d6be-45df-a7cf-032f6baa4697/r\n",
"\n",
"In particular, the trace shows how the tool schema was populated."
]
},
{
"cell_type": "markdown",
"id": "4c5e0197",
Expand All @@ -331,7 +319,7 @@
},
{
"cell_type": "code",
"execution_count": 11,
"execution_count": 15,
"id": "36c9b1c2",
"metadata": {},
"outputs": [
Expand Down Expand Up @@ -391,7 +379,7 @@
},
{
"cell_type": "code",
"execution_count": 12,
"execution_count": 16,
"id": "32b3ba7b",
"metadata": {},
"outputs": [
Expand Down Expand Up @@ -467,7 +455,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.8"
"version": "3.12.4"
}
},
"nbformat": 4,
Expand Down
Loading

0 comments on commit 8e3e532

Please sign in to comment.