From 86a3f6edf1dc0d1f66663613ab3b23fb98b0fc1f Mon Sep 17 00:00:00 2001 From: Bagatur <22008038+baskaryan@users.noreply.github.com> Date: Mon, 10 Jun 2024 12:50:50 -0700 Subject: [PATCH] docs: standardize ChatVertexAI (#22686) Part of #22296. Part two of https://github.com/langchain-ai/langchain-google/pull/287 --- .../chat/google_vertex_ai_palm.ipynb | 570 ++++-------------- 1 file changed, 104 insertions(+), 466 deletions(-) diff --git a/docs/docs/integrations/chat/google_vertex_ai_palm.ipynb b/docs/docs/integrations/chat/google_vertex_ai_palm.ipynb index aa3156edae176..74e1dd87e1c9b 100644 --- a/docs/docs/integrations/chat/google_vertex_ai_palm.ipynb +++ b/docs/docs/integrations/chat/google_vertex_ai_palm.ipynb @@ -2,33 +2,50 @@ "cells": [ { "cell_type": "raw", + "id": "afaf8039", "metadata": {}, "source": [ "---\n", "sidebar_label: Google Cloud Vertex AI\n", - "keywords: [gemini, vertex, ChatVertexAI, gemini-pro]\n", "---" ] }, { "cell_type": "markdown", + "id": "e49f1e0d", "metadata": {}, "source": [ "# ChatVertexAI\n", "\n", - "Note: This is separate from the Google PaLM integration. Google has chosen to offer an enterprise version of PaLM through GCP, and this supports the models made available through there. \n", + "This page provides a quick overview for getting started with VertexAI [chat models](/docs/concepts/#chat-models). For detailed documentation of all ChatVertexAI features and configurations head to the [API reference](https://api.python.langchain.com/en/latest/chat_models/langchain_google_vertexai.chat_models.ChatVertexAI.html).\n", "\n", - "ChatVertexAI exposes all foundational models available in Google Cloud:\n", + "ChatVertexAI exposes all foundational models available in Google Cloud, like `gemini-1.5-pro`, `gemini-1.5-flash`, etc. For a full and updated list of available models visit [VertexAI documentation](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/overview).\n", "\n", - "- Gemini (`gemini-pro` and `gemini-pro-vision`)\n", - "- PaLM 2 for Text (`text-bison`)\n", - "- Codey for Code Generation (`codechat-bison`)\n", + ":::info Google Cloud VertexAI vs Google PaLM\n", "\n", - "For a full and updated list of available models visit [VertexAI documentation](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/overview).\n", + "The Google Cloud VertexAI integration is separate from the [Google PaLM integration](/docs/integrations/chat/google_generative_ai/). Google has chosen to offer an enterprise version of PaLM through GCP, and this supports the models made available through there. \n", "\n", - "By default, Google Cloud [does not use](https://cloud.google.com/vertex-ai/docs/generative-ai/data-governance#foundation_model_development) customer data to train its foundation models as part of Google Cloud`s AI/ML Privacy Commitment. More details about how Google processes data can also be found in [Google's Customer Data Processing Addendum (CDPA)](https://cloud.google.com/terms/data-processing-addendum).\n", + ":::\n", "\n", - "To use `Google Cloud Vertex AI` PaLM you must have the `langchain-google-vertexai` Python package installed and either:\n", + "## Overview\n", + "### Integration details\n", + "\n", + "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/v0.2/docs/integrations/chat/google_vertex_ai) | Package downloads | Package latest |\n", + "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", + "| [ChatVertexAI](https://api.python.langchain.com/en/latest/chat_models/langchain_google_vertexai.chat_models.ChatVertexAI.html) | [langchain-google-vertexai](https://api.python.langchain.com/en/latest/google_vertexai_api_reference.html) | ❌ | beta | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-google-vertexai?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-google-vertexai?style=flat-square&label=%20) |\n", + "\n", + "### Model features\n", + "| [Tool calling](/docs/how_to/tool_calling/) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", + "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", + "| ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | \n", + "\n", + "## Setup\n", + "\n", + "To access VertexAI models you'll need to create a Google Cloud Platform account, set up credentials, and install the `langchain-google-vertexai` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "To use the integration you must:\n", "- Have credentials configured for your environment (gcloud, workload identity, etc...)\n", "- Store the path to a service account JSON file as the GOOGLE_APPLICATION_CREDENTIALS environment variable\n", "\n", @@ -37,432 +54,156 @@ "For more information, see: \n", "- https://cloud.google.com/docs/authentication/application-default-credentials#GAC\n", "- https://googleapis.dev/python/google-auth/latest/reference/google.auth.html#module-google.auth\n", - "\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "%pip install --upgrade --quiet langchain-google-vertexai" + "\n", + "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" ] }, { "cell_type": "code", "execution_count": 1, + "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de", "metadata": {}, "outputs": [], "source": [ - "from langchain_core.prompts import ChatPromptTemplate\n", - "from langchain_google_vertexai import ChatVertexAI" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "AIMessage(content=\" J'aime la programmation.\")" - ] - }, - "execution_count": null, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "system = \"You are a helpful assistant who translate English to French\"\n", - "human = \"Translate this sentence from English to French. I love programming.\"\n", - "prompt = ChatPromptTemplate.from_messages([(\"system\", system), (\"human\", human)])\n", - "\n", - "chat = ChatVertexAI()\n", - "\n", - "chain = prompt | chat\n", - "chain.invoke({})" + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" ] }, { "cell_type": "markdown", + "id": "0730d6a1-c893-4840-9817-5e5251676d5d", "metadata": {}, "source": [ - "Gemini doesn't support SystemMessage at the moment, but it can be added to the first human message in the row. If you want such behavior, just set the `convert_system_message_to_human` to `True`:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "AIMessage(content=\"J'aime la programmation.\")" - ] - }, - "execution_count": null, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "system = \"You are a helpful assistant who translate English to French\"\n", - "human = \"Translate this sentence from English to French. I love programming.\"\n", - "prompt = ChatPromptTemplate.from_messages([(\"system\", system), (\"human\", human)])\n", - "\n", - "chat = ChatVertexAI(model=\"gemini-pro\", convert_system_message_to_human=True)\n", + "### Installation\n", "\n", - "chain = prompt | chat\n", - "chain.invoke({})" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "If we want to construct a simple chain that takes user specified parameters:" + "The LangChain VertexAI integration lives in the `langchain-google-vertexai` package:" ] }, { "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "AIMessage(content=' プログラミングが大好きです')" - ] - }, - "execution_count": null, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "system = (\n", - " \"You are a helpful assistant that translates {input_language} to {output_language}.\"\n", - ")\n", - "human = \"{text}\"\n", - "prompt = ChatPromptTemplate.from_messages([(\"system\", system), (\"human\", human)])\n", - "\n", - "chat = ChatVertexAI()\n", - "\n", - "chain = prompt | chat\n", - "\n", - "chain.invoke(\n", - " {\n", - " \"input_language\": \"English\",\n", - " \"output_language\": \"Japanese\",\n", - " \"text\": \"I love programming\",\n", - " }\n", - ")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Code generation chat models\n", - "You can now leverage the Codey API for code chat within Vertex AI. The model available is:\n", - "- `codechat-bison`: for code assistance" - ] - }, - { - "cell_type": "code", - "execution_count": null, + "execution_count": 2, + "id": "652d6238-1f87-422a-b135-f5abbb8652fc", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - " ```python\n", - "def is_prime(n):\n", - " \"\"\"\n", - " Check if a number is prime.\n", - "\n", - " Args:\n", - " n: The number to check.\n", - "\n", - " Returns:\n", - " True if n is prime, False otherwise.\n", - " \"\"\"\n", - "\n", - " # If n is 1, it is not prime.\n", - " if n == 1:\n", - " return False\n", - "\n", - " # Iterate over all numbers from 2 to the square root of n.\n", - " for i in range(2, int(n ** 0.5) + 1):\n", - " # If n is divisible by any number from 2 to its square root, it is not prime.\n", - " if n % i == 0:\n", - " return False\n", - "\n", - " # If n is divisible by no number from 2 to its square root, it is prime.\n", - " return True\n", - "\n", - "\n", - "def find_prime_numbers(n):\n", - " \"\"\"\n", - " Find all prime numbers up to a given number.\n", - "\n", - " Args:\n", - " n: The upper bound for the prime numbers to find.\n", - "\n", - " Returns:\n", - " A list of all prime numbers up to n.\n", - " \"\"\"\n", - "\n", - " # Create a list of all numbers from 2 to n.\n", - " numbers = list(range(2, n + 1))\n", - "\n", - " # Iterate over the list of numbers and remove any that are not prime.\n", - " for number in numbers:\n", - " if not is_prime(number):\n", - " numbers.remove(number)\n", - "\n", - " # Return the list of prime numbers.\n", - " return numbers\n", - "```\n" + "Note: you may need to restart the kernel to use updated packages.\n" ] } ], "source": [ - "chat = ChatVertexAI(model=\"codechat-bison\", max_tokens=1000, temperature=0.5)\n", - "\n", - "message = chat.invoke(\"Write a Python function generating all prime numbers\")\n", - "print(message.content)" + "%pip install -qU langchain-google-vertexai" ] }, { "cell_type": "markdown", + "id": "a38cde65-254d-4219-a441-068766c0d4b5", "metadata": {}, "source": [ - "## Full generation info\n", - "\n", - "We can use the `generate` method to get back extra metadata like [safety attributes](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/responsible-ai#safety_attribute_confidence_scoring) and not just chat completions\n", - "\n", - "Note that the `generation_info` will be different depending if you're using a gemini model or not.\n", - "\n", - "### Gemini model\n", + "## Instantiation\n", "\n", - "`generation_info` will include:\n", - "\n", - "- `is_blocked`: whether generation was blocked or not\n", - "- `safety_ratings`: safety ratings' categories and probability labels" + "Now we can instantiate our model object and generate chat completions:" ] }, { "cell_type": "code", - "execution_count": 4, + "execution_count": 3, + "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae", "metadata": {}, "outputs": [], "source": [ - "from pprint import pprint\n", - "\n", - "from langchain_core.messages import HumanMessage\n", - "from langchain_google_vertexai import HarmBlockThreshold, HarmCategory" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "{'citation_metadata': None,\n", - " 'is_blocked': False,\n", - " 'safety_ratings': [{'blocked': False,\n", - " 'category': 'HARM_CATEGORY_HATE_SPEECH',\n", - " 'probability_label': 'NEGLIGIBLE'},\n", - " {'blocked': False,\n", - " 'category': 'HARM_CATEGORY_DANGEROUS_CONTENT',\n", - " 'probability_label': 'NEGLIGIBLE'},\n", - " {'blocked': False,\n", - " 'category': 'HARM_CATEGORY_HARASSMENT',\n", - " 'probability_label': 'NEGLIGIBLE'},\n", - " {'blocked': False,\n", - " 'category': 'HARM_CATEGORY_SEXUALLY_EXPLICIT',\n", - " 'probability_label': 'NEGLIGIBLE'}],\n", - " 'usage_metadata': {'candidates_token_count': 6,\n", - " 'prompt_token_count': 12,\n", - " 'total_token_count': 18}}\n" - ] - } - ], - "source": [ - "human = \"Translate this sentence from English to French. I love programming.\"\n", - "messages = [HumanMessage(content=human)]\n", - "\n", - "\n", - "chat = ChatVertexAI(\n", - " model_name=\"gemini-pro\",\n", - " safety_settings={\n", - " HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE\n", - " },\n", - ")\n", - "\n", - "result = chat.generate([messages])\n", - "pprint(result.generations[0][0].generation_info)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Non-gemini model\n", - "\n", - "`generation_info` will include:\n", - "\n", - "- `is_blocked`: whether generation was blocked or not\n", - "- `safety_attributes`: a dictionary mapping safety attributes to their scores" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "{'errors': (),\n", - " 'grounding_metadata': {'citations': [], 'search_queries': []},\n", - " 'is_blocked': False,\n", - " 'safety_attributes': [{'Derogatory': 0.1, 'Insult': 0.1, 'Sexual': 0.2}],\n", - " 'usage_metadata': {'candidates_billable_characters': 88.0,\n", - " 'candidates_token_count': 24.0,\n", - " 'prompt_billable_characters': 58.0,\n", - " 'prompt_token_count': 12.0}}\n" - ] - } - ], - "source": [ - "chat = ChatVertexAI() # default is `chat-bison`\n", - "\n", - "result = chat.generate([messages])\n", - "pprint(result.generations[0][0].generation_info)" + "from langchain_google_vertexai import ChatVertexAI\n", + "\n", + "llm = ChatVertexAI(\n", + " model=\"gemini-1.5-flash-001\",\n", + " temperature=0,\n", + " max_tokens=None,\n", + " max_retries=6,\n", + " stop=None,\n", + " # other params...\n", + ")" ] }, { "cell_type": "markdown", + "id": "2b4f3e15", "metadata": {}, "source": [ - "## Tool calling (a.k.a. function calling) with Gemini\n", - "\n", - "We can pass tool definitions to Gemini models to get the model to invoke those tools when appropriate. This is useful not only for LLM-powered tool use but also for getting structured outputs out of models more generally.\n", - "\n", - "With `ChatVertexAI.bind_tools()`, we can easily pass in Pydantic classes, dict schemas, LangChain tools, or even functions as tools to the model. Under the hood these are converted to a Gemini tool schema, which looks like:\n", - "```python\n", - "{\n", - " \"name\": \"...\", # tool name\n", - " \"description\": \"...\", # tool description\n", - " \"parameters\": {...} # tool input schema as JSONSchema\n", - "}\n", - "```" + "## Invocation" ] }, { "cell_type": "code", - "execution_count": 2, - "metadata": {}, + "execution_count": 4, + "id": "62e0dbc3", + "metadata": { + "tags": [] + }, "outputs": [ { "data": { "text/plain": [ - "AIMessage(content='', additional_kwargs={'function_call': {'name': 'GetWeather', 'arguments': '{\"location\": \"San Francisco, CA\"}'}}, response_metadata={'is_blocked': False, 'safety_ratings': [{'category': 'HARM_CATEGORY_HATE_SPEECH', 'probability_label': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_DANGEROUS_CONTENT', 'probability_label': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_HARASSMENT', 'probability_label': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_SEXUALLY_EXPLICIT', 'probability_label': 'NEGLIGIBLE', 'blocked': False}], 'citation_metadata': None, 'usage_metadata': {'prompt_token_count': 41, 'candidates_token_count': 7, 'total_token_count': 48}}, id='run-05e760dc-0682-4286-88e1-5b23df69b083-0', tool_calls=[{'name': 'GetWeather', 'args': {'location': 'San Francisco, CA'}, 'id': 'cd2499c4-4513-4059-bfff-5321b6e922d0'}])" + "AIMessage(content=\"J'adore programmer. \\n\", response_metadata={'is_blocked': False, 'safety_ratings': [{'category': 'HARM_CATEGORY_HATE_SPEECH', 'probability_label': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_DANGEROUS_CONTENT', 'probability_label': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_HARASSMENT', 'probability_label': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_SEXUALLY_EXPLICIT', 'probability_label': 'NEGLIGIBLE', 'blocked': False}], 'usage_metadata': {'prompt_token_count': 20, 'candidates_token_count': 7, 'total_token_count': 27}}, id='run-7032733c-d05c-4f0c-a17a-6c575fdd1ae0-0', usage_metadata={'input_tokens': 20, 'output_tokens': 7, 'total_tokens': 27})" ] }, - "execution_count": 2, + "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "from langchain.pydantic_v1 import BaseModel, Field\n", - "\n", - "\n", - "class GetWeather(BaseModel):\n", - " \"\"\"Get the current weather in a given location\"\"\"\n", - "\n", - " location: str = Field(..., description=\"The city and state, e.g. San Francisco, CA\")\n", - "\n", - "\n", - "llm = ChatVertexAI(model=\"gemini-pro\", temperature=0)\n", - "llm_with_tools = llm.bind_tools([GetWeather])\n", - "ai_msg = llm_with_tools.invoke(\n", - " \"what is the weather like in San Francisco\",\n", - ")\n", + "messages = [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", + " ),\n", + " (\"human\", \"I love programming.\"),\n", + "]\n", + "ai_msg = llm.invoke(messages)\n", "ai_msg" ] }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The tool calls can be access via the `AIMessage.tool_calls` attribute, where they are extracted in a model-agnostic format:" - ] - }, { "cell_type": "code", - "execution_count": 3, + "execution_count": 5, + "id": "d86145b3-bfef-46e8-b227-4dda5c9c2705", "metadata": {}, "outputs": [ { - "data": { - "text/plain": [ - "[{'name': 'GetWeather',\n", - " 'args': {'location': 'San Francisco, CA'},\n", - " 'id': 'cd2499c4-4513-4059-bfff-5321b6e922d0'}]" - ] - }, - "execution_count": 3, - "metadata": {}, - "output_type": "execute_result" + "name": "stdout", + "output_type": "stream", + "text": [ + "J'adore programmer. \n", + "\n" + ] } ], "source": [ - "ai_msg.tool_calls" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "For a complete guide on tool calling [head here](/docs/how_to/function_calling)." + "print(ai_msg.content)" ] }, { "cell_type": "markdown", + "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8", "metadata": {}, "source": [ - "## Structured outputs\n", - "\n", - "Many applications require structured model outputs. Tool calling makes it much easier to do this reliably. The [with_structured_outputs](https://api.python.langchain.com/en/latest/chat_models/langchain_google_vertexai.chat_models.ChatVertexAI.html) constructor provides a simple interface built on top of tool calling for getting structured outputs out of a model. For a complete guide on structured outputs [head here](/docs/how_to/structured_output).\n", + "## Chaining\n", "\n", - "### ChatVertexAI.with_structured_outputs()\n", - "\n", - "To get structured outputs from our Gemini model all we need to do is to specify a desired schema, either as a Pydantic class or as a JSON schema, " + "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" ] }, { "cell_type": "code", "execution_count": 6, + "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b", "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "Person(name='Stefan', age=13)" + "AIMessage(content='Ich liebe Programmieren. \\n', response_metadata={'is_blocked': False, 'safety_ratings': [{'category': 'HARM_CATEGORY_HATE_SPEECH', 'probability_label': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_DANGEROUS_CONTENT', 'probability_label': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_HARASSMENT', 'probability_label': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_SEXUALLY_EXPLICIT', 'probability_label': 'NEGLIGIBLE', 'blocked': False}], 'usage_metadata': {'prompt_token_count': 15, 'candidates_token_count': 8, 'total_token_count': 23}}, id='run-c71955fd-8dc1-422b-88a7-853accf4811b-0', usage_metadata={'input_tokens': 15, 'output_tokens': 8, 'total_tokens': 23})" ] }, "execution_count": 6, @@ -471,139 +212,36 @@ } ], "source": [ - "class Person(BaseModel):\n", - " \"\"\"Save information about a person.\"\"\"\n", - "\n", - " name: str = Field(..., description=\"The person's name.\")\n", - " age: int = Field(..., description=\"The person's age.\")\n", - "\n", - "\n", - "structured_llm = llm.with_structured_output(Person)\n", - "structured_llm.invoke(\"Stefan is already 13 years old\")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### [Legacy] Using `create_structured_runnable()`\n", - "\n", - "The legacy wasy to get structured outputs is using the `create_structured_runnable` constructor:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_google_vertexai import create_structured_runnable\n", - "\n", - "chain = create_structured_runnable(Person, llm)\n", - "chain.invoke(\"My name is Erick and I'm 27 years old\")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Asynchronous calls\n", - "\n", - "We can make asynchronous calls via the Runnables [Async Interface](/docs/concepts#interface)." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# for running these examples in the notebook:\n", - "import asyncio\n", - "\n", - "import nest_asyncio\n", + "from langchain_core.prompts import ChatPromptTemplate\n", "\n", - "nest_asyncio.apply()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "AIMessage(content=' अहं प्रोग्रामनं प्रेमामि')" - ] - }, - "execution_count": null, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "system = (\n", - " \"You are a helpful assistant that translates {input_language} to {output_language}.\"\n", + "prompt = ChatPromptTemplate.from_messages(\n", + " [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", + " ),\n", + " (\"human\", \"{input}\"),\n", + " ]\n", ")\n", - "human = \"{text}\"\n", - "prompt = ChatPromptTemplate.from_messages([(\"system\", system), (\"human\", human)])\n", "\n", - "chat = ChatVertexAI(model=\"chat-bison\", max_tokens=1000, temperature=0.5)\n", - "chain = prompt | chat\n", - "\n", - "asyncio.run(\n", - " chain.ainvoke(\n", - " {\n", - " \"input_language\": \"English\",\n", - " \"output_language\": \"Sanskrit\",\n", - " \"text\": \"I love programming\",\n", - " }\n", - " )\n", + "chain = prompt | llm\n", + "chain.invoke(\n", + " {\n", + " \"input_language\": \"English\",\n", + " \"output_language\": \"German\",\n", + " \"input\": \"I love programming.\",\n", + " }\n", ")" ] }, { "cell_type": "markdown", + "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3", "metadata": {}, "source": [ - "## Streaming calls\n", - "\n", - "We can also stream outputs via the `stream` method:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - " The five most populous countries in the world are:\n", - "1. China (1.4 billion)\n", - "2. India (1.3 billion)\n", - "3. United States (331 million)\n", - "4. Indonesia (273 million)\n", - "5. Pakistan (220 million)" - ] - } - ], - "source": [ - "import sys\n", - "\n", - "prompt = ChatPromptTemplate.from_messages(\n", - " [(\"human\", \"List out the 5 most populous countries in the world\")]\n", - ")\n", - "\n", - "chat = ChatVertexAI()\n", - "\n", - "chain = prompt | chat\n", + "## API reference\n", "\n", - "for chunk in chain.stream({}):\n", - " sys.stdout.write(chunk.content)\n", - " sys.stdout.flush()" + "For detailed documentation of all ChatVertexAI features and configurations, like how to send multimodal inputs and configure safety settings, head to the API reference: https://api.python.langchain.com/en/latest/chat_models/langchain_google_vertexai.chat_models.ChatVertexAI.html" ] } ], @@ -627,5 +265,5 @@ } }, "nbformat": 4, - "nbformat_minor": 4 + "nbformat_minor": 5 }