-
Notifications
You must be signed in to change notification settings - Fork 8.3k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[Security Solution] [Elastic AI Assistant] LangChain integration (exp…
…erimental) (#164908) ## [Security Solution] [Elastic AI Assistant] LangChain integration (experimental) This PR integrates [LangChain](https://www.langchain.com/) with the [Elastic AI Assistant](https://www.elastic.co/blog/introducing-elastic-ai-assistant) as an experimental, alternative execution path. ### How it works - There are virtually no client side changes to the assistant, apart from a new branch in `x-pack/packages/kbn-elastic-assistant/impl/assistant/api.tsx` that chooses a path based on the value of the `assistantLangChain` flag: ```typescript const path = assistantLangChain ? `/internal/elastic_assistant/actions/connector/${apiConfig?.connectorId}/_execute` : `/api/actions/connector/${apiConfig?.connectorId}/_execute`; ``` Execution of the LangChain chain happens server-side. The new route still executes the request via the `connectorId` in the route, but the connector won't execute the request exactly as it was sent by the client. Instead, the connector will execute one (or more) prompts that are generated by LangChain. Requests routed to `/internal/elastic_assistant/actions/connector/${apiConfig?.connectorId}/_execute` will be processed by a new Kibana plugin located in: ``` x-pack/plugins/elastic_assistant ``` - Requests are processed in the `postActionsConnectorExecuteRoute` handler in `x-pack/plugins/elastic_assistant/server/routes/post_actions_connector_execute.ts`. The `postActionsConnectorExecuteRoute` route handler: 1. Extracts the chat messages sent by the assistant 2. Converts the extracted messages to the format expected by LangChain 3. Passes the converted messages to `executeCustomLlmChain` - The `executeCustomLlmChain` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/execute_custom_llm_chain/index.ts`: 1. Splits the messages into `pastMessages` and `latestMessage`, where the latter contains only the last message sent by the user 2. Wraps the conversation history in the `BufferMemory` LangChain abstraction 3. Executes the chain, kicking it off with `latestMessage` ```typescript const llm = new ActionsClientLlm({ actions, connectorId, request }); const pastMessages = langchainMessages.slice(0, -1); // all but the last message const latestMessage = langchainMessages.slice(-1); // the last message const memory = new BufferMemory({ chatHistory: new ChatMessageHistory(pastMessages), }); const chain = new ConversationChain({ llm, memory }); await chain.call({ input: latestMessage[0].content }); // kick off the chain with the last message }; ``` - When LangChain executes the chain, it will invoke `ActionsClientLlm`'s `_call` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/llm/actions_client_llm.ts` one or more times. The `_call` function's signature is defined by LangChain: ``` async _call(prompt: string): Promise<string> ``` - The contents of `prompt` are completely determined by LangChain. - The string returned by the promise is the "answer" from the LLM The `ActionsClientLlm` extends LangChain's LLM interface: ```typescript export class ActionsClientLlm extends LLM ``` This let's us do additional "work" in the `_call` function: 1. Create a new assistant message using the contents of the `prompt` (`string`) argument to `_call` 2. Create a request body in the format expected by the connector 3. Create an actions client from the authenticated request context 4. Execute the actions client with the request body 5. Save the raw response from the connector, because that's what the assistant expects 6. Return the result as a plain string, as per the contact of `_call` ## Desk testing This experimental LangChain integration may NOT be enabled via a feature flag (yet). Set ```typescript assistantLangChain={true} ``` in `x-pack/plugins/security_solution/public/app/app.tsx` to enable this experimental feature in development environments.
- Loading branch information
1 parent
31e9557
commit 3935548
Showing
46 changed files
with
1,954 additions
and
36 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Validating CODEOWNERS rules …
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
129 changes: 129 additions & 0 deletions
129
x-pack/packages/kbn-elastic-assistant/impl/assistant/api.test.tsx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,129 @@ | ||
| /* | ||
| * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one | ||
| * or more contributor license agreements. Licensed under the Elastic License | ||
| * 2.0; you may not use this file except in compliance with the Elastic License | ||
| * 2.0. | ||
| */ | ||
|
|
||
| import { HttpSetup } from '@kbn/core-http-browser'; | ||
| import { OpenAiProviderType } from '@kbn/stack-connectors-plugin/public/common'; | ||
|
|
||
| import { fetchConnectorExecuteAction, FetchConnectorExecuteAction } from './api'; | ||
| import type { Conversation, Message } from '../assistant_context/types'; | ||
| import { API_ERROR } from './translations'; | ||
|
|
||
| jest.mock('@kbn/core-http-browser'); | ||
|
|
||
| const mockHttp = { | ||
| fetch: jest.fn(), | ||
| } as unknown as HttpSetup; | ||
|
|
||
| const apiConfig: Conversation['apiConfig'] = { | ||
| connectorId: 'foo', | ||
| model: 'gpt-4', | ||
| provider: OpenAiProviderType.OpenAi, | ||
| }; | ||
|
|
||
| const messages: Message[] = [ | ||
| { content: 'This is a test', role: 'user', timestamp: new Date().toLocaleString() }, | ||
| ]; | ||
|
|
||
| describe('fetchConnectorExecuteAction', () => { | ||
| beforeEach(() => { | ||
| jest.clearAllMocks(); | ||
| }); | ||
|
|
||
| it('calls the internal assistant API when assistantLangChain is true', async () => { | ||
| const testProps: FetchConnectorExecuteAction = { | ||
| assistantLangChain: true, | ||
| http: mockHttp, | ||
| messages, | ||
| apiConfig, | ||
| }; | ||
|
|
||
| await fetchConnectorExecuteAction(testProps); | ||
|
|
||
| expect(mockHttp.fetch).toHaveBeenCalledWith( | ||
| '/internal/elastic_assistant/actions/connector/foo/_execute', | ||
| { | ||
| body: '{"params":{"subActionParams":{"body":"{\\"model\\":\\"gpt-4\\",\\"messages\\":[{\\"role\\":\\"user\\",\\"content\\":\\"This is a test\\"}],\\"n\\":1,\\"stop\\":null,\\"temperature\\":0.2}"},"subAction":"test"}}', | ||
| headers: { 'Content-Type': 'application/json' }, | ||
| method: 'POST', | ||
| signal: undefined, | ||
| } | ||
| ); | ||
| }); | ||
|
|
||
| it('calls the actions connector api when assistantLangChain is false', async () => { | ||
| const testProps: FetchConnectorExecuteAction = { | ||
| assistantLangChain: false, | ||
| http: mockHttp, | ||
| messages, | ||
| apiConfig, | ||
| }; | ||
|
|
||
| await fetchConnectorExecuteAction(testProps); | ||
|
|
||
| expect(mockHttp.fetch).toHaveBeenCalledWith('/api/actions/connector/foo/_execute', { | ||
| body: '{"params":{"subActionParams":{"body":"{\\"model\\":\\"gpt-4\\",\\"messages\\":[{\\"role\\":\\"user\\",\\"content\\":\\"This is a test\\"}],\\"n\\":1,\\"stop\\":null,\\"temperature\\":0.2}"},"subAction":"test"}}', | ||
| headers: { 'Content-Type': 'application/json' }, | ||
| method: 'POST', | ||
| signal: undefined, | ||
| }); | ||
| }); | ||
|
|
||
| it('returns API_ERROR when the response status is not ok', async () => { | ||
| (mockHttp.fetch as jest.Mock).mockResolvedValue({ status: 'error' }); | ||
|
|
||
| const testProps: FetchConnectorExecuteAction = { | ||
| assistantLangChain: false, | ||
| http: mockHttp, | ||
| messages, | ||
| apiConfig, | ||
| }; | ||
|
|
||
| const result = await fetchConnectorExecuteAction(testProps); | ||
|
|
||
| expect(result).toBe(API_ERROR); | ||
| }); | ||
|
|
||
| it('returns API_ERROR when there are no choices', async () => { | ||
| (mockHttp.fetch as jest.Mock).mockResolvedValue({ status: 'ok', data: {} }); | ||
| const testProps: FetchConnectorExecuteAction = { | ||
| assistantLangChain: false, | ||
| http: mockHttp, | ||
| messages, | ||
| apiConfig, | ||
| }; | ||
|
|
||
| const result = await fetchConnectorExecuteAction(testProps); | ||
|
|
||
| expect(result).toBe(API_ERROR); | ||
| }); | ||
|
|
||
| it('return the trimmed first `choices` `message` `content` when the API call is successful', async () => { | ||
| (mockHttp.fetch as jest.Mock).mockResolvedValue({ | ||
| status: 'ok', | ||
| data: { | ||
| choices: [ | ||
| { | ||
| message: { | ||
| content: ' Test response ', // leading and trailing whitespace | ||
| }, | ||
| }, | ||
| ], | ||
| }, | ||
| }); | ||
|
|
||
| const testProps: FetchConnectorExecuteAction = { | ||
| assistantLangChain: false, | ||
| http: mockHttp, | ||
| messages, | ||
| apiConfig, | ||
| }; | ||
|
|
||
| const result = await fetchConnectorExecuteAction(testProps); | ||
|
|
||
| expect(result).toBe('Test response'); | ||
| }); | ||
| }); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.