Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
  • Loading branch information
disenone committed Sep 30, 2024
2 parents d9a24f5 + 05b022c commit f298e6a
Show file tree
Hide file tree
Showing 3 changed files with 76 additions and 81 deletions.
70 changes: 35 additions & 35 deletions docs/en/ue-插件-AIChatPlus.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,34 +19,34 @@ tags:
- Claude
- Gemini
- Ollama
description: UE plugin AIChatPlus documentation
description: UE Plugin AIChatPlus Documentation
---

<meta property="og:title" content="UE 插件 AIChatPlus 说明文档" />

#UE Plugin AIChatPlus Documentation

##Public repository
##Public Repository

[UE.AIChatPlus.Public](https://github.com/disenone/UE.AIChatPlus.Public)

##Plugin acquisition
##Obtain Plugin

[AIChatPlus](https://www.unrealengine.com/marketplace/zh-CN/product/aichatplus-ai-chat-integration-openai-azure-claude-gemini)

##Introduction to the plugin
##Introduction to the Plugin

This plugin supports UE5.2+.

UE.AIChatPlus is a UnrealEngine plugin that enables communication with various GPT AI chat services. Currently supported services include OpenAI (ChatGPT, DALL-E), Azure OpenAI (ChatGPT, DALL-E), Claude, Google Gemini, Ollama, and local offline llama.cpp. In the future, more service providers will be supported. Its implementation is based on asynchronous REST requests, ensuring high efficiency performance, making it convenient for UE developers to integrate these AI chat services.
UE.AIChatPlus is a plugin for Unreal Engine that enables communication with various GPT AI chat services. Currently, it supports services such as OpenAI (ChatGPT, DALL-E), Azure OpenAI (ChatGPT, DALL-E), Claude, Google Gemini, Ollama, and local offline llama.cpp. In the future, it will continue to support more service providers. Its implementation is based on asynchronous REST requests, ensuring high efficiency and making it convenient for UE developers to integrate these AI chat services.

At the same time, UE.AIChatPlus also includes an editor tool that allows you to use these AI chat services directly in the editor to generate text and images, analyze images, and more.

##Instructions for use

###Editor Chat Tool

Menu bar Tools -> AIChatPlus -> AIChat can open the editing chat tool provided by the plugin.
Menu bar Tools -> AIChatPlus -> AIChat can open the editor chat tool provided by the plugin

![](assets/img/2024-ue-aichatplus/chat_tool3.png)

Expand All @@ -61,92 +61,92 @@ The interface of the tool is roughly as follows:

####Main Functions

Offline Large Models: Integrated llama.cpp library, supporting local offline execution of large models.
Offline Large Model: Integrated llama.cpp library, supporting local offline execution of large models

Text Chat: Click the `New Chat` button in the bottom left corner to start a new text chat session.
Text chat: Click on the `New Chat` button in the bottom left corner to start a new text chat session.

Image Generation: Click the `New Image Chat` button in the bottom left corner to start a new image generation session.
Image Generation: Click the `New Image Chat` button in the lower left corner to create a new image generation session.

Image Analysis: Some chat services in `New Chat` support sending images, such as Claude and Google Gemini. You can click on the 🖼️ or 🎨 button above the input box to upload the image you want to send.
Image Analysis: Some chat services in `New Chat` support sending images, such as Claude, Google Gemini. You can simply click the 🖼️ or 🎨 button above the input box to load the image you want to send.

Support Blueprint: Support Blueprint to create API requests, complete text chat, image generation, and other functions.
Support Blueprint: Support blueprint to create API requests, complete text chatting, image generation, and other functions.

Set the current chat role: The dropdown box above the chat box can set the current character for sending text, allowing you to adjust AI chat by simulating different characters.
Set the current chat character: The dropdown menu above the chat box allows you to select the current character for sending text. You can simulate different characters to adjust AI chat.

Clear chat: Clicking on the ❌ icon at the top of the chat box will clear the history of the current session.
Clear Chat: Clicking the ❌ button at the top of the chat box can clear the history of the current conversation.

**Global Settings**: Click on the `Setting` button in the bottom left corner to open the global settings window. You can adjust default text chat, image generation API services, and configure specific parameters for each API service. The settings will be automatically saved in the project's path `$(ProjectFolder)/Saved/AIChatPlusEditor`.
Global Settings: Click the `Setting` button in the lower left corner to open the global settings window. You can configure default text chat, image generation API services, and specify parameters for each API service. The settings will be automatically saved in the project directory `$(ProjectFolder)/Saved/AIChatPlusEditor`.

Conversation Settings: Click the settings button above the chat box to open the settings window for the current conversation. You can modify the conversation name, change the API service used for the conversation, and independently set specific parameters for each session's API usage. Conversation settings are automatically saved in `$(ProjectFolder)/Saved/AIChatPlusEditor/Sessions`.
Conversation Settings: Click on the settings button above the chat box to open the settings window for the current conversation. You can modify the conversation name, change the API service used in the conversation, and independently adjust specific parameters for each session's API. Conversation settings are automatically saved in `$(ProjectFolder)/Saved/AIChatPlusEditor/Sessions`.

Modify chat content: Hover over the chat content, a settings button will appear for that specific chat content, supporting options like regenerating content, modifying content, copying content, deleting content, and regenerating content below (for content owned by the user role).
Edit Chat Content: When hovering over a chat message, a settings button for that message will appear. It supports options like regenerating content, editing, copying, deleting, and regenerating content below (for messages from user role).

* Image Browsing: For image generation, clicking on the image will open the image viewing window (ImageViewer), supporting saving the image as PNG/UE Texture. Textures can be directly viewed in the Content Browser, making it convenient to use images within the editor. Additionally, it supports functions such as deleting images, regenerating images, and generating more images. For editors on Windows, there is also support for copying images, allowing you to copy images directly to the clipboard for easy use. The images generated during the session will be automatically saved in each session folder, usually located at `$(ProjectFolder)/Saved/AIChatPlusEditor/Sessions/${GUID}/images`.
* Image browsing: For image generation, clicking on an image will open the image viewer window (ImageViewer), supporting saving images as PNG/UE Texture. Textures can be viewed directly in the Content Browser, making it convenient to use images within the editor. Additionally, there is support for deleting images, regenerating images, and continuing to generate more images. For the editor on Windows, there is also the option to copy images, allowing them to be copied directly to the clipboard for easy use. Images generated during a session will also be automatically saved in each session folder, usually located at ` $(ProjectFolder)/Saved/AIChatPlusEditor/Sessions/${GUID}/images`.

Blueprint:

![blueprint](assets/img/2024-ue-aichatplus/blueprint.png)

Global settings:
Global Settings:

![global settings](assets/img/2024-ue-aichatplus/global_setting.png)

Conversation settings:

![session settings](assets/img/2024-ue-aichatplus/session_setting.png)

Edit chat content:
Edit Chat Content:

![chat edit](assets/img/2024-ue-aichatplus/chat_edit.png)

Image Viewer:

![image viewer](assets/img/2024-ue-aichatplus/image_viewer.png)

###Introduction to the core code
###Introduction to Core Code

Currently, the plugin is divided into the following modules:
The plugins are currently divided into the following modules:

*AIChatPlusCommon: Runtime module, responsible for handling various AI API interface requests and parsing response content.*
AIChatPlusCommon: Runtime module, responsible for processing various AI API interface requests and parsing reply content.

* AIChatPlusEditor: Editor module, responsible for implementing the AI chat tool within the editor.
* AIChatPlusEditor: Editor module, responsible for implementing the AI chat tool editor.

AIChatPlusCllama: Runtime module responsible for encapsulating the interface and parameters of llama.cpp to achieve offline execution of large models
* AIChatPlusCllama: Runtime module responsible for encapsulating the interface and parameters of llama.cpp to achieve offline execution of large models

* Thirdparty/LLAMACpp: Third-party module at runtime, integrating dynamic library and header files of llama.cpp.
Thirdparty/LLAMACpp: Third-party module at runtime, integrating llama.cpp dynamic library and header files.

The UClass responsible for sending requests is FAIChatPlus_xxxChatRequest specifically. Each API service has its own independent Request UClass. The replies to the requests are obtained through two types of UClass, UAIChatPlus_ChatHandlerBase and UAIChatPlus_ImageHandlerBase, and only require registering the corresponding callback delegate.
The specific UClass responsible for sending requests is FAIChatPlus_xxxChatRequest. Each API service has its own independent Request UClass. The replies to the requests are obtained through two types of UClass: UAIChatPlus_ChatHandlerBase and UAIChatPlus_ImageHandlerBase. You only need to register the corresponding callback delegate.

Before sending a request, you need to set the parameters and the message to be sent for the API. This part is set using FAIChatPlus_xxxChatRequestBody. The specific response content is also parsed into FAIChatPlus_xxxChatResponseBody. When receiving a callback, you can obtain the ResponseBody through a specific interface.
Before sending a request, you need to set the parameters and messages for the API. This is done by setting up the FAIChatPlus_xxxChatRequestBody. The specific response content is also parsed into FAIChatPlus_xxxChatResponseBody. When receiving a callback, you can retrieve the ResponseBody through a specific interface.

More source code details can be obtained on the UE marketplace: [AIChatPlus](https://www.unrealengine.com/marketplace/zh-CN/product/aichatplus-ai-chat-integration-openai-azure-claude-gemini)
More source code details can be found on UE Marketplace: [AIChatPlus](https://www.unrealengine.com/marketplace/zh-CN/product/aichatplus-ai-chat-integration-openai-azure-claude-gemini)


###Release Notes

### v1.3.0 - 2024.9.23
#### v1.3.0 - 2024.9.23

Major Update
Heavy update

Integrated llama.cpp to support local offline execution of large models.
Integrated llama.cpp, supporting local offline execution of large models.

#### v1.2.0 - 2024.08.20

Support OpenAI Image Edit/Image Variation
Support OpenAI Image Edit/Image Variation.

Support the Ollama API, support automatically obtaining the list of models supported by Ollama
Support Ollama API, support automatically fetching the list of models supported by Ollama.

#### v1.1.0 - 2024.08.07

Support blueprint
Support blueprint.

#### v1.0.0 - 2024.08.05

Complete basic functionality

Support OpenAI, Azure, Claude, Gemini

Integrated feature-rich editor chat tool
Built-in feature-rich chat tool editor

--8<-- "footer_en.md"

Expand Down
Loading

0 comments on commit f298e6a

Please sign in to comment.