Skip to content

Latest commit

 

History

History
336 lines (251 loc) · 14.1 KB

README.md

File metadata and controls

336 lines (251 loc) · 14.1 KB
Raw
page_type languages products urlFragment name description
sample
azdeveloper
java
bicep
javascript
html
azure
azure-spring-apps
azure-blob-storage
azure-openai
spring-chatgpt-sample
Spring ChatGPT Application using Azure OpenAI on Azure Spring Apps (Java)
This sample shows how to build a ChatGPT like application with Java in Spring and run on Azure Spring Apps. It enables ChatGPT to use your private data to answer the questions.

Spring ChatGPT Sample

This sample shows how to build a ChatGPT like application in Spring and run on Azure Spring Apps. It enables ChatGPT to use your private data to answer the questions.

How it works

Workflow

  1. Query flow (Web API)
    1. Convert the user's query text to an embedding.
    2. Query Top-K nearest text chunks from the vector store (by cosine similarity).
    3. Populate the prompt template with the chunks.
    4. Call to OpenAI text completion API.
  2. Indexing flow (CLI)
    1. Load the documents from the local disk / Azure storage.
    2. Split the text into chunks.
    3. Convert text chunks into embeddings
    4. Save the embeddings into Vector Store

Run with Azure Developer CLI (AZD)

You can provision Azure resources and run this application on Azure with Azure Developer CLI (AZD), which can help you build and run your application in the cloud quickly without running multiple az cli commands step by step. For more information, see Azure Developer CLI.

Let's jump in and get this up and running in Azure. When you are finished, you will have a fully functional chatgpt app deployed to the cloud. In later steps, you'll see how to setup a pipeline and run the application.

"Screenshot of deployed chatgpt app"

Screenshot of the deployed chatgpt app

Prerequisites

The following prerequisites are required to use this application. Please ensure that you have them all installed locally.

If you are using Azure Developer CLI with the version lower than 1.2.0, then you will need to enable the feature for Azure Spring Apps support manually by the following command:

azd config set alpha.springapp on

Quickstart

To learn how to get started with any template, follow the steps in this quickstart with this template(Azure-Samples/spring-chatgpt-sample).

This quickstart will show you how to authenticate on Azure, initialize using a template, provision infrastructure and deploy code on Azure via the following commands:

# Log in to azd. Only required once per-install.
azd auth login

# First-time project setup. Initialize a project in the current directory, using this template. 
azd init --template Azure-Samples/spring-chatgpt-sample

# Provision and deploy to Azure
azd up

The template by default uses a pre-built documentation vector store of the public documents of the Azure Spring Apps.

If you want to load your own documents to the vector store, you can use the following command before running azd up:

# under the root of the project
mvn clean package
java -jar spring-chatgpt-sample-cli/target/spring-chatgpt-sample-cli-0.0.1-SNAPSHOT.jar --from=/<path>/<to>/<your>/<documents>

Application Architecture

This application utilizes the following Azure resources:

Here's a high level architecture diagram that illustrates these components. Notice that these are all contained within a single resource group, that will be created for you when you create the resources.

"Application architecture diagram"

This template provisions resources to an Azure subscription that you will select upon provisioning them. Please refer to the Pricing calculator for Microsoft Azure and, if needed, update the included Azure resource definitions found in infra/main.bicep to suit your needs.

Application Code

This template is structured to follow the Azure Developer CLI. You can learn more about azd architecture in the official documentation.

Next Steps

At this point, you have a complete application deployed on Azure. But there is much more that the Azure Developer CLI can do. These next steps will introduce you to additional commands that will make creating applications on Azure much easier. Using the Azure Developer CLI, you can delete the resources easily.

  • azd down - to delete all the Azure resources created with this template

Additional azd commands

The Azure Developer CLI includes many other commands to help with your Azure development experience. You can view these commands at the terminal by running azd help. You can also view the full list of commands on our Azure Developer CLI command page.

Run with Azure CLI

As an alternative to AZD, you can run this sample app using the Azure CLI by following these steps.

Prerequisites

  • JDK 17
  • Maven
  • Azure CLI
  • An Azure subscription with access granted to Azure OpenAI (see more here)

Prepare Azure Spring Apps instance

  1. Use the following commands to define variables for this quickstart with the names of your resources and desired settings:

    LOCATION="eastus"
RESOURCE_GROUP="<resource-group-name>"
MANAGED_ENVIRONMENT="<Azure-Container-Apps-environment-name>"
SERVICE_NAME="<Azure-Spring-Apps-instance-name>"
APP_NAME="<Spring-app-name>"
OPENAI_RESOURCE_NAME="<Azure-OpenAI-resource-name>"

  2. Use the following command to create a resource group:

    az group create \
    --resource-group ${RESOURCE_GROUP} \
    --location ${LOCATION}

  3. An Azure Container Apps environment creates a secure boundary around a group of applications. Apps deployed to the same environment are deployed in the same virtual network and write logs to the same log analytics workspace. For more information, see Log Analytics workspace overview. Use the following command to create the environment:

    az containerapp env create \
    --resource-group ${RESOURCE_GROUP} \
    --name ${MANAGED_ENVIRONMENT} \
    --location ${LOCATION} \
    --enable-workload-profiles

  4. Use the following command to create a variable to store the environment resource ID:

    MANAGED_ENV_RESOURCE_ID=$(az containerapp env show \
    --resource-group ${RESOURCE_GROUP} \
    --name ${MANAGED_ENVIRONMENT} \
    --query id \
    --output tsv)

  5. Use the following command to create an Azure Spring Apps service instance. An instance of the Azure Spring Apps Standard consumption and dedicated plan is built on top of the Azure Container Apps environment. Create your Azure Spring Apps instance by specifying the resource ID of the environment you created.

    az spring create \
    --resource-group ${RESOURCE_GROUP} \
    --name ${SERVICE_NAME} \
    --managed-environment ${MANAGED_ENV_RESOURCE_ID} \
    --sku standardGen2 \
    --location ${LOCATION}

Prepare Azure OpenAI Service

  1. Run the following command to create an Azure OpenAI resource in the the resource group.

    az cognitiveservices account create \
   -n ${OPENAI_RESOURCE_NAME} \
   -g ${RESOURCE_GROUP} \
   -l ${LOCATION} \
   --kind OpenAI \
   --sku s0 \
   --custom-domain ${OPENAI_RESOURCE_NAME}

  2. Create the model deployments for text-embedding-ada-002 and gpt-35-turbo in your Azure OpenAI service.

    az cognitiveservices account deployment create \
   -g ${RESOURCE_GROUP} \
   -n ${OPENAI_RESOURCE_NAME} \
   --deployment-name text-embedding-ada-002 \
   --model-name text-embedding-ada-002 \
   --model-version "2"  \
   --model-format OpenAI

az cognitiveservices account deployment create \
   -g ${RESOURCE_GROUP} \
   -n ${OPENAI_RESOURCE_NAME} \
   --deployment-name gpt-35-turbo \
   --model-name gpt-35-turbo \
   --model-version "0301"  \
   --model-format OpenAI

Clone and Build the repo

  1. Run git clone https://github.com/Azure-Samples/spring-chatgpt-sample.git
  2. Run cd spring-chatgpt-sample.
  3. Run cp env.sh.sample env.sh and substitute the placeholders.
  4. Build with mvn clean package.

Preprocess the documents

Before running the web app, you need to preprocess the documents and load them into the vector store:

source env.sh
java -jar spring-chatgpt-sample-cli/target/spring-chatgpt-sample-cli-0.0.1-SNAPSHOT.jar --from=/<path>/<to>/<your>/<documents>

Or dowload the pre-built vector store of the public documents of the Azure Spring Apps.

Run in local

To run the demo in the local machine, please follow these steps:

  1. Launch the web app

    source env.sh
java -jar spring-chatgpt-sample-webapi/target/spring-chatgpt-sample-webapi-0.0.1-SNAPSHOT.jar

  2. Open http://localhost:8080 in your browser.

Run in Azure Spring Apps

  1. Use the following command to specify the app name on Azure Spring Apps and to allocate required resources:

    az spring app create \
   --resource-group ${RESOURCE_GROUP} \
   --service ${SERVICE_NAME} \
   --name ${APP_NAME} \
   --cpu 2 \
   --memory 4Gi \
   --min-replicas 2 \
   --max-replicas 2 \
   --assign-endpoint true

  2. Create a Azure storage account and a file share, then add the storage link in the Azure Container Apps environment by using the following commands. The az containerapp env storage set command creates a link between the environment and the file share.

    STORAGE_ACCOUNT_NAME="<storage-account-name>"
FILE_SHARE_NAME="vectorstore"
STORAGE_MOUNT_NAME="vectorstore"

az storage account create \
   --resource-group ${RESOURCE_GROUP} \
   --name ${STORAGE_ACCOUNT_NAME} \
   --kind StorageV2 \
   --sku Standard_LRS \
   --enable-large-file-share \
   --output none   

az storage share-rm create \
   --resource-group ${RESOURCE_GROUP} \
   --storage-account ${STORAGE_ACCOUNT_NAME} \
   --name ${FILE_SHARE_NAME} \
   --quota 1024 \
   --enabled-protocols SMB \
   --output none

STORAGE_ACCOUNT_KEY=$(az storage account keys list \
   --resource-group ${RESOURCE_GROUP} \
   --account-name ${STORAGE_ACCOUNT_NAME} \
   --query "[0].value" \
   -o tsv)

az containerapp env storage set \
   --resource-group ${RESOURCE_GROUP} \
   --name ${MANAGED_ENVIRONMENT} \
   --storage-name ${STORAGE_MOUNT_NAME} \
   --azure-file-account-name ${STORAGE_ACCOUNT_NAME} \
   --azure-file-account-key ${STORAGE_ACCOUNT_KEY} \
   --azure-file-share-name ${FILE_SHARE_NAME} \
   --access-mode ReadOnly

  3. Add the persistent storage to the app by using the following command:

    az spring app append-persistent-storage \
   --resource-group ${RESOURCE_GROUP} \
   --service ${SERVICE_NAME} \
   --name ${APP_NAME} \
   --persistent-storage-type AzureFileVolume \
   --mount-path /opt/spring-chatgpt-sample \
   --storage-name ${STORAGE_MOUNT_NAME}

  4. Upload the vector store file to the Azure storage account built in the previous step.

    az storage file upload -s ${FILE_SHARE_NAME} --account-name ${STORAGE_ACCOUNT_NAME} \
   --account-key ${STORAGE_ACCOUNT_KEY} --source ./doc_store.json

  5. Use the following command to deploy the .jar file for the app:

    az spring app deploy \
   --resource-group ${RESOURCE_GROUP} \
   --service ${SERVICE_NAME} \
   --name ${APP_NAME} \
   --artifact-path spring-chatgpt-sample-webapi/target/spring-chatgpt-sample-webapi-0.0.1-SNAPSHOT.jar \
   --env AZURE_OPENAI_ENDPOINT=<your_azure_openai_endpoint> AZURE_OPENAI_APIKEY=<your_api_key> AZURE_OPENAI_CHATDEPLOYMENTID=gpt-35-turbo AZURE_OPENAI_EMBEDDINGDEPLOYMENTID=text-embedding-ada-002 VECTORSTORE_FILE=/opt/spring-chatgpt-sample/doc_store.json \
   --runtime-version Java_17

Reporting Issues and Feedback

If you have any feature requests, issues, or areas for improvement, please file an issue. To keep up-to-date, ask questions, or share suggestions, join our GitHub Discussions. You may also contact us via [email protected].