Skip to content

Commit

Permalink
Documentation for Backend (#120)
Browse files Browse the repository at this point in the history
* docs: added documentation for Google, HuggingFace and LangChain

* docs: added assets for documentation

* chore: removed assets to avoid CI issues

---------

Signed-off-by: Kannav02 <[email protected]>
Co-authored-by: Song Luar <[email protected]>
  • Loading branch information
Kannav02 and luarss authored Jan 16, 2025
1 parent 5f57e2e commit 1441a9f
Showing 1 changed file with 116 additions and 0 deletions.
116 changes: 116 additions & 0 deletions backend/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
# Backend For ORAssistant

This folder contains the backend code for the ORAssistant. Follow the instructions below to set up the environment and run the backend.

## Preparing the Environment Variables

To configure the application, you'll need to set up the environment variables in your `.env` file:

The given command would copy the template for environment variables to a local `.env` file


```
cp .env.example .env
```


### Setting Up Google API Key and Credentials Variables

There are 2 variables that needs to be set up

- `GOOGLE_API_KEY`

This key is used to access the various google cloud functions.
- Go to [Google Cloud Console](https://console.cloud.google.com/)
- Create new project or select existing one
- Enable required APIs:
- Google Gemini API
- Vertex AI API
- Go to APIs & Services > Credentials
- Click "Create Credentials" > "API Key"
- Copy the generated key and it to the `.env` file

- `GOOGLE_APPLICATION_CREDENTIALS`
Since most of the GCP functions / services would be used by our app, we need to have a special credential that would allow `ORAssistant`'s access to the GCP
Steps to set up Service Account Credentials:
- In Google Cloud Console, go to IAM & Admin > Service Accounts
- Click "Create Service Account"
Fill in service account details
- Grant required roles:
- Vertex AI User
- Vertex AI Service Agent
- Create key (JSON format)
- Download JSON file
- Store securely and add path to .env:
`GOOGLE_APPLICATION_CREDENTIALS=/path/to/credentials.json`

**NOTE**: The user might need billing to be set up on google cloud account and make sure to name the file as `credentials.json` as this would be ignored by `.git` and wouldn't be exposed on Github

### Setting Up LangChain Variables

There are 4 variables that needs to be set up

- `LANGCHAIN_TRACING_V2`

This is used to enable LangChain's debugging and monitoring features,
can be set to either `true` or `false`

- `LANGCHAIN_ENDPOINT`

The URL endpoint for LangSmith (LangChain's monitoring platform).
Default value should be `https://api.smith.langchain.com` for cloud-hosted LangSmith.
Used to send trace data, metrics, and debugging information from your LangChain applications.

- `LANGCHAIN_API_KEY`

API key required to authenticate with LangSmith platform.
- Get your key from: https://smith.langchain.com/
- Create account if you don't have one
- Navigate to Settings > API Keys
- Create new API key
- Format: starts with `lsv2_` followed by a unique string

- `LANGCHAIN_PROJECT`

Project identifier in LangSmith to organize and track your traces.
- Create new project in LangSmith dashboard
- Use the project name or ID provided
- Example: "my-rag-project"
- Helps organize different applications/environments
- Multiple apps can share same project


### Setting Up Huggingface User Access Token

To set up the `HF_TOKEN` variable in `.env` file , go through the following instructions:

- Go the official website for [Huggingface](https://huggingface.co/) and either Login or Sign up.
- On the main page click on user access token
- Click on create access token
- Provide only Read Instruction for the token and Click on Generate Token




Provide the value for `HF_TOKEN` with the token that is generated

## Running the Application

### Install Required Packages

Ensure you have the necessary dependencies installed by running:

```bash
pip install -r requirements.txt
```
### Docker Command

If you want to run an isolated container for backend, you can use the following command

```bash
docker build -t (image_name) .
```

Make sure you are in the backend folder before running the above command.

**NOTE**: The project does support a `docker-compose` file that would run all of the containers together

1 comment on commit 1441a9f

@luarss
Copy link
Collaborator

@luarss luarss commented on 1441a9f Jan 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

===================================
==> Dataset: EDA Corpus
==> Running tests for agent-retriever
/home/luarss/actions-runner/_work/ORAssistant/ORAssistant/evaluation/.venv/lib/python3.12/site-packages/deepeval/init.py:49: UserWarning: You are using deepeval version 1.4.9, however version 2.1.7 is available. You should consider upgrading via the "pip install --upgrade deepeval" command.
warnings.warn(

Fetching 2 files: 0%| | 0/2 [00:00<?, ?it/s]
Fetching 2 files: 50%|█████ | 1/2 [00:00<00:00, 7.99it/s]
Fetching 2 files: 100%|██████████| 2/2 [00:00<00:00, 9.52it/s]

Evaluating: 0%| | 0/100 [00:00<?, ?it/s]
Evaluating: 1%| | 1/100 [00:14<24:15, 14.70s/it]
Evaluating: 2%|▏ | 2/100 [00:25<20:06, 12.31s/it]
Evaluating: 3%|▎ | 3/100 [00:39<21:29, 13.29s/it]
Evaluating: 4%|▍ | 4/100 [00:49<18:58, 11.86s/it]
Evaluating: 5%|▌ | 5/100 [01:00<18:32, 11.72s/it]
Evaluating: 6%|▌ | 6/100 [01:11<17:52, 11.41s/it]
Evaluating: 7%|▋ | 7/100 [01:24<18:31, 11.96s/it]
Evaluating: 8%|▊ | 8/100 [01:42<21:03, 13.73s/it]
Evaluating: 9%|▉ | 9/100 [01:58<22:05, 14.57s/it]
Evaluating: 10%|█ | 10/100 [02:09<20:14, 13.50s/it]
Evaluating: 11%|█ | 11/100 [02:20<18:44, 12.63s/it]
Evaluating: 12%|█▏ | 12/100 [02:30<17:07, 11.68s/it]
Evaluating: 13%|█▎ | 13/100 [02:42<17:09, 11.83s/it]
Evaluating: 14%|█▍ | 14/100 [02:54<16:58, 11.84s/it]
Evaluating: 15%|█▌ | 15/100 [03:06<16:58, 11.98s/it]
Evaluating: 16%|█▌ | 16/100 [03:18<16:42, 11.93s/it]
Evaluating: 17%|█▋ | 17/100 [03:30<16:40, 12.06s/it]
Evaluating: 18%|█▊ | 18/100 [03:40<15:47, 11.55s/it]
Evaluating: 19%|█▉ | 19/100 [03:50<14:55, 11.06s/it]
Evaluating: 20%|██ | 20/100 [04:01<14:42, 11.03s/it]
Evaluating: 21%|██ | 21/100 [04:12<14:29, 11.00s/it]
Evaluating: 22%|██▏ | 22/100 [04:25<14:49, 11.41s/it]
Evaluating: 23%|██▎ | 23/100 [04:36<14:37, 11.39s/it]
Evaluating: 24%|██▍ | 24/100 [04:46<13:57, 11.02s/it]
Evaluating: 25%|██▌ | 25/100 [04:56<13:12, 10.57s/it]
Evaluating: 26%|██▌ | 26/100 [05:05<12:38, 10.25s/it]
Evaluating: 27%|██▋ | 27/100 [05:16<12:49, 10.55s/it]
Evaluating: 28%|██▊ | 28/100 [05:28<13:04, 10.90s/it]
Evaluating: 29%|██▉ | 29/100 [05:39<12:47, 10.81s/it]
Evaluating: 30%|███ | 30/100 [05:50<12:50, 11.01s/it]
Evaluating: 31%|███ | 31/100 [06:00<12:20, 10.73s/it]
Evaluating: 32%|███▏ | 32/100 [06:13<12:53, 11.38s/it]
Evaluating: 33%|███▎ | 33/100 [06:24<12:37, 11.31s/it]
Evaluating: 34%|███▍ | 34/100 [06:36<12:39, 11.50s/it]
Evaluating: 35%|███▌ | 35/100 [06:51<13:24, 12.38s/it]
Evaluating: 36%|███▌ | 36/100 [07:01<12:36, 11.81s/it]
Evaluating: 37%|███▋ | 37/100 [07:11<11:56, 11.37s/it]
Evaluating: 38%|███▊ | 38/100 [07:22<11:33, 11.19s/it]
Evaluating: 39%|███▉ | 39/100 [07:32<10:56, 10.76s/it]
Evaluating: 40%|████ | 40/100 [07:41<10:20, 10.35s/it]
Evaluating: 41%|████ | 41/100 [07:53<10:36, 10.79s/it]
Evaluating: 42%|████▏ | 42/100 [08:04<10:23, 10.75s/it]
Evaluating: 43%|████▎ | 43/100 [08:16<10:40, 11.23s/it]
Evaluating: 44%|████▍ | 44/100 [08:29<10:48, 11.59s/it]
Evaluating: 45%|████▌ | 45/100 [08:41<10:55, 11.92s/it]
Evaluating: 46%|████▌ | 46/100 [08:53<10:42, 11.89s/it]
Evaluating: 47%|████▋ | 47/100 [09:05<10:33, 11.96s/it]
Evaluating: 48%|████▊ | 48/100 [09:17<10:25, 12.03s/it]
Evaluating: 49%|████▉ | 49/100 [09:28<09:48, 11.55s/it]
Evaluating: 50%|█████ | 50/100 [09:40<09:52, 11.86s/it]
Evaluating: 51%|█████ | 51/100 [09:50<09:10, 11.24s/it]
Evaluating: 52%|█████▏ | 52/100 [10:03<09:17, 11.62s/it]
Evaluating: 53%|█████▎ | 53/100 [10:14<08:55, 11.39s/it]
Evaluating: 54%|█████▍ | 54/100 [10:26<08:51, 11.55s/it]
Evaluating: 55%|█████▌ | 55/100 [10:37<08:34, 11.43s/it]
Evaluating: 56%|█████▌ | 56/100 [10:47<08:06, 11.05s/it]
Evaluating: 57%|█████▋ | 57/100 [10:58<07:55, 11.06s/it]
Evaluating: 58%|█████▊ | 58/100 [11:09<07:40, 10.98s/it]
Evaluating: 59%|█████▉ | 59/100 [11:19<07:27, 10.91s/it]
Evaluating: 60%|██████ | 60/100 [11:32<07:37, 11.43s/it]
Evaluating: 61%|██████ | 61/100 [11:42<07:09, 11.01s/it]
Evaluating: 62%|██████▏ | 62/100 [11:53<06:56, 10.96s/it]
Evaluating: 63%|██████▎ | 63/100 [12:07<07:23, 11.99s/it]
Evaluating: 64%|██████▍ | 64/100 [12:18<06:53, 11.50s/it]
Evaluating: 65%|██████▌ | 65/100 [12:34<07:29, 12.85s/it]
Evaluating: 66%|██████▌ | 66/100 [12:46<07:11, 12.69s/it]
Evaluating: 67%|██████▋ | 67/100 [12:57<06:43, 12.22s/it]
Evaluating: 68%|██████▊ | 68/100 [13:07<06:06, 11.46s/it]
Evaluating: 69%|██████▉ | 69/100 [13:18<05:51, 11.34s/it]
Evaluating: 70%|███████ | 70/100 [13:28<05:29, 10.99s/it]
Evaluating: 71%|███████ | 71/100 [13:40<05:23, 11.14s/it]
Evaluating: 72%|███████▏ | 72/100 [13:50<05:06, 10.96s/it]
Evaluating: 73%|███████▎ | 73/100 [14:02<05:07, 11.38s/it]
Evaluating: 74%|███████▍ | 74/100 [14:13<04:51, 11.22s/it]
Evaluating: 75%|███████▌ | 75/100 [14:25<04:43, 11.34s/it]
Evaluating: 76%|███████▌ | 76/100 [14:36<04:31, 11.31s/it]
Evaluating: 77%|███████▋ | 77/100 [14:48<04:23, 11.46s/it]
Evaluating: 78%|███████▊ | 78/100 [14:59<04:08, 11.27s/it]
Evaluating: 79%|███████▉ | 79/100 [15:10<03:53, 11.13s/it]
Evaluating: 80%|████████ | 80/100 [15:21<03:41, 11.05s/it]
Evaluating: 81%|████████ | 81/100 [15:31<03:25, 10.83s/it]
Evaluating: 82%|████████▏ | 82/100 [15:41<03:12, 10.71s/it]
Evaluating: 83%|████████▎ | 83/100 [15:52<03:00, 10.60s/it]
Evaluating: 84%|████████▍ | 84/100 [16:02<02:48, 10.54s/it]
Evaluating: 85%|████████▌ | 85/100 [16:17<02:57, 11.84s/it]
Evaluating: 86%|████████▌ | 86/100 [16:29<02:46, 11.90s/it]
Evaluating: 87%|████████▋ | 87/100 [16:39<02:28, 11.42s/it]
Evaluating: 88%|████████▊ | 88/100 [16:50<02:16, 11.35s/it]
Evaluating: 89%|████████▉ | 89/100 [17:01<02:03, 11.23s/it]
Evaluating: 90%|█████████ | 90/100 [17:15<01:59, 11.90s/it]
Evaluating: 91%|█████████ | 91/100 [17:29<01:53, 12.59s/it]
Evaluating: 92%|█████████▏| 92/100 [17:40<01:36, 12.05s/it]
Evaluating: 93%|█████████▎| 93/100 [17:54<01:28, 12.63s/it]
Evaluating: 94%|█████████▍| 94/100 [18:04<01:11, 11.99s/it]
Evaluating: 95%|█████████▌| 95/100 [18:15<00:58, 11.68s/it]
Evaluating: 96%|█████████▌| 96/100 [18:26<00:45, 11.27s/it]
Evaluating: 97%|█████████▋| 97/100 [18:36<00:33, 11.10s/it]
Evaluating: 98%|█████████▊| 98/100 [18:48<00:22, 11.42s/it]
Evaluating: 99%|█████████▉| 99/100 [19:00<00:11, 11.33s/it]
Evaluating: 100%|██████████| 100/100 [19:12<00:00, 11.61s/it]
Evaluating: 100%|██████████| 100/100 [19:12<00:00, 11.52s/it]
✨ You're running DeepEval's latest Contextual Precision Metric! (using
gemini-1.5-pro-002, strict=False, async_mode=True)...
✨ You're running DeepEval's latest Contextual Recall Metric! (using
gemini-1.5-pro-002, strict=False, async_mode=True)...
✨ You're running DeepEval's latest Hallucination Metric! (using
gemini-1.5-pro-002, strict=False, async_mode=True)...

Evaluating 100 test case(s) in parallel: | | 0% (0/100) [Time Taken: 00:00, ?test case/s]
Evaluating 100 test case(s) in parallel: | | 1% (1/100) [Time Taken: 00:11, 11.30s/test case]
Evaluating 100 test case(s) in parallel: |▏ | 2% (2/100) [Time Taken: 00:14, 6.47s/test case]
Evaluating 100 test case(s) in parallel: |▌ | 5% (5/100) [Time Taken: 00:15, 2.03s/test case]
Evaluating 100 test case(s) in parallel: |▌ | 6% (6/100) [Time Taken: 00:15, 1.54s/test case]
Evaluating 100 test case(s) in parallel: |▉ | 9% (9/100) [Time Taken: 00:15, 1.22test case/s]
Evaluating 100 test case(s) in parallel: |█ | 10% (10/100) [Time Taken: 00:15, 1.44test case/s]
Evaluating 100 test case(s) in parallel: |█▏ | 12% (12/100) [Time Taken: 00:16, 1.83test case/s]
Evaluating 100 test case(s) in parallel: |█▌ | 15% (15/100) [Time Taken: 00:16, 2.72test case/s]
Evaluating 100 test case(s) in parallel: |█▊ | 18% (18/100) [Time Taken: 00:17, 3.04test case/s]
Evaluating 100 test case(s) in parallel: |██▎ | 23% (23/100) [Time Taken: 00:17, 5.47test case/s]
Evaluating 100 test case(s) in parallel: |██▌ | 25% (25/100) [Time Taken: 00:18, 6.23test case/s]
Evaluating 100 test case(s) in parallel: |██▊ | 28% (28/100) [Time Taken: 00:18, 8.05test case/s]
Evaluating 100 test case(s) in parallel: |███ | 30% (30/100) [Time Taken: 00:18, 6.98test case/s]
Evaluating 100 test case(s) in parallel: |███▏ | 32% (32/100) [Time Taken: 00:18, 7.67test case/s]
Evaluating 100 test case(s) in parallel: |███▍ | 34% (34/100) [Time Taken: 00:20, 3.81test case/s]
Evaluating 100 test case(s) in parallel: |███▌ | 35% (35/100) [Time Taken: 00:21, 2.51test case/s]
Evaluating 100 test case(s) in parallel: |███▌ | 36% (36/100) [Time Taken: 00:21, 2.44test case/s]
Evaluating 100 test case(s) in parallel: |███▋ | 37% (37/100) [Time Taken: 00:21, 2.62test case/s]
Evaluating 100 test case(s) in parallel: |███▊ | 38% (38/100) [Time Taken: 00:22, 2.41test case/s]
Evaluating 100 test case(s) in parallel: |███▉ | 39% (39/100) [Time Taken: 00:23, 2.04test case/s]
Evaluating 100 test case(s) in parallel: |████▏ | 42% (42/100) [Time Taken: 00:23, 3.83test case/s]
Evaluating 100 test case(s) in parallel: |████▌ | 45% (45/100) [Time Taken: 00:23, 5.89test case/s]
Evaluating 100 test case(s) in parallel: |████▊ | 48% (48/100) [Time Taken: 00:23, 8.04test case/s]
Evaluating 100 test case(s) in parallel: |█████ | 50% (50/100) [Time Taken: 00:23, 9.50test case/s]
Evaluating 100 test case(s) in parallel: |█████▏ | 52% (52/100) [Time Taken: 00:23, 10.97test case/s]
Evaluating 100 test case(s) in parallel: |█████▍ | 54% (54/100) [Time Taken: 00:24, 5.37test case/s]
Evaluating 100 test case(s) in parallel: |█████▌ | 56% (56/100) [Time Taken: 00:24, 6.68test case/s]
Evaluating 100 test case(s) in parallel: |█████▊ | 58% (58/100) [Time Taken: 00:25, 6.31test case/s]
Evaluating 100 test case(s) in parallel: |██████ | 60% (60/100) [Time Taken: 00:25, 7.67test case/s]
Evaluating 100 test case(s) in parallel: |██████▏ | 62% (62/100) [Time Taken: 00:25, 9.02test case/s]
Evaluating 100 test case(s) in parallel: |██████▍ | 64% (64/100) [Time Taken: 00:25, 8.90test case/s]
Evaluating 100 test case(s) in parallel: |██████▌ | 66% (66/100) [Time Taken: 00:25, 9.35test case/s]
Evaluating 100 test case(s) in parallel: |██████▊ | 68% (68/100) [Time Taken: 00:25, 9.67test case/s]
Evaluating 100 test case(s) in parallel: |███████ | 70% (70/100) [Time Taken: 00:27, 4.27test case/s]
Evaluating 100 test case(s) in parallel: |███████ | 71% (71/100) [Time Taken: 00:27, 3.84test case/s]
Evaluating 100 test case(s) in parallel: |███████▏ | 72% (72/100) [Time Taken: 00:27, 3.47test case/s]
Evaluating 100 test case(s) in parallel: |███████▍ | 74% (74/100) [Time Taken: 00:28, 4.40test case/s]
Evaluating 100 test case(s) in parallel: |███████▌ | 75% (75/100) [Time Taken: 00:30, 1.72test case/s]
Evaluating 100 test case(s) in parallel: |███████▌ | 76% (76/100) [Time Taken: 00:30, 1.79test case/s]
Evaluating 100 test case(s) in parallel: |███████▋ | 77% (77/100) [Time Taken: 00:30, 2.04test case/s]
Evaluating 100 test case(s) in parallel: |███████▊ | 78% (78/100) [Time Taken: 00:30, 2.49test case/s]
Evaluating 100 test case(s) in parallel: |███████▉ | 79% (79/100) [Time Taken: 00:31, 2.25test case/s]
Evaluating 100 test case(s) in parallel: |████████ | 81% (81/100) [Time Taken: 00:31, 3.46test case/s]
Evaluating 100 test case(s) in parallel: |████████▎ | 83% (83/100) [Time Taken: 00:31, 4.63test case/s]
Evaluating 100 test case(s) in parallel: |████████▍ | 84% (84/100) [Time Taken: 00:32, 4.42test case/s]
Evaluating 100 test case(s) in parallel: |████████▌ | 85% (85/100) [Time Taken: 00:32, 4.62test case/s]
Evaluating 100 test case(s) in parallel: |████████▌ | 86% (86/100) [Time Taken: 00:32, 4.10test case/s]
Evaluating 100 test case(s) in parallel: |████████▊ | 88% (88/100) [Time Taken: 00:33, 3.84test case/s]
Evaluating 100 test case(s) in parallel: |████████▉ | 89% (89/100) [Time Taken: 00:33, 4.08test case/s]
Evaluating 100 test case(s) in parallel: |█████████ | 90% (90/100) [Time Taken: 00:33, 4.64test case/s]
Evaluating 100 test case(s) in parallel: |█████████▏| 92% (92/100) [Time Taken: 00:33, 5.86test case/s]
Evaluating 100 test case(s) in parallel: |█████████▎| 93% (93/100) [Time Taken: 00:34, 3.58test case/s]
Evaluating 100 test case(s) in parallel: |█████████▍| 94% (94/100) [Time Taken: 00:35, 2.73test case/s]
Evaluating 100 test case(s) in parallel: |█████████▌| 95% (95/100) [Time Taken: 00:36, 1.79test case/s]
Evaluating 100 test case(s) in parallel: |█████████▌| 96% (96/100) [Time Taken: 00:36, 2.21test case/s]
Evaluating 100 test case(s) in parallel: |█████████▋| 97% (97/100) [Time Taken: 00:36, 1.95test case/s]
Evaluating 100 test case(s) in parallel: |█████████▊| 98% (98/100) [Time Taken: 00:37, 2.24test case/s]
Evaluating 100 test case(s) in parallel: |█████████▉| 99% (99/100) [Time Taken: 00:37, 2.30test case/s]
Evaluating 100 test case(s) in parallel: |██████████|100% (100/100) [Time Taken: 00:39, 1.10test case/s]
Evaluating 100 test case(s) in parallel: |██████████|100% (100/100) [Time Taken: 00:39, 2.52test case/s]
✓ Tests finished 🎉! Run 'deepeval login' to save and analyze evaluation results
on Confident AI.
‼️ Friendly reminder 😇: You can also run evaluations with ALL of deepeval's
metrics directly on Confident AI instead.
Average Metric Scores:
Contextual Precision 0.7209007936507936
Contextual Recall 0.8573571428571428
Hallucination 0.5286605339105339
Metric Passrates:
Contextual Precision 0.68
Contextual Recall 0.82
Hallucination 0.54

Please sign in to comment.