From e69d61637bf96d2314d2bde0203b689ad491aced Mon Sep 17 00:00:00 2001 From: Aravind Ramaraju <51247042+aramaraju@users.noreply.github.com> Date: Wed, 3 Apr 2024 17:41:27 -0700 Subject: [PATCH 1/3] Update TESTING.md --- TESTING.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/TESTING.md b/TESTING.md index d74de1a5bc..15624a60ca 100644 --- a/TESTING.md +++ b/TESTING.md @@ -35,6 +35,7 @@ - [ ] [Automatic repository recognition](#automatic-repository-recognition) - [ ] [Persistent custom repository](#persistent-custom-repository) - [ ] [Colour scheme changes](#colour-scheme-changes) + - [ ] [Claude 3 Enterprise Testing](#claude-3-enterprise-testing) - Context - [ ] [PLG / SYMF / Local Keyword search](#local-keyword-search) - Model dropdown @@ -531,3 +532,14 @@ Repeat the above starting from different themes. #### Expected behaviour Changing theme should lead to full repaint of the colours according to the current theme. + + +### Claude 3 Enterprise Testing +#### Cody Gateway Instance +- [ ] Ask Cody a question in a new chat window. The question should include a request for Cody to generate code with the site-admin configuration is set to **Claude 3 Sonnet** for chatModel and **Claude 3 Haiku** for completionModel +- [ ] Ask Cody a question in a new chat window. The question should include a request for Cody to generate code with the site-admin configuration is set to **Claude 3 Opus** for chatModel and **Claude 3 Haiku** for completionModel +- [ ] Generate a completion using the manual-trigger key binding on any primary languages (Javascript, Typescript, TypescriptReact, Python, Go). + +#### AWS Bedrock Instance +- [ ] Ask Cody a question in a new chat window. The question should include a request for Cody to generate code with the site-admin configuration is set to **Claude 3 Sonnet** for chatModel and **Claude 3 Haiku** for completionModel +- [ ] Generate a completion using the manual-trigger key binding on any primary languages (Javascript, Typescript, TypescriptReact, Python, Go). From 917791e4f4d1f33fdc696fd27baea7e6a405aad9 Mon Sep 17 00:00:00 2001 From: Aravind Ramaraju Date: Fri, 12 Apr 2024 13:06:03 -0700 Subject: [PATCH 2/3] Created two cases for Enterprise testing, one for Cody Gateway instance and one for AWS Bedrock. Removed the validation on the configuration page for the Enterprise testing, the team will simply test against whatever configuration is on the instances --- TESTING.md | 51 ++++++++++++++++++++++++++++++++++----------------- 1 file changed, 34 insertions(+), 17 deletions(-) diff --git a/TESTING.md b/TESTING.md index 15624a60ca..f2f08bf3da 100644 --- a/TESTING.md +++ b/TESTING.md @@ -477,20 +477,48 @@ To open the context menu: 2. Create new chat 3. User doesn't see model dropdown -### Enterprise account +### Cody Enterprise - Cody Gateway Instance #### Chat -1. Login to Cody Enterprise account -2. Create new chat -3. User should see the default model in the dropdown but is unable to change it +1. Login to the Cody Enterprise Gateway Instance +2. Create a new chat +3. User should not be able to change the LLM model on the chat window +4. Ask Cody a question in the new chat window. #### Commands -1. Login to Cody Enterprise account +1. Login to the Cody Enterprise Cody Gateway Instance 2. Go to commands panel 3. Trigger command -4. Command should be executed with default model +4. Command should be executed + +#### Completions + +1. Login to the Cody Enterprise Cody Gateway Instance +2. Generate a completion using the manual-trigger key binding on any primary languages (Javascript, Typescript, TypescriptReact, Python, Go). + + +### Cody Enterprise - AWS Bedrock Instance + +#### Chat + +1. Login to the Cody Enterprise AWS Bedrock Instance +2. Create a new chat +3. User should not be able to change the LLM model on the chat window +4. Ask Cody a question in the new chat window. + +#### Commands + +1. Login to the Cody Enterprise AWS Bedrock Instance +2. Go to commands panel +3. Trigger command +4. Command should be executed + +#### Completions + +1. Login to the Cody Enterprise AWS Bedrock Instance +2. Generate a completion using the manual-trigger key binding on any primary languages (Javascript, Typescript, TypescriptReact, Python, Go). ## Other @@ -532,14 +560,3 @@ Repeat the above starting from different themes. #### Expected behaviour Changing theme should lead to full repaint of the colours according to the current theme. - - -### Claude 3 Enterprise Testing -#### Cody Gateway Instance -- [ ] Ask Cody a question in a new chat window. The question should include a request for Cody to generate code with the site-admin configuration is set to **Claude 3 Sonnet** for chatModel and **Claude 3 Haiku** for completionModel -- [ ] Ask Cody a question in a new chat window. The question should include a request for Cody to generate code with the site-admin configuration is set to **Claude 3 Opus** for chatModel and **Claude 3 Haiku** for completionModel -- [ ] Generate a completion using the manual-trigger key binding on any primary languages (Javascript, Typescript, TypescriptReact, Python, Go). - -#### AWS Bedrock Instance -- [ ] Ask Cody a question in a new chat window. The question should include a request for Cody to generate code with the site-admin configuration is set to **Claude 3 Sonnet** for chatModel and **Claude 3 Haiku** for completionModel -- [ ] Generate a completion using the manual-trigger key binding on any primary languages (Javascript, Typescript, TypescriptReact, Python, Go). From 4f9adb9145a345e28c8109c0736cb6e9c31f4dcb Mon Sep 17 00:00:00 2001 From: Aravind Ramaraju <51247042+aramaraju@users.noreply.github.com> Date: Sun, 21 Apr 2024 13:45:10 -0700 Subject: [PATCH 3/3] Update TESTING.md Co-authored-by: Dominic Cooney --- TESTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/TESTING.md b/TESTING.md index f2f08bf3da..6078ccbb57 100644 --- a/TESTING.md +++ b/TESTING.md @@ -484,7 +484,7 @@ To open the context menu: 1. Login to the Cody Enterprise Gateway Instance 2. Create a new chat 3. User should not be able to change the LLM model on the chat window -4. Ask Cody a question in the new chat window. +4. Ask Cody a question in the new chat window, Cody should respond. #### Commands