[Backport 2.10] Feature/conversation backport to 2.x #1297
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![opensearch-trigger-bot[bot]](https://avatars.githubusercontent.com/u/80134844?s=60&v=4){kind=small}
* Conversational Memory for GenAI Apps (#1196) * moved code over Signed-off-by: HenryL27 <[email protected]> * added actions to MLPlugin; fixed io lib stuff Signed-off-by: HenryL27 <[email protected]> * fixed copyrights again Signed-off-by: HenryL27 <[email protected]> * Fix nullptr exception in .equals Signed-off-by: HenryL27 <[email protected]> * preserve thread context across action calls Signed-off-by: HenryL27 <[email protected]> * remove MissingResourceException from CreatInteractionRequest in favor of IOException Signed-off-by: HenryL27 <[email protected]> * move ConversationMet, Interaction, and Constants to common/conversational Signed-off-by: HenryL27 <[email protected]> * Sequentialize createInteraction to remove data race Signed-off-by: HenryL27 <[email protected]> * allow disorder when conversations have same timestamp Signed-off-by: HenryL27 <[email protected]> * lombokify Signed-off-by: HenryL27 <[email protected]> * add some unit testing Signed-off-by: HenryL27 <[email protected]> * Increase unit test coverage Signed-off-by: HenryL27 <[email protected]> * fix naming Signed-off-by: HenryL27 <[email protected]> * finish code coverage for actions Signed-off-by: HenryL27 <[email protected]> * Leave null values out of XContent per #1196 (comment) Signed-off-by: HenryL27 <[email protected]> * Add integration tests for rest actions Signed-off-by: HenryL27 <[email protected]> * apply spotless Signed-off-by: HenryL27 <[email protected]> * Complete unit testing for Index classes Signed-off-by: HenryL27 <[email protected]> * update build.gradle Signed-off-by: HenryL27 <[email protected]> * Finish unit tests Signed-off-by: HenryL27 <[email protected]> * Fail closed on missing convo access Signed-off-by: HenryL27 <[email protected]> * address code review/walkthrough comments Signed-off-by: HenryL27 <[email protected]> * re-add prompt temlplate and metadata fields at interaction level Signed-off-by: HenryL27 <[email protected]> * parse request body, not params, for post requests Signed-off-by: HenryL27 <[email protected]> * restructure with memory as higher-level term Signed-off-by: HenryL27 <[email protected]> * clean up build.gradle Signed-off-by: HenryL27 <[email protected]> * apply spotless Signed-off-by: HenryL27 <[email protected]> * change interaction field names timestamp -> create_time metadata -> additional_info Signed-off-by: HenryL27 <[email protected]> * fix GetInteractionsResponse xcontent tests Signed-off-by: HenryL27 <[email protected]> * propagate name change to variables and parameters Signed-off-by: HenryL27 <[email protected]> * clean logging and fix typos Signed-off-by: HenryL27 <[email protected]> * fix final convtructor according to find-and-replace Signed-off-by: HenryL27 <[email protected]> * append plugin-ml- to index names Signed-off-by: HenryL27 <[email protected]> --------- Signed-off-by: HenryL27 <[email protected]> * Feature/conversation memory feature flag (#1271) * add feature flag and checks to transport actions Signed-off-by: HenryL27 <[email protected]> * add feature flag tests Signed-off-by: HenryL27 <[email protected]> * fix typos for real with find-and-replace Signed-off-by: HenryL27 <[email protected]> * rename conversational-memory directory to memory Signed-off-by: HenryL27 <[email protected]> * fix settings.gradle with new dir name Signed-off-by: HenryL27 <[email protected]> * re-add feature flag checks and tests to transport layer Signed-off-by: HenryL27 <[email protected]> * fix feature flag with updateConsumer Signed-off-by: HenryL27 <[email protected]> * remove redundant settings update Signed-off-by: HenryL27 <[email protected]> * clean up feature var initialization to avoid unchecked conversion warning Signed-off-by: HenryL27 <[email protected]> --------- Signed-off-by: HenryL27 <[email protected]> * Use Search Pipeline processors, Remote Inference and HttpConnector to enable Retrieval Augmented Generation (RAG) (#1195) * Use Search Pipeline processors, Remote Inference and HttpConnector to enable Retrieval Augmented Generation (RAG) (#1150) Signed-off-by: Austin Lee <[email protected]> * Address test coverage. Signed-off-by: Austin Lee <[email protected]> * Fix/update imports due to changes coming from core. Signed-off-by: Austin Lee <[email protected]> * Update license header. Signed-off-by: Austin Lee <[email protected]> * Address comments. Signed-off-by: Austin Lee <[email protected]> * Use List for context fields so we can pull contexts from multiple fields when constructing contexts for LLMs. Signed-off-by: Austin Lee <[email protected]> * Address review comments. Signed-off-by: Austin Lee <[email protected]> * Fix spotless issue. Signed-off-by: Austin Lee <[email protected]> * Update README. Signed-off-by: Austin Lee <[email protected]> * Fix ml-client shadowJar implicit dependency issue. Signed-off-by: Austin Lee <[email protected]> * Add a wrapper client for ML predict. Signed-off-by: Austin Lee <[email protected]> * Add tests for the internal ML client. Signed-off-by: Austin Lee <[email protected]> --------- Signed-off-by: Austin Lee <[email protected]> Signed-off-by: Austin Lee <[email protected]> Signed-off-by: HenryL27 <[email protected]> * [Feature] Add Retrieval Augmented Generation search processors (#1275) * Put RAG pipeline behind a feature flag. Signed-off-by: Austin Lee <[email protected]> * Add support for chat history in RAG using the Conversational Memory API Signed-off-by: Austin Lee <[email protected]> * Fix spotless Signed-off-by: Austin Lee <[email protected]> * Fix RAG feature flag enablement. Signed-off-by: Austin Lee <[email protected]> * Address review comments and suggestions. Signed-off-by: Austin Lee <[email protected]> * Address comments. Signed-off-by: Austin Lee <[email protected]> * Add unit tests for MachineLearningPlugin Signed-off-by: Austin Lee <[email protected]> --------- Signed-off-by: Austin Lee <[email protected]> Signed-off-by: HenryL27 <[email protected]> * Allow RAG pipeline feature flag to be enabled and disabled dynamically (#1293) * Allow RAG pipeline feature flag to be enabled and disabled dynamically. Signed-off-by: Austin Lee <[email protected]> * Address review comments. Signed-off-by: Austin Lee <[email protected]> * Add negative test cases for RAG feature flag being turned off. Signed-off-by: Austin Lee <[email protected]> * Improve error checking. Signed-off-by: Austin Lee <[email protected]> --------- Signed-off-by: Austin Lee <[email protected]> Signed-off-by: HenryL27 <[email protected]> * apply spotless Signed-off-by: HenryL27 <[email protected]> --------- Signed-off-by: HenryL27 <[email protected]> Signed-off-by: Austin Lee <[email protected]> Signed-off-by: Austin Lee <[email protected]> Co-authored-by: Austin Lee <[email protected]> (cherry picked from commit 1112612)
opensearch-trigger-bot
bot
had a problem deploying
to
ml-commons-cicd-env
opensearch-trigger-bot
bot
had a problem deploying
to
ml-commons-cicd-env
opensearch-trigger-bot
bot
temporarily deployed
to
ml-commons-cicd-env
opensearch-trigger-bot
bot
temporarily deployed
to
ml-commons-cicd-env
opensearch-trigger-bot
bot
temporarily deployed
to
ml-commons-cicd-env
opensearch-trigger-bot
bot
had a problem deploying
to
ml-commons-cicd-env
opensearch-trigger-bot
bot
temporarily deployed
to
ml-commons-cicd-env
opensearch-trigger-bot
bot
temporarily deployed
to
ml-commons-cicd-env
Codecov Report
@@ Coverage Diff @@
## 2.10 #1297 +/- ##
============================================
+ Coverage 78.86% 79.34% +0.47%
- Complexity 2142 2246 +104
============================================
Files 168 188 +20
Lines 8741 9074 +333
Branches 877 903 +26
============================================
+ Hits 6894 7200 +306
- Misses 1446 1465 +19
- Partials 401 409 +8
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 2 files with indirect coverage changes 📢 Have feedback on the report? Share it here. |
opensearch-trigger-bot
bot
temporarily deployed
to
ml-commons-cicd-env
opensearch-trigger-bot
bot
temporarily deployed
to
ml-commons-cicd-env
opensearch-trigger-bot
bot
temporarily deployed
to
ml-commons-cicd-env
opensearch-trigger-bot
bot
temporarily deployed
to
ml-commons-cicd-env
dhrubo-os
approved these changes
Sep 7, 2023
jngz-es
approved these changes
Sep 7, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport 1112612 from #1286