New features: inline broker request-response, temporary queues, improved docs and examples and better testing

[efunneko]

The following comprehensive updates have been made to the Solace AI Event Connector repository, which includes code refactoring, enhancements, and expanded documentation.

Documentation

1. README.md:

   • Moved the "Getting Started Quickly" section after the Overview.

2. Advanced Component Features:

   • Added to describe advanced features available for custom components, such as Broker Request-Response, Cache Manager, and Timer Features.

3. Component Documentation:

   • Detailed documentation has been added for aggregate, assembly, broker_input, error_input, and other components. This includes configuration details, input/output schema, and example configurations.

4. Usage and Configuration:

   • Removed the previous usage.md.
   • Extended the main configuration file structure, specifying log/trace configurations, shared configurations, and detailed expression syntax.
   • invoke_keyword: Detailed invocation configurations using custom functions or invoke_functions.

5. Custom Components:

   • Added custom_components.md discussing how to extend the functionality by creating custom components, providing required structure, methods, and advanced features.

6. Guides:

   • Added a comprehensive guide on RAG (Retrieval-Augmented Generation), LLM models, and embeddings, leveraging OpenAI with ChromaDB via the Solace AI Connector.

Features and Enhancements

1. Broker Request-Response:

   • Implemented in broker_request_response.py to handle both simple and streaming responses. Configurations are detailed in the documentation.

2. Components Enhancements:

   • aggregate.py: Extended to include new configuration parameters, better handling and documentation.
   • assembly.py: New component added for assembling messages with specified configurations.
   • error_input.py and broker_input.py: Refactored and aligned with new configurations (e.g., input_selection instead of component_input).

3. Transforms Enhancements:

   • Refined structured handling of evaluate_expression keyword with better documentation.
   • General refactor for better clarity and reusability, ensuring all transform functions are working as expected.

4. Extensibility via Custom Components:

   • Provided detailed guidelines and examples on how to create scheduleable timer events, handle cache expiry, and perform request-response operations.

Examples

1. New and Updated Examples:
   • Added examples to illustrate setups for OpenAI configurations, Request-Response flows, and more.
   • Examples directory (examples/llm) updated with detailed flow configurations using different AI/ML models and vector databases.

Testing

1. Test Utilities:
   • Enhanced utils_for_test_files.py for better test handling with importing and message validation across flows.
   • Introduced new tests for request-response functionalities, aggregate, and iterate components.

Code Refactoring

1. Imports and Path Handling:

   • Refactored import statements and path handling for modularity and clarity.
   • Enhanced error handling and logging throughout the project.

2. Modularization and Clean Code:

   • Split and clean components and transform files for better maintainability.
   • Removed redundant storage implementations and debugging remnants.