Support for Amazon Bedrock

[vkameswaran]

Is your feature request related to a problem? Please describe.
No current support for Amazon Bedrock

Describe the solution you'd like
Support added for Amazon Bedrock

Due to privacy concerns with OpenAI and it's sub products, I'd love to use this with Amazon Bedrock. Direct integration with the go SDK would be awesome but either way, love to see this added.

[greptile-apps-local]

To add support for Amazon Bedrock, you'll need to follow these steps, tailored to the existing structure of the greptileai/zep codebase:

1. Define Configuration Parameters: Extend the LLM struct in /config/models.go to include configuration parameters for Amazon Bedrock, such as API keys and endpoint URLs.

   type LLM struct {
       // Existing fields...
       BedrockAPIKey     string `mapstructure:"bedrock_api_key"`
       BedrockEndpoint   string `mapstructure:"bedrock_endpoint"`
   }

2. Implement the LLM Interface for Bedrock: Create a new file, llm_bedrock.go, in the /pkg/llms directory. Implement the ZepLLM interface for Amazon Bedrock, similar to how ZepOpenAILLM and ZepAnthropicLLM are implemented. This includes initializing the Bedrock client, handling authentication, and implementing methods for text processing and embeddings.

3. Add Bedrock to the LLM Factory: Modify llm_base.go to include a case for Amazon Bedrock in the LLM factory method. This ensures that the Bedrock LLM is correctly instantiated based on the configuration.

   switch {
   // Existing cases...
   case cfg.Service == "bedrock":
       return NewBedrockLLM(context.Background(), cfg)
   }

4. Unit Tests: Add unit tests in llm_bedrock_test.go to ensure your Bedrock integration works as expected. Mimic the structure and tests found in llm_openai_test.go.

By following these steps, you'll integrate Amazon Bedrock into the greptileai/zep codebase, allowing users to choose it as an LLM provider alongside OpenAI and Anthropic.

References

greptileai/zep/config/models.go
greptileai/zep/pkg/llms/llm_base.go
greptileai/zep/pkg/llms/llm_openai_test.go
greptileai/zep/pkg/llms

Ask Greptile

[greptile-apps]

To add support for Amazon Bedrock in the `greptileai/zep` repository, follow these steps:

1. **Update Configuration Model**:
   - Modify `/config/models.go` to include Amazon Bedrock specific settings. This involves adding a new struct for Bedrock configuration, similar to how `OpenAI` and `Anthropic` configurations are structured.

2. **Extend LLM Service Enumeration**:
   - In `/config/models.go`, extend the `LLM` struct to include a new case for Amazon Bedrock, ensuring it can be selected as an LLM service through the configuration.

3. **Implement Amazon Bedrock Client**:
   - Create a new file, perhaps `pkg/llms/bedrock_llm.go`, implementing the `models.ZepLLM` interface for Amazon Bedrock. This includes methods like `Call`, `EmbedTexts`, and `GetTokenCount`, tailored to Amazon Bedrock's API.

4. **Integrate Amazon Bedrock in LLM Client Initialization**:
   - In `/pkg/llms/llm_base.go`, extend the `NewLLMClient` function to include a case for initializing the Amazon Bedrock client when the service is selected in the configuration.

5. **Update Configuration File**:
   - Add Amazon Bedrock specific settings in `/config.yaml`, such as API keys, endpoints, and any other necessary configuration parameters.

6. **Documentation and Testing**:
   - Update the documentation to include instructions on how to configure and use Amazon Bedrock as an LLM service.
   - Implement unit tests for the Amazon Bedrock integration to ensure reliability and maintainability.

**Example Code Snippet for Step 3 (Implement Amazon Bedrock Client)**:
```go
package llms

import (
    "context"
    "github.com/getzep/zep/config"
    "github.com/getzep/zep/pkg/models"
    // Import Amazon Bedrock SDK
)

type BedrockLLM struct {
    // Bedrock client and other necessary fields
}

func NewBedrockLLM(ctx context.Context, cfg *config.Config) (models.ZepLLM, error) {
    // Initialize and return a new BedrockLLM client
}

func (b *BedrockLLM) Call(ctx context.Context, prompt string, options ...llms.CallOption) (string, error) {
    // Implement calling Amazon Bedrock with the provided prompt
}

func (b *BedrockLLM) EmbedTexts(ctx context.Context, texts []string) ([][]float32, error) {
    // Implement text embedding using Amazon Bedrock
}

func (b *BedrockLLM) GetTokenCount(text string) (int, error) {
    // Implement token count retrieval for Amazon Bedrock
}

func (b *BedrockLLM) Init(ctx context.Context, cfg *config.Config) error {
    // Initialize the Bedrock client with configuration
}

Note: Ensure to handle authentication and any specific API rate limiting or error handling as per Amazon Bedrock's documentation.