Support Bedrock , in particular, support bedrock claude models

[yangcheng]

Is your feature request related to a problem? Please describe.
We have an enterprise account on AWS and would like to use bedrock API

Describe the solution you'd like
I want to input aws access key / custom role so I can use bedrock

Describe alternatives you've considered
I build a proxy server to convert bedrock anthropic model to be compatible with antrhopic, but I still need a place to set base_url, and this add an extra layer of network latency

Additional context

[mmabrouk]

Hello @yangcheng,

We are planning to add a model zoo that enables the configuration of custom models like bedrock and azure. This should happen in the next six weeks.

[yangcheng]

thanks @mmabrouk , is there a way to setup custom base url so I can utlized more openai compatible services?

[mmabrouk]

Hi @yangcheng , yes that will be available in December. We are currently working on this issue

[JTXYH]

Hi @yangcheng , yes that will be available in December. We are currently working on this issue您好，是的，将于 12 月推出。我们目前正在研究这个问题

Looking forward to updates

[hgulersen]

Hi @mmabrouk - would like to understand if this is still in your roadmap and being prioritized?

[mmabrouk]

@hgulersen , yes it is our highest prio! The current roadmap looks like the following:

• Next release (next week), new playground running on a single service (the infra requirement to be able to add a model registry)
• The release after (in ±3 weeks), model registry with ability to add custom deployments (bedrock, azure)