Add Amazon Titan #2165
Add Amazon Titan #2165
Conversation
|
|
||
| if assumed_role: | ||
| sts = session.client("sts") | ||
| response = sts.assume_role(RoleArn=str(assumed_role), RoleSessionName="langchain-llm-1") |
There was a problem hiding this comment.
Why is this called langchain?
There was a problem hiding this comment.
The notebook that this code was sourced from (see the code comment for the URL) also had some LangChain tutorials, so I assume they just called it langchain because of that.
From the assume_role docs, it looks like this is just an arbitrary user-defined tag, so I'll set it to "crfm-helm" instead.
|
@yifanmai , do we want to merge this? |
|
@JosselinSomervilleRoberts this is ready for review, could you take a look? |
| # - https://aws.amazon.com/about-aws/whats-new/2023/11/amazon-titan-models-express-lite-bedrock/ | ||
| - name: amazon/titan-text-lite-v1 | ||
| display_name: Amazon Titan Text Lite | ||
| description: Amazon Titan Text Lite is a lightweight, efficient model perfect for fine-tuning English-language tasks like summarization and copywriting. It caters to customers seeking a smaller, cost-effective, and highly customizable model. It supports various formats, including text generation, code generation, rich text formatting, and orchestration (agents). Key model attributes encompass fine-tuning, text generation, code generation, and rich text formatting. |
There was a problem hiding this comment.
These descriptions seem a bit subjectives. I thought we were making descriptions about scientific content not so much use cases
There was a problem hiding this comment.
These are copy and pasted from the linked blog post. I wasn't able to find a model card or paper with a more suitable description.
There was a problem hiding this comment.
Shouldn't we have the optional args provided here ? Like bedrock_model_id ?
| raw_request = self.convert_request_to_raw_request(request) | ||
|
|
||
| # modelId isn't part of raw_request, so it must be explicitly passed into the input to | ||
| raw_request_for_cache: Dict = {"modelId": model_id, **deepcopy(raw_request)} |
There was a problem hiding this comment.
We usually don't use camel case for cache keys
There was a problem hiding this comment.
Using camel case will avoid having to do extra post-processing before sending to the server. Generally, we want the cache key to be as close to the actual parameters that we sent to the API as possible. See AI21Client for another example of a camel case request.
| "inputText": request.prompt, | ||
| "textGenerationConfig": { | ||
| "maxTokenCount": request.max_tokens, | ||
| # Sending a non-empty stopSequences results in an error: | ||
| # https://github.com/boto/boto3/issues/3993 | ||
| "stopSequences": request.stop_sequences or [], | ||
| "temperature": request.temperature, | ||
| "topP": request.top_p, |
There was a problem hiding this comment.
same here
yifanmai
force-pushed
the
yifanmai/fix-add-titan
branch
from
1cdb05c to
63b5dce
Compare
Also adds
BedrockClientwhich can be specialized to run other models on Amazon Bedrock.