Bedrock Claude 3.5 Sonnet v2 is not supporting new attachments (PDF)

[ac-shivamaggarwal]

Describe the bug

Recently Anthropic has released a new version of Claude 3.5 Sonnet v2 and it's also available via Amazon Bedrock. Breaking changes in it includes the support of PDF attachments (earlier it was limited to images). This is in beta state and can be enabled with anthropic-beta header (See docs). But when I use their model via Bedrock, it is giving me error.

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

PDF support should work in Anthropic's Claude 3.5 Sonnet v2 model with anthropic-beta header.

Current Behavior

Error:

Exception in thread "main" software.amazon.awssdk.services.bedrockruntime.model.ValidationException : messages.0.content.1: Input tag 'document' found using 'type' does not match any of the expected tags: 'text', 'image', 'tool_use', 'tool_result' (Service: BedrockRuntime, Status Code: 400, Request ID: something

Reproduction Steps

Python

import os
import json
from io import BytesIO
from base64 import b64encode
from dotenv import load_dotenv

import boto3
from botocore.config import Config
from anthropic import AnthropicBedrock

def query_claude_v2():
    load_dotenv()

    prompt = "extract this file"
    input_pdf = "/Users/shivam/Downloads/11-1-43-9-11.pdf"
    message_content = [{"type": "text", "text": prompt}]

    with open(input_pdf, "rb") as file:
        message_content.append(
            {
                "type": "document",
                "attrs": {
                    "format": "pdf",
                    "name": "12.pdf",
                    "source": {
                        "bytes": list(file.read())
                    }
                }
            }
        )
    
    client = boto3.Session().client(
        service_name="bedrock-runtime",
        region_name=os.environ["region"],
        aws_access_key_id=os.environ["AWS_ACCESS_KEY_ID"],
        aws_secret_access_key=os.environ["AWS_SECRET_ACCESS_KEY"],
        aws_session_token=os.environ["AWS_SESSION_TOKEN"],
        config=Config(read_timeout=600) # 10 minutes
    )

    def add_custom_headers(request, **kwargs):
        request.headers['anthropic-beta'] = 'pdfs-2024-09-25'
        print(request)

    client.meta.events.register('before-send.bedrock-runtime.InvokeModel', add_custom_headers)

    native_request = {
        "anthropic_version": "bedrock-2023-05-31",
        "max_tokens": 8192,
        "messages": [{"role": "user", "content": message_content}],
        "temperature": 0.2
    }
    request = json.dumps(native_request)
    response = client.invoke_model(
        modelId="anthropic.claude-3-5-sonnet-20241022-v2:0",
        body=request
    )
    response_body = json.loads(response["body"].read())
    response_text = response_body["content"][0]["text"]

    return response_text

def query_claude_v2_anthropic():
    load_dotenv()

    prompt = "extract this file"
    input_pdf = "/Users/shivam/Downloads/11-1-43-9-11.pdf"
    with open(input_pdf, "rb") as f:
        pdf_stream = BytesIO(f.read())
    
    bedrock_client = AnthropicBedrock(
        aws_access_key=os.environ["AWS_ACCESS_KEY_ID"],
        aws_secret_key=os.environ["AWS_SECRET_ACCESS_KEY"],
        aws_session_token=os.environ["AWS_SESSION_TOKEN"],
        aws_region=os.environ["region"],
        timeout=600
    )
    response = bedrock_client.beta.messages.create(
        max_tokens=8192,
        messages=[{
            "role": "user", "content": [
                {"type": "text", "text": prompt},
                {"type": "document", "source": {
                    "type": "base64",
                    "media_type": "application/pdf",
                    "data": b64encode(pdf_stream.getvalue()).decode("utf-8")
                }}
            ]
        }],
        model="anthropic.claude-3-5-sonnet-20241022-v2:0",
        temperature=0.2,
        betas=['pdfs-2024-09-25']
    )

    return response.content[0].text


# print(query_claude_v2())
print(query_claude_v2_anthropic())

Java

package org.example;

// Use the native inference API to send a text message to Anthropic Claude.

import org.json.JSONArray;
import org.json.JSONObject;
import org.json.JSONPointer;
import software.amazon.awssdk.auth.credentials.AwsSessionCredentials;
import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
import software.amazon.awssdk.core.SdkBytes;
import software.amazon.awssdk.core.exception.SdkClientException;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.bedrockruntime.BedrockRuntimeClient;

import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.util.Base64;

public class InvokeModel {

    String access_key = "something";
    String secret_key = "something";
    String session_token = "something";

    public String invokeModel(String pdfPath) throws IOException {

        // Create a Bedrock Runtime client in the AWS Region you want to use.
        // Replace the DefaultCredentialsProvider with your preferred credentials provider.
        AwsSessionCredentials awsSessionCredentials = AwsSessionCredentials.create(
                access_key, secret_key, session_token
        );
        BedrockRuntimeClient client = BedrockRuntimeClient.builder()
                .credentialsProvider(StaticCredentialsProvider.create(awsSessionCredentials))
                .region(Region.US_WEST_2)
                .build();

        // Set the model ID, e.g., Claude 3 Haiku.
        String modelId = "anthropic.claude-3-5-sonnet-20241022-v2:0";

        // The InvokeModel API uses the model's native payload.
        // Learn more about the available inference parameters and response fields at:
        // https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-anthropic-claude-messages.html
        String nativeRequest = new JSONObject()
                .put("anthropic_version", "bedrock-2023-05-31")
                .put("max_tokens", 512)
                .put("temperature", 0.5)
                .put("messages", new JSONArray()
                        .put(new JSONObject()
                                .put("role", "user")
                                .put("content", new JSONArray()
                                        .put(new JSONObject()
                                                .put("type", "text")
                                                .put("text", "Describe the attached PDF")
                                        )
                                        .put(new JSONObject()
                                                .put("type", "document")
                                                .put("source", new JSONObject()
                                                        .put("type", "base64")
                                                        .put("media_type", "application/pdf")
                                                        .put("data", encodeFileToBase64(pdfPath))
                                                )
                                        )
                                )
                        )
                ).toString();

        try {
            // Encode and send the request to the Bedrock Runtime.
            var response = client.invokeModel(request -> request
                    .body(SdkBytes.fromUtf8String(nativeRequest))
                    .overrideConfiguration(cfg -> cfg.putHeader("anthropic-beta", "pdfs-2024-09-25"))
                    .modelId(modelId)
            );

            // Decode the response body.
            var responseBody = new JSONObject(response.body().asUtf8String());

            // Retrieve the generated text from the model's response.
            var text = new JSONPointer("/content/0/text").queryFrom(responseBody).toString();

            return text;

        } catch (SdkClientException e) {
            System.err.printf("ERROR: Can't invoke '%s'. Reason: %s", modelId, e.getMessage());
            throw new RuntimeException(e);
        }
    }

    private String encodeFileToBase64(String filePath) throws IOException {
        File file = new File(filePath);
        FileInputStream fileInputStream = new FileInputStream(file);
        byte[] bytes = new byte[(int) file.length()];
        fileInputStream.read(bytes);
        fileInputStream.close();

        // Convert the byte array to a Base64 string
        return Base64.getEncoder().encodeToString(bytes);
    }
}

Possible Solution

I have tried using both Python and Java SDK so it seems to be not a problem in SDK, it's more related to the internal implementation. I have also raised an issue on Anthropic's repository. Either the AWS is not passing my anthropic-beta header to Anthropic or is running this content type validation on its own.

I don't know where to report this internal issue so I'm reporting it here. Please forward my request to the internal team.

Additional Information/Context

No response

SDK version used

• Python: boto3 1.34.151
• Java: 2.29.6

Environment details (OS name and version, etc.)

macOS 15.0.1

[tim-finnigan]

Thanks for reaching out. This use case is currently not supported. The Bedrock team would need to add support in order to enable AWS SDKs like Boto3 and the Java SDK to support this, as they maintain the underlying APIs like InvokeModel.

I searched internally and found that the team is already tracking this feature request in their backlog. We cannot guarantee if or when a service would implement a new feature like this, but I recommend referring to the blog and CHANGELOG for updates going forward. I will close this as not planned on the Boto3 side because the Bedrock team would need to implement this change.

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.

[ac-shivamaggarwal]

Just for clarification, AWS does not support the beta features of Anthropic? Will this start working when Anthropic make this feature non-beta?

[diegocatao]

I’m excited for Bedrock to add PDF file support for Anthropic models.

[JasonB9527]

Please upgrade sdk to latest.

[pawelwasylyszyn]

Hi,

I just upgraded my boto3 to boto3-1.35.59 but I still see the same issue, PDF not being supported. Using the same code as an example I am getting:

anthropic.BadRequestError: Error code: 400 - {'message': "messages.0.content.1: Input tag 'document' found using 'type' does not match any of the expected tags: 'text', 'image', 'tool_use', 'tool_result'"}

Is there any planned date to update the boto3 library as well? I cannot see this change in the Changelog.

Many thanks!

[ashokrs]

Please provide an update when this issue is addressed.