s3.deleteObjects sends CRC32 checksum even if requestChecksumCalculation is set to WHEN_REQUIRED

[trivikr]

Checkboxes for prior research

• I've gone through Developer Guide and API reference
• I've checked AWS Forums and StackOverflow.
• I've searched for previous similar issues and didn't find any solution.

Describe the bug

s3.deleteObjects sends CRC32 checksum even if requestChecksumCalculation is set to WHEN_REQUIRED

Regression Issue

• Select this option if this issue appears to be a regression.

SDK version number

@aws-sdk/[email protected]

Which JavaScript Runtime is this issue in?

Node.js

Details of the browser/Node.js/ReactNative version

All

Reproduction Steps

import { S3 } from "@aws-sdk/client-s3";
import { NodeHttpHandler } from "@smithy/node-http-handler";

class CustomHandler extends NodeHttpHandler {
  constructor() {
    super();
  }

  printChecksumHeaders(prefix, headers) {
    for (const [header, value] of Object.entries(headers)) {
      if (
        header === "content-md5" ||
        header.startsWith("x-amz-checksum-") ||
        header.startsWith("x-amz-sdk-checksum-")
      ) {
        console.log(`${prefix}['${header}']: '${value}'`);
      }
    }
  }

  async handle(request, options) {
    this.printChecksumHeaders("request", request.headers);
    const response = await super.handle(request, options);
    this.printChecksumHeaders("response", response.response.headers);
    return response;
  }
}

const client = new S3({
  requestHandler: new CustomHandler(),
  requestChecksumCalculation: "WHEN_REQUIRED",
});

const Bucket = "test-flexible-checksums-v2"; // Update to your test bucket name.

const prepareClient = new S3();
// Populate an object in the bucket, as deleteObjects requires at least one.
await prepareClient.putObject({ Bucket, Key: "helloworld.txt", Body: "helloworld" });

// Get list of objects to delete.
const response = await prepareClient.listObjectsV2({ Bucket });
const Objects = response.Contents.map(({ Key }) => ({ Key }));

await client.deleteObjects({ Bucket, Delete: { Objects } });

Observed Behavior

The CRC32 checksum is sent even if requestChecksumCalculation is set to WHEN_REQUIRED

$ node test.mjs
request['x-amz-sdk-checksum-algorithm']: 'CRC32'
request['x-amz-checksum-crc32']: 'uDrkBQ=='

Expected Behavior

When requestChecksumCalculation is set to WHEN_REQUIRED, should it send md5 like it was done in <=v3.729.0?

Example output in v3.726.0

$ node test.mjs
request['content-md5']: 'rl/UEqX25ygZIHP8TWK6/g=='

Possible Solution

No response

Additional Information/Context

No response

[trivikr]

This is not a regression, as default algorithm was changed to CRC32 as per discussions with S3 team in #6749

Adding a guidance label.

[trivikr]

What's the issue that you're facing @OIRNOIR?

I verified through the minimal repro provided above that deleteObjects call is succesful.
Is it unsuccessful in your case? If yes, can you provided a minimal repro?

[OIRNOIR]

What's the issue that you're facing @OIRNOIR?

I verified through the minimal repro provided above that deleteObjects call is succesful. Is it unsuccessful in your case? If yes, can you provided a minimal repro?

I'm using Cloudflare R2, and I can reproduce the issue this way:

import fs from "node:fs";
import path from "node:path";
import {
	S3Client,
	PutObjectCommand,
	ListObjectsV2Command,
	DeleteObjectsCommand
} from "@aws-sdk/client-s3";

const __dirname = path.dirname(decodeURIComponent(new URL(import.meta.url).pathname));

const config = JSON.parse(
	fs.readFileSync(path.join(__dirname, "/config.json")),
);

const S3 = new S3Client({
	region: "auto",
	endpoint: `https://${config.r2.accountId}.r2.cloudflarestorage.com`,
	credentials: {
		accessKeyId: config.r2.keyId,
		secretAccessKey: config.r2.secret,
	},
	requestChecksumCalculation: "WHEN_REQUIRED",
	responseChecksumValidation: "WHEN_REQUIRED"
});

// Put dummy object
await S3.send(
	new PutObjectCommand({
		Bucket: "testing",
		Key: `test-${new Date().toISOString()}`,
		Body: "helloworld"
	})
);

const res = await S3.send(
	new ListObjectsV2Command({
		Bucket: "testing"
	})
);
const Objects = res.Contents.map(({Key}) => ({Key}));

console.log("Sending the DeleteObjectsCommand"); // Crashes after this line
await S3.send(
	new DeleteObjectsCommand({
		Bucket: "testing",
		Delete: {
			Objects
		}
	})
);

[OIRNOIR]

It crashes with a 501 error: NotImplemented: Header 'x-amz-sdk-checksum-algorithm' with value 'CRC32' not implemented

[OIRNOIR]

This is not a regression, as default algorithm was changed to CRC32 as per discussions with S3 team in #6749

Adding a guidance label.

It is a regression because not all third party S3 apis (like Cloudflare R2) support CRC32 and return 501 whenever CRC32 (or any other checksum algorithm) is attempted. Implementing your CustomHandler, the observed code logs the same two headers, even with requestChecksumCalculation set to WHEN_REQUIRED. However, upon downgrading to @aws-sdk/[email protected], the only header listed is request['content-md5']: 'kbedJZRuTVhyWMNpO7amyA==' and the request succeeds.

My guess as to why you can't reproduce the error is that you are not using Cloudflare R2.

[trivikr]

Closing as AWS SDK for JavaScript is designed to be used with AWS services.
We've provided a minimal repro where deleteObjects call succeeds with Amazon S3.

If you're using a different cloud service, please refer to their documentation on compatibility.

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.