OpenSSL-fips cause a failure

[mq067]

Confirm by changing [ ] to [x] below to ensure that it's a bug:

• [x ] I've gone though Developer Guide for v3 and API reference
• [x ] I've checked AWS Forums and StackOverflow for answers
• [x ] I've searched for previous similar issues and didn't find any solution

Describe the bug
On FIPS complaint systems or with OpenSSL-fips the gem failed badly. All OpenSSL::Digest::MD5.new are blocked by openssl lib. and this gem is using it a few times.

Gem name ('aws-sdk', 'aws-sdk-resources' or service gems like 'aws-sdk-s3') and its version
aws-sdk-s3-1.75.0

Version of Ruby, OS environment
ruby -version
ruby 2.7.1p83 (2020-03-31 revision a0c7c23c9c) [x86_64-linux]

To Reproduce (observed behavior)
When using the openssl-fips (OpenSSL 1.0.2k-fips 26 Jan 2017 - in my case) gem produce an error.

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots

Additional context
OS - RHEL 7.6

[alextwoods]

Could you provide a stack trace? What operations are you trying to use? Many S3 control plane operations require content-md5. To make these apis work we would need S3 to remove the requirement for the content-md5 header for all operations. See: boto/botocore#1700

[mq067]

The screenshot contains the error from the td-agent.log file.
The td-agent config is trivial, source type is set to tail and match is set to s3. Error appears when td-agent starts, so when any attempt to communicate/send sth to s3 bucket take place.
Provided link is accurate boto/botocore#1700. Any call-outs of MD5 from openssl-fips would fail.
I would be happy to provide stack trace, just need a tip how you want to collect it.

[alextwoods]

I'm not sure how you've wrapped s3 calls or how your logging it setup - but you may be able to change the log level (eg to debug) or some other way - what I want to confirm is which S3 calls are failing.

[mq067]

@alextwoods - log level changed to trace
Full log file attached (anonymized)
2020-09-02 16:23:13 -0500 [trace]: #0 dequeueing a chunk instance=2800 2020-09-02 16:23:13 -0500 [trace]: #0 chunk dequeued instance=2800 metadata=#<struct Fluent::Plugin::Buffer::Metadata timekey=1596183000, tag="astro", variables=nil, seq=0> 2020-09-02 16:23:13 -0500 [trace]: #0 trying flush for a chunk chunk="5ad570a90862a111f937fbbe518a1bd3" 2020-09-02 16:23:13 -0500 [trace]: #0 adding write count instance=2820 2020-09-02 16:23:13 -0500 [trace]: #0 executing sync write chunk="5ad570a90862a111f937fbbe518a1bd3" 2020-09-02 16:23:13 -0500 [debug]: #0 out_s3: write chunk 5ad570a90862a111f937fbbe518a1bd3 with metadata #<struct Fluent::Plugin::Buffer::Metadata timekey=1596183000, tag="xxxxxx", variables=nil, seq=0> to s3://xxxxxxxxxx 2020-09-02 16:23:13 -0500 [debug]: #0 taking back chunk for errors. chunk="5ad570a90862a111f937fbbe518a1bd3" 2020-09-02 16:23:13 -0500 [trace]: #0 taking back a chunk instance=2800 chunk_id="5ad570a90862a111f937fbbe518a1bd3" 2020-09-02 16:23:13 -0500 [trace]: #0 chunk taken back instance=2800 chunk_id="5ad570a90862a111f937fbbe518a1bd3" metadata=#<struct Fluent::Plugin::Buffer::Metadata timekey=1596183000, tag="astro", variables=nil, seq=0> 2020-09-02 16:23:13 -0500 [warn]: #0 failed to flush the buffer. retry_time=1 next_retry_seconds=2020-09-02 16:23:14 7601761559524660660137/17592186044416000000000 -0500 chunk="5ad570a90862a111f937fbbe518a1bd3" error_class=OpenSSL::Digest::DigestError error="Digest initialization failed: disabled for fips" 2020-09-02 16:23:13 -0500 [warn]: #0 suppressed same stacktrace

s3_trace_system_ano.txt

[alextwoods]

Closing this out since content-md5 is required on the APIs and not allowed through openssl FIPS. If this is still an issue for you, consider creating an issue in the cross-sdk repo: https://github.com/aws/aws-sdk

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.