[s3] Add ability to set endpoint_url from AWS_ENDPOINT_URL env variable #773
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Title
[s3] Add ability to set endpoint_url from AWS_ENDPOINT_URL env variable
Motivation
The reason for this modification is that the addition of this functionality to the "smart-open" library addresses a longstanding PR in the "boto3" library. For years, there has been a lack of progress in implementing this feature directly in "boto3". However, "boto3" allows for the configuration of certain parameters, such as the aws_secret_access_key or aws_access_key_id, through environment variables. By extending the "smart-open" library to support the "AWS_ENDPOINT_URL" environment variable, users can leverage a more flexible approach to configuring the custom endpoint URL for S3 connections. This modification provides a workaround for the limitation in "boto3" and enables users to easily configure the S3 endpoint URL without relying solely on changes to the "boto3" library itself.
The addition of this functionality to the "smart-open" library is valuable as it allows for easy configuration of different S3 connections without the need to modify code paths, providing convenience and flexibility when working with multiple environments
Tests
Added test_smart_open::SmartOpenTest::test_s3_endpoint_from_env_var