add setting s3 endpoint-url via env var

[harelwa]

Why are these changes needed?

Support providing custom S3 endpoint URL via environment variable.

This is needed for when ray is used with S3-like storage (MinIO, VAST, etc.)

I have added support for two standard env var names:

• AWS_ENDPOINT_URL
• S3_ENDPOINT_URL

I will update the docs accordingly once this is approved

Related issue number

Closes #30601

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[harelwa]

hi,

can you pls merge this?

thanks,

harel

[stale]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

• If you'd like to keep this open, just leave any comment, and the stale label will be removed.

[harelwa]

hi,

can you pls merge this?

thanks,

harel

[stale]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

• If you'd like to keep this open, just leave any comment, and the stale label will be removed.

[architkulkarni]

Hi @harelwa, thanks for the contribution and sorry for the delay in getting this reviewed. According to this boto/boto3#2099 (comment), am I understanding correctly that this functionality is now included in boto3 itself?

[stale]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

• If you'd like to keep this open, just leave any comment, and the stale label will be removed.

[chris-aeviator]

would love to see this contribution merged since minio is a critical part of on-prem infrastructure for many today