Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update datasets download URL #12840

Merged
merged 1 commit into from
Feb 24, 2023
Merged

Update datasets download URL #12840

merged 1 commit into from
Feb 24, 2023

Conversation

jjacobelli
Copy link
Contributor

Description

Update datasets download URL to reduce latency and costs

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@jjacobelli jjacobelli added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Feb 24, 2023
@jjacobelli jjacobelli requested a review from a team as a code owner February 24, 2023 09:51
@github-actions github-actions bot added the Python Affects Python cuDF API. label Feb 24, 2023
@jjacobelli
Update datasets download URL
e3cc92b 
Signed-off-by: Jordan Jacobelli <[email protected]>
@ajschmidt8
Copy link
Member

To provide additional context...

data.rapids.ai serves the contents of the rapidsai-data S3 bucket via an AWS CloudFront distribution.

The benefits of using a CloudFront distribution are:

  • dataset files are served from an endpoint closest to individual end-users. Therefore latency is decreased, which makes datasets download faster
  • data transfer costs out of AWS are cheaper

Therefore, it's in everyone's best interest to start using the new data.rapids.ai URLs for downloading datasets.

At some point in the future, the S3 URLs will be disabled and datasets will only be retrievable from data.rapids.ai.

@ajschmidt8
Copy link
Member

/merge

@rapids-bot rapids-bot bot merged commit 54ee14e into rapidsai:branch-23.04 Feb 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@galipremsagar galipremsagar galipremsagar approved these changes

@ajschmidt8 ajschmidt8 ajschmidt8 approved these changes

@isVoid isVoid Awaiting requested review from isVoid isVoid is a code owner automatically assigned from rapidsai/cudf-python-codeowners

Assignees
No one assigned
Labels
improvement Improvement / enhancement to an existing function non-breaking Non-breaking change Python Affects Python cuDF API.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@jjacobelli @ajschmidt8 @galipremsagar