Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable wheels CI scripts to run locally #57

Merged
merged 11 commits into from
Jul 19, 2023

Conversation

divyegala
Copy link
Member

@divyegala divyegala commented May 26, 2023

Taking inspiration from #35 and #62

This PR adds / updates:

  • rapids-configure-sccache (new) : configures sccache for CI/local, with the intention being to remove this information from CI containers
  • rapids-date-string (new) : configures date string
  • rapids-configure-conda-channels (new) : modifies conda channels based on build type
  • rapids-env-update (modified) : calls the rapids-configure-sccache
  • rapids-prompt-local-repo-config (new) : consolidates prompts for users to configure repo information locally
  • rapids-download-wheels/conda-from-s3 (updated) : uses rapids-prompt-local-repo-config
  • rapids-upload-wheels-to-s3 (modifed) : guards to ensure local builds don't try to upload

@vyasr
Copy link
Contributor

vyasr commented Jul 13, 2023

Is there a way to avoid duplicating this logic from the conda builds? Maybe putting it into a separate tool? I agree that we don't want to (nor can we really) lump the logic into rapids-package-name, but I also would like to share conda/wheel logic on this.

@vyasr
Copy link
Contributor

vyasr commented Jul 13, 2023

Also you may want to make sure that whatever you do supports #62 (another reason to avoid duplicating if possible).

@divyegala
Copy link
Member Author

I already copied over #62 here.

I'll think on how to minimize duplication, maybe we can combine the prompts as part of another tool.

@vyasr
Copy link
Contributor

vyasr commented Jul 13, 2023

Yeah another tool seems like the lowest hanging fruit.

Copy link
Member

@ajschmidt8 ajschmidt8 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good. one minor request.

Can you remove the else statements and replace them with early exit 0 statements?

That's the convention that I've followed for the scripts in this repository and I'd like these changes to be consistent.

@divyegala
Copy link
Member Author

@ajschmidt8 as we move towards local builds, @vyasr and I discussed that some steps may be entirely skippable locally and the build would still be able to succeed. Uploading to S3 is one such step, and I don't think we should exit early.

tools/rapids-configure-sccache Show resolved Hide resolved
tools/rapids-configure-sccache Outdated Show resolved Hide resolved
tools/rapids-prompt-local-repo-config Show resolved Hide resolved
tools/rapids-configure-conda-channels Outdated Show resolved Hide resolved
tools/rapids-configure-sccache Show resolved Hide resolved
tools/rapids-date-string Outdated Show resolved Hide resolved
Copy link
Contributor

@vyasr vyasr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything looks good to me now, pending resolution of AJ's open question regarding RAPIDS_BUILD_TYPE and assuming the gh CLI installation is all good.

@ajschmidt8 ajschmidt8 merged commit 71f9e9a into rapidsai:main Jul 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants