Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enforce wheel size limits, README formatting in CI #17284

Merged
merged 13 commits into from
Nov 12, 2024

Conversation

jameslamb
Copy link
Member

@jameslamb jameslamb commented Nov 8, 2024

Description

Contributes to rapidsai/build-planning#110

Proposes adding 2 types of validation on wheels in CI, to ensure we continue to produce wheels that are suitable for PyPI.

Notes for Reviewers

How I tested this

Initially set the size threshold for libcudf to a value that I knew it'd violate (75MB compressed, when the wheels are 400+ MB compressed).

Saw CI fail as expected, and print a summary with the expected contents.

checking 'final_dist/libcudf_cu11-24.12.0a333-py3-none-manylinux_2_28_aarch64.whl'
----- package inspection summary -----
file size
  * compressed size: 0.4G
  * uncompressed size: 0.6G
  * compression space saving: 34.6%
contents
  * directories: 164
  * files: 1974 (2 compiled)
size by extension
  * .so - 0.6G (97.0%)
  * .h - 6.7M (1.0%)
  * no-extension - 4.8M (0.7%)
  * .cuh - 3.8M (0.6%)
  * .hpp - 2.2M (0.3%)
  * .a - 1.1M (0.2%)
  * .inl - 0.8M (0.1%)
  * .cmake - 0.1M (0.0%)
  * .md - 8.3K (0.0%)
  * .py - 4.0K (0.0%)
  * .pc - 0.2K (0.0%)
  * .txt - 34.0B (0.0%)
largest files
  * (0.6G) libcudf/lib64/libcudf.so
  * (3.3M) libcudf/bin/flatc
  * (1.0M) libcudf/lib64/libflatbuffers.a
  * (0.5M) libcudf/include/libcudf/rapids/libcudacxx/cuda/std/__atomic/functions/cuda_ptx_generated.h
  * (0.2M) libcudf_cu11-24.12.0a333.dist-info/RECORD
------------ check results -----------
1. [distro-too-large-compressed] Compressed size 0.4G is larger than the allowed size (75.0M).
errors found while checking: 1

(build link)

Updated that threshold in python/libcudf/pyproject.toml, and saw the build succeed (but the summary still printed).

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@jameslamb jameslamb added 2 - In Progress Currently a work in progress 5 - DO NOT MERGE Hold off on merging; see PR for details improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Nov 8, 2024
@github-actions github-actions bot added Python Affects Python cuDF API. cudf.polars Issues specific to cudf.polars pylibcudf Issues specific to the pylibcudf package labels Nov 8, 2024
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be implemented in https://github.com/rapidsai/gha-tools?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed offline -- maybe we can centralize this in the future. There may be some differences in how paths / current directories are handled across repositories. For now this is not a priority.

@jameslamb jameslamb changed the title WIP: [DO NOT MERGE] enforce wheel size limits, README formatting in CI enforce wheel size limits, README formatting in CI Nov 12, 2024
ci/validate_wheel.sh Outdated Show resolved Hide resolved
@jameslamb jameslamb marked this pull request as ready for review November 12, 2024 18:04
@jameslamb jameslamb requested review from a team as code owners November 12, 2024 18:04
@jameslamb jameslamb requested a review from bdice November 12, 2024 18:04
@jameslamb jameslamb removed 2 - In Progress Currently a work in progress 5 - DO NOT MERGE Hold off on merging; see PR for details labels Nov 12, 2024
@jameslamb
Copy link
Member Author

/merge

@rapids-bot rapids-bot bot merged commit 1f9ad2f into rapidsai:branch-24.12 Nov 12, 2024
105 checks passed
@jameslamb jameslamb deleted the wheel-validation branch November 12, 2024 22:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cudf.polars Issues specific to cudf.polars improvement Improvement / enhancement to an existing function non-breaking Non-breaking change pylibcudf Issues specific to the pylibcudf package Python Affects Python cuDF API.
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

2 participants