Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow users to set jitify cache file limit via an environment variable #8132

Merged

Conversation

trxcllnt
Copy link
Contributor

@trxcllnt trxcllnt commented May 1, 2021

Jitify limits the file count of the kernel cache, defaulting to the number of JIT kernels to keep in memory.

Limiting to 100 files on disk leads to thrashing if running workloads that exercise all the binops (like unit tests).

This PR adds a LIBCUDF_KERNEL_CACHE_FILE_LIMIT environment variable so users can control the number of files Jitify should keep on disk.

@trxcllnt trxcllnt requested a review from a team as a code owner May 1, 2021 06:49
@trxcllnt trxcllnt requested review from devavret and davidwendt May 1, 2021 06:49
@github-actions github-actions bot added the libcudf Affects libcudf (C++/CUDA) code. label May 1, 2021
@trxcllnt trxcllnt requested a review from cwharris May 1, 2021 06:50
@trxcllnt trxcllnt added 3 - Ready for Review Ready for review by team improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels May 1, 2021
@codecov
Copy link

codecov bot commented May 1, 2021

Codecov Report

Merging #8132 (f1d46ae) into branch-0.20 (51336df) will increase coverage by 0.03%.
The diff coverage is n/a.

Impacted file tree graph

@@               Coverage Diff               @@
##           branch-0.20    #8132      +/-   ##
===============================================
+ Coverage        82.88%   82.92%   +0.03%     
===============================================
  Files              103      104       +1     
  Lines            17668    17896     +228     
===============================================
+ Hits             14645    14840     +195     
- Misses            3023     3056      +33     
Impacted Files Coverage Δ
python/cudf/cudf/_fuzz_testing/parquet.py 0.00% <ø> (ø)
python/cudf/cudf/_fuzz_testing/utils.py 0.00% <ø> (ø)
python/cudf/cudf/core/abc.py 91.66% <ø> (+0.17%) ⬆️
python/cudf/cudf/core/algorithms.py 82.35% <ø> (ø)
python/cudf/cudf/core/column/__init__.py 100.00% <ø> (ø)
python/cudf/cudf/core/column/categorical.py 92.37% <ø> (+0.13%) ⬆️
python/cudf/cudf/core/column/column.py 88.20% <ø> (-0.44%) ⬇️
python/cudf/cudf/core/column/datetime.py 88.03% <ø> (-1.88%) ⬇️
python/cudf/cudf/core/column/decimal.py 91.04% <ø> (-1.89%) ⬇️
python/cudf/cudf/core/tools/datetimes.py 80.42% <0.00%> (-4.11%) ⬇️
... and 36 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 44f21b3...f1d46ae. Read the comment docs.

cpp/src/jit/cache.cpp Outdated Show resolved Hide resolved
@trxcllnt trxcllnt requested a review from devavret May 4, 2021 15:55
@cwharris cwharris requested review from davidwendt and devavret May 4, 2021 21:26
cpp/src/jit/cache.cpp Outdated Show resolved Hide resolved
cpp/src/jit/cache.cpp Outdated Show resolved Hide resolved
Copy link
Contributor

@cwharris cwharris left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so code much wow 🐕

@trxcllnt
Copy link
Contributor Author

trxcllnt commented May 5, 2021

@gpucibot merge

@rapids-bot rapids-bot bot merged commit c0f8176 into rapidsai:branch-0.20 May 5, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3 - Ready for Review Ready for review by team improvement Improvement / enhancement to an existing function libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants