Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Enzyme batch size and gradient #557

Merged
merged 2 commits into from
Oct 9, 2024
Merged

Conversation

gdalle
Copy link
Member

@gdalle gdalle commented Oct 9, 2024

DI extensions

Enzyme:

  • Threshold batch size to input size (to avoid propagating 16 tangents when the input has size 2)
  • Use Enzyme.gradient instead of Enzyme.autodiff for out-of-place operations, because it works with StaticArrays out of the box (see also Test Enzyme with StaticArrays #558)

@codecov-commenter
Copy link

codecov-commenter commented Oct 9, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.39%. Comparing base (3470e14) to head (5944472).
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #557   +/-   ##
=======================================
  Coverage   98.39%   98.39%           
=======================================
  Files         107      107           
  Lines        4739     4739           
=======================================
  Hits         4663     4663           
  Misses         76       76           
Flag Coverage Δ
DI 98.63% <100.00%> (ø)
DIT 97.90% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@gdalle gdalle changed the title Adaptive Enzyme batch size Improve Enzyme batch size and gradient Oct 9, 2024
@gdalle gdalle marked this pull request as ready for review October 9, 2024 16:39
@gdalle gdalle merged commit f663225 into main Oct 9, 2024
84 checks passed
@gdalle gdalle deleted the gd/adaptive_enzyme_batchsize branch October 9, 2024 17:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants