Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

efficiency improvements in autograd derivatives #2015

Merged
merged 1 commit into from
Oct 12, 2024

Conversation

tylerflex
Copy link
Collaborator

Orders of magnitude improvement to autograd gradient handling, especially for large field data and many structures.

Achieved through following:

  1. assuming sorted in field data interpolation for adjoint gradients.
  2. not deep copying field data containing DerivativeInfo objects when passing around.

Still work in progress as I need to figure out where else I could apply these fixes.

@tylerflex tylerflex force-pushed the tyler/autograd_/efficiency branch 2 times, most recently from 674d3c3 to 8df2720 Compare October 11, 2024 15:23
@tylerflex tylerflex changed the title efficiency improvements autograd derivatives efficiency improvements in autograd derivatives Oct 11, 2024
@tylerflex tylerflex force-pushed the tyler/autograd_/efficiency branch from 8df2720 to 0027cdb Compare October 11, 2024 17:51
Copy link
Collaborator

@yaugenst-flex yaugenst-flex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great! I guess the only thing I wonder is that derivative_info contents are never mutated? Basically after this we need to make sure that never happens, but seems like that shouldn't be the case anyways?

@tylerflex
Copy link
Collaborator Author

it should never be the case. the derivative info is basically just a container to avoid having to pass Ex_fwd, Ey_fwd, etc into all of the derivative computation methods and then interpolate into them. We'd never mutate them

@tylerflex tylerflex merged commit 1eaa8d3 into develop Oct 12, 2024
15 checks passed
@tylerflex tylerflex deleted the tyler/autograd_/efficiency branch October 12, 2024 19:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants