Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up notebook run action #1147

Merged
merged 13 commits into from
Nov 3, 2024
Merged

Speed up notebook run action #1147

merged 13 commits into from
Nov 3, 2024

Conversation

wd60622
Copy link
Contributor

@wd60622 wd60622 commented Nov 1, 2024

Closes #1141

This runs into some issues with sampling Flat distribution, loading the model, and some sample_stats

This potentially could reduce the run time of the notebooks to 1/3 of the original time. One local notebook run when from 2.5 minutes to 19 seconds.

CC: @juanitorduz

@wd60622 wd60622 marked this pull request as draft November 1, 2024 21:17
@github-actions github-actions bot added docs Improvements or additions to documentation maintenance no releasenotes labels Nov 1, 2024
Copy link

codecov bot commented Nov 1, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.61%. Comparing base (31def91) to head (dfd1792).
Report is 13 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1147   +/-   ##
=======================================
  Coverage   95.61%   95.61%           
=======================================
  Files          39       39           
  Lines        4063     4063           
=======================================
  Hits         3885     3885           
  Misses        178      178           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Collaborator

@juanitorduz juanitorduz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @wd60622 ! I left some small questions and comments! Looks cool!

scripts/run_notebooks/runner.py Outdated Show resolved Hide resolved
scripts/run_notebooks/runner.py Outdated Show resolved Hide resolved
scripts/run_notebooks/runner.py Outdated Show resolved Hide resolved
scripts/run_notebooks/runner.py Outdated Show resolved Hide resolved
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@github-actions github-actions bot added the MMM label Nov 2, 2024
@wd60622
Copy link
Contributor Author

wd60622 commented Nov 2, 2024

I am looking at some of the logs. Seems like it was taken down from 40 minutes to 10 minutes 😆

@wd60622 wd60622 marked this pull request as ready for review November 2, 2024 15:30
@wd60622 wd60622 added the CLV label Nov 2, 2024
@juanitorduz
Copy link
Collaborator

@wd60622 This is a new trick for me, hehe. How does the injected code "replace" the sampling step in the notebook? Can you maybe push an error in a notebook and see if it get caught (we can then revert the commit). This seems like magic XD

@wd60622
Copy link
Contributor Author

wd60622 commented Nov 2, 2024

@wd60622 This is a new trick for me, hehe. How does the injected code "replace" the sampling step in the notebook? Can you maybe push an error in a notebook and see if it get caught (we can then revert the commit). This seems like magic XD

The injected first cell will run the "injected.py". This is before any imports which makes python think that pm.sample is mock_sample and pm.Flat = pm.Normal and pm.HalfFlat = pm.HalfFlat because they are cached even when they are imported from our modules

Copy link
Collaborator

@juanitorduz juanitorduz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok! I thnik I get it now :D Very nice! After removing the commented lines (see comment above), feel free to merge 🚀

@wd60622 wd60622 mentioned this pull request Nov 3, 2024
@wd60622 wd60622 merged commit 7a84c64 into main Nov 3, 2024
14 checks passed
@wd60622 wd60622 deleted the speed-up-notebooks branch November 3, 2024 16:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLV docs Improvements or additions to documentation maintenance MMM no releasenotes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Speed up make notebook
2 participants