Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(deps): bump torch from 1.11.0 to 2.3.0 #9726

Merged
merged 11 commits into from
Aug 8, 2024

Conversation

dependabot[bot]
Copy link
Contributor

@dependabot dependabot bot commented on behalf of github Jul 25, 2024

Bumps torch from 1.11.0 to 2.3.0.

Release notes

Sourced from torch's releases.

PyTorch 2.2: FlashAttention-v2, AOTInductor

PyTorch 2.2 Release Notes

  • Highlights
  • Backwards Incompatible Changes
  • Deprecations
  • New Features
  • Improvements
  • Bug fixes
  • Performance
  • Documentation

Highlights

We are excited to announce the release of PyTorch® 2.2! PyTorch 2.2 offers ~2x performance improvements to scaled_dot_product_attention via FlashAttention-v2 integration, as well as AOTInductor, a new ahead-of-time compilation and deployment tool built for non-python server-side deployments.

This release also includes improved torch.compile support for Optimizers, a number of new inductor optimizations, and a new logging mechanism called TORCH_LOGS.

Please note that we are deprecating macOS x86 support, and PyTorch 2.2.x will be the last version that supports macOS x64.

Along with 2.2, we are also releasing a series of updates to the PyTorch domain libraries. More details can be found in the library updates blog.

This release is composed of 3,628 commits and 521 contributors since PyTorch 2.1. We want to sincerely thank our dedicated community for your contributions. As always, we encourage you to try these out and report any issues as we improve 2.2. More information about how to get started with the PyTorch 2-series can be found at our Getting Started page.

Summary:

  • scaled_dot_product_attention (SDPA) now supports FlashAttention-2, yielding around 2x speedups compared to previous versions.
  • PyTorch 2.2 introduces a new ahead-of-time extension of TorchInductor called AOTInductor, designed to compile and deploy PyTorch programs for non-python server-side.
  • torch.distributed supports a new abstraction for initializing and representing ProcessGroups called device_mesh.
  • PyTorch 2.2 ships a standardized, configurable logging mechanism called TORCH_LOGS.
  • A number of torch.compile improvements are included in PyTorch 2.2, including improved support for compiling Optimizers and improved TorchInductor fusion and layout optimizations.
  • Please note that we are deprecating macOS x86 support, and PyTorch 2.2.x will be the last version that supports macOS x64.
  • torch.ao.quantization now offers a prototype torch.export based flow

... (truncated)

Changelog

Sourced from torch's changelog.

Releasing PyTorch

Release Compatibility Matrix

Following is the Release Compatibility Matrix for PyTorch releases:

... (truncated)

Commits

Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    You can disable automated security fix PRs for this repo from the Security Alerts page.

@dependabot dependabot bot requested a review from a team as a code owner July 25, 2024 10:30
@dependabot dependabot bot requested a review from gt2345 July 25, 2024 10:30
@dependabot dependabot bot added dependencies Pull requests that update a dependency file python Pull requests that update Python code labels Jul 25, 2024
@cla-bot cla-bot bot added the cla-signed label Jul 25, 2024
Copy link

netlify bot commented Jul 25, 2024

Deploy Preview for determined-ui canceled.

Name Link
🔨 Latest commit e147539
🔍 Latest deploy log https://app.netlify.com/sites/determined-ui/deploys/66b4c796a9a35b00085ddf8b

Copy link

codecov bot commented Jul 25, 2024

Codecov Report

Attention: Patch coverage is 47.36842% with 10 lines in your changes missing coverage. Please review.

Project coverage is 54.04%. Comparing base (786f258) to head (e147539).
Report is 17 commits behind head on main.

Files Patch % Lines
model_hub/model_hub/mmdetection/utils.py 25.00% 3 Missing ⚠️
harness/determined/pytorch/_pytorch_trial.py 50.00% 2 Missing ⚠️
harness/determined/pytorch/_trainer.py 0.00% 2 Missing ⚠️
...mined/pytorch/experimental/_torch_batch_process.py 0.00% 2 Missing ⚠️
model_hub/model_hub/mmdetection/_trial.py 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #9726      +/-   ##
==========================================
- Coverage   54.07%   54.04%   -0.03%     
==========================================
  Files        1260     1260              
  Lines      155574   155574              
  Branches     3503     3504       +1     
==========================================
- Hits        84120    84085      -35     
- Misses      71308    71343      +35     
  Partials      146      146              
Flag Coverage Δ
backend 44.90% <ø> (-0.07%) ⬇️
harness 72.61% <47.36%> (-0.01%) ⬇️
web 53.23% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
harness/determined/pytorch/_experimental.py 90.90% <100.00%> (ø)
harness/determined/pytorch/_load.py 80.43% <100.00%> (ø)
harness/determined/pytorch/_pytorch_context.py 72.88% <100.00%> (ø)
...s/determined/pytorch/deepspeed/_deepspeed_trial.py 78.60% <100.00%> (ø)
...s/tests/experiment/pytorch/test_pytorch_context.py 100.00% <100.00%> (ø)
model_hub/model_hub/mmdetection/_trial.py 49.65% <0.00%> (ø)
harness/determined/pytorch/_pytorch_trial.py 79.97% <50.00%> (ø)
harness/determined/pytorch/_trainer.py 55.55% <0.00%> (ø)
...mined/pytorch/experimental/_torch_batch_process.py 75.00% <0.00%> (ø)
model_hub/model_hub/mmdetection/utils.py 78.04% <25.00%> (ø)

... and 9 files with indirect coverage changes

@determined-ci determined-ci requested a review from a team July 29, 2024 17:25
@determined-ci
Copy link
Collaborator

Docsite preview being generated for this PR.
You can (eventually) find the generated docsite here.

@determined-ci determined-ci added the documentation Improvements or additions to documentation label Jul 29, 2024
@determined-ci
Copy link
Collaborator

Docsite preview being generated for this PR.
You can (eventually) find the generated docsite here.

5 similar comments
@determined-ci
Copy link
Collaborator

Docsite preview being generated for this PR.
You can (eventually) find the generated docsite here.

@determined-ci
Copy link
Collaborator

Docsite preview being generated for this PR.
You can (eventually) find the generated docsite here.

@determined-ci
Copy link
Collaborator

Docsite preview being generated for this PR.
You can (eventually) find the generated docsite here.

@determined-ci
Copy link
Collaborator

Docsite preview being generated for this PR.
You can (eventually) find the generated docsite here.

@determined-ci
Copy link
Collaborator

Docsite preview being generated for this PR.
You can (eventually) find the generated docsite here.

Copy link
Contributor

@tara-det-ai tara-det-ai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@gt2345 gt2345 force-pushed the dependabot/pip/e2e_tests/tests/torch-2.2.0 branch from f893ec0 to 2529cdc Compare August 5, 2024 18:18
@determined-ci
Copy link
Collaborator

Docsite preview being generated for this PR.
You can (eventually) find the generated docsite here.

@determined-ci determined-ci requested a review from a team August 5, 2024 18:18
@determined-ci
Copy link
Collaborator

Docsite preview being generated for this PR.
You can (eventually) find the generated docsite here.

@determined-ci
Copy link
Collaborator

Docsite preview being generated for this PR.
You can (eventually) find the generated docsite here.

@jgongd jgongd force-pushed the dependabot/pip/e2e_tests/tests/torch-2.2.0 branch from c8e02c2 to b581a0d Compare August 5, 2024 21:42
@determined-ci
Copy link
Collaborator

Docsite preview being generated for this PR.
You can (eventually) find the generated docsite here.

2 similar comments
@determined-ci
Copy link
Collaborator

Docsite preview being generated for this PR.
You can (eventually) find the generated docsite here.

@determined-ci
Copy link
Collaborator

Docsite preview being generated for this PR.
You can (eventually) find the generated docsite here.

@jgongd jgongd force-pushed the dependabot/pip/e2e_tests/tests/torch-2.2.0 branch from dcf7b72 to 2566307 Compare August 7, 2024 15:23
@determined-ci
Copy link
Collaborator

Docsite preview being generated for this PR.
You can (eventually) find the generated docsite here.

@determined-ci
Copy link
Collaborator

Docsite preview being generated for this PR.
You can (eventually) find the generated docsite here.

@jgongd jgongd changed the title chore(deps): bump torch from 1.11.0 to 2.2.0 in /e2e_tests/tests chore(deps): bump torch from 1.11.0 to 2.3.0 in /e2e_tests/tests Aug 7, 2024
@determined-ci
Copy link
Collaborator

Docsite preview being generated for this PR.
You can (eventually) find the generated docsite here.

1 similar comment
@determined-ci
Copy link
Collaborator

Docsite preview being generated for this PR.
You can (eventually) find the generated docsite here.

@jgongd jgongd changed the title chore(deps): bump torch from 1.11.0 to 2.3.0 in /e2e_tests/tests chore(deps): bump torch from 1.11.0 to 2.3.0 Aug 7, 2024
@determined-ci
Copy link
Collaborator

Docsite preview being generated for this PR.
You can (eventually) find the generated docsite here.

Copy link
Contributor

@MikhailKardash MikhailKardash left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM


[mypy-torch.*]
; starting in torch 2.2.0
; site-packages/torch/distributed/optim/apply_optimizer_in_backward.py: : note: please use --show-traceback to print a traceback when reporting a bug
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this note for?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! I meant to put the actual error log as the comment. I will update this.

@gt2345 gt2345 self-requested a review August 7, 2024 22:01
Copy link
Contributor

@gt2345 gt2345 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work, thanks for your effort!

@determined-ci
Copy link
Collaborator

Docsite preview being generated for this PR.
You can (eventually) find the generated docsite here.

@determined-ci determined-ci requested a review from a team August 8, 2024 13:26
@jgongd jgongd merged commit a11e9e8 into main Aug 8, 2024
85 of 98 checks passed
@jgongd jgongd deleted the dependabot/pip/e2e_tests/tests/torch-2.2.0 branch August 8, 2024 17:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-signed dependencies Pull requests that update a dependency file documentation Improvements or additions to documentation python Pull requests that update Python code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants