Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[TPU] Set the compute_dtype with XLAFSDP #18497

Merged
merged 6 commits into from
Sep 7, 2023
Merged

[TPU] Set the compute_dtype with XLAFSDP #18497

merged 6 commits into from
Sep 7, 2023

Conversation

carmocca
Copy link
Contributor

@carmocca carmocca commented Sep 6, 2023 •
edited by github-actions bot
Loading

What does this PR do?

This logic was missing. Not updating the CHANGELOG since this is unreleased

cc @Borda @carmocca @justusschock @awaelchli @JackCaoG @Liyang90 @gkroiz @steventk-g

@carmocca carmocca added bug Something isn't working fabric lightning.fabric.Fabric strategy: xla labels Sep 6, 2023
@carmocca carmocca added this to the 2.1 milestone Sep 6, 2023
@carmocca carmocca requested a review from awaelchli as a code owner September 6, 2023 18:00
@carmocca carmocca self-assigned this Sep 6, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Sep 6, 2023 •
edited
Loading

⚡ Required checks status: All passing 🟢

Groups summary

🟢 pytorch_lightning: Tests workflow
Check ID Status
pl-cpu (macOS-11, lightning, 3.8, 1.11) success ✅
pl-cpu (macOS-11, lightning, 3.9, 1.12) success ✅
pl-cpu (macOS-11, lightning, 3.10, 1.13) success ✅
pl-cpu (macOS-11, lightning, 3.10, 2.0) success ✅
pl-cpu (macOS-11, lightning, 3.8, 1.11, oldest) success ✅
pl-cpu (ubuntu-20.04, lightning, 3.8, 1.11) success ✅
pl-cpu (ubuntu-20.04, lightning, 3.9, 1.12) success ✅
pl-cpu (ubuntu-20.04, lightning, 3.10, 1.13) success ✅
pl-cpu (ubuntu-20.04, lightning, 3.10, 2.0) success ✅
pl-cpu (ubuntu-20.04, lightning, 3.8, 1.11, oldest) success ✅
pl-cpu (windows-2022, lightning, 3.8, 1.11) success ✅
pl-cpu (windows-2022, lightning, 3.9, 1.12) success ✅
pl-cpu (windows-2022, lightning, 3.10, 1.13) success ✅
pl-cpu (windows-2022, lightning, 3.10, 2.0) success ✅
pl-cpu (windows-2022, lightning, 3.8, 1.11, oldest) success ✅
pl-cpu (macOS-11, pytorch, 3.8, 1.13) success ✅
pl-cpu (ubuntu-20.04, pytorch, 3.8, 1.13) success ✅
pl-cpu (windows-2022, pytorch, 3.8, 1.13) success ✅
pl-cpu (macOS-12, pytorch, 3.11, 2.0) success ✅
pl-cpu (ubuntu-22.04, pytorch, 3.11, 2.0) success ✅
pl-cpu (windows-2022, pytorch, 3.11, 2.0) success ✅

These checks are required after the changes to src/lightning/fabric/strategies/xla_fsdp.py.

🟢 pytorch_lightning: Azure GPU
Check ID Status
[pytorch-lightning (GPUs) (testing Lightning latest)](https://dev.azure.com/Lightning-AI/72ab7ed8-b00f-4b6e-b131-3388f7ffafa7/_build/results?buildId=173441&view=logs&jobId=47e66f3c-897a-5428-da11-bf5c7745762e) success
[pytorch-lightning (GPUs) (testing PyTorch latest)](https://dev.azure.com/Lightning-AI/72ab7ed8-b00f-4b6e-b131-3388f7ffafa7/_build/results?buildId=173441&view=logs&jobId=3f274fac-2e11-54ca-487e-194c91f3ae9f) success

These checks are required after the changes to src/lightning/fabric/strategies/xla_fsdp.py.

🟢 pytorch_lightning: Benchmarks
Check ID Status
lightning.Benchmarks success ✅

These checks are required after the changes to src/lightning/fabric/strategies/xla_fsdp.py.

🟢 fabric: Docs
Check ID Status
docs-make (fabric, doctest) success ✅
docs-make (fabric, html) success ✅

These checks are required after the changes to src/lightning/fabric/strategies/xla_fsdp.py.

🟢 lightning_fabric: CPU workflow
Check ID Status
fabric-cpu (macOS-11, lightning, 3.8, 1.11) success ✅
fabric-cpu (macOS-11, lightning, 3.9, 1.12) success ✅
fabric-cpu (macOS-11, lightning, 3.10, 1.13) success ✅
fabric-cpu (macOS-11, lightning, 3.10, 2.0) success ✅
fabric-cpu (macOS-11, lightning, 3.8, 1.11, oldest) success ✅
fabric-cpu (ubuntu-20.04, lightning, 3.8, 1.11) success ✅
fabric-cpu (ubuntu-20.04, lightning, 3.9, 1.12) success ✅
fabric-cpu (ubuntu-20.04, lightning, 3.10, 1.13) success ✅
fabric-cpu (ubuntu-20.04, lightning, 3.10, 2.0) success ✅
fabric-cpu (ubuntu-20.04, lightning, 3.8, 1.11, oldest) success ✅
fabric-cpu (windows-2022, lightning, 3.8, 1.11) success ✅
fabric-cpu (windows-2022, lightning, 3.9, 1.12) success ✅
fabric-cpu (windows-2022, lightning, 3.10, 1.13) success ✅
fabric-cpu (windows-2022, lightning, 3.10, 2.0) success ✅
fabric-cpu (windows-2022, lightning, 3.8, 1.11, oldest) success ✅
fabric-cpu (macOS-11, fabric, 3.8, 1.13) success ✅
fabric-cpu (ubuntu-20.04, fabric, 3.8, 1.13) success ✅
fabric-cpu (windows-2022, fabric, 3.8, 1.13) success ✅
fabric-cpu (macOS-12, fabric, 3.11, 2.0) success ✅
fabric-cpu (ubuntu-22.04, fabric, 3.11, 2.0) success ✅
fabric-cpu (windows-2022, fabric, 3.11, 2.0) success ✅

These checks are required after the changes to src/lightning/fabric/strategies/xla_fsdp.py, tests/tests_fabric/conftest.py, tests/tests_fabric/strategies/test_xla_fsdp.py.

🟢 lightning_fabric: Azure GPU
Check ID Status
[lightning-fabric (GPUs) (testing Fabric latest)](https://dev.azure.com/Lightning-AI/72ab7ed8-b00f-4b6e-b131-3388f7ffafa7/_build/results?buildId=173443&view=logs&jobId=3f274fac-2e11-54ca-487e-194c91f3ae9f) success
[lightning-fabric (GPUs) (testing Lightning latest)](https://dev.azure.com/Lightning-AI/72ab7ed8-b00f-4b6e-b131-3388f7ffafa7/_build/results?buildId=173443&view=logs&jobId=b2def368-7fa8-5edf-f15e-38e7ac88d76c) success

These checks are required after the changes to src/lightning/fabric/strategies/xla_fsdp.py, tests/tests_fabric/conftest.py, tests/tests_fabric/strategies/test_xla_fsdp.py.

🟢 mypy
Check ID Status
mypy success ✅

These checks are required after the changes to src/lightning/fabric/strategies/xla_fsdp.py.

🟢 install
Check ID Status
install-pkg (ubuntu-22.04, app, 3.8) success ✅
install-pkg (ubuntu-22.04, app, 3.11) success ✅
install-pkg (ubuntu-22.04, fabric, 3.8) success ✅
install-pkg (ubuntu-22.04, fabric, 3.11) success ✅
install-pkg (ubuntu-22.04, pytorch, 3.8) success ✅
install-pkg (ubuntu-22.04, pytorch, 3.11) success ✅
install-pkg (ubuntu-22.04, lightning, 3.8) success ✅
install-pkg (ubuntu-22.04, lightning, 3.11) success ✅
install-pkg (ubuntu-22.04, notset, 3.8) success ✅
install-pkg (ubuntu-22.04, notset, 3.11) success ✅
install-pkg (macOS-12, app, 3.8) success ✅
install-pkg (macOS-12, app, 3.11) success ✅
install-pkg (macOS-12, fabric, 3.8) success ✅
install-pkg (macOS-12, fabric, 3.11) success ✅
install-pkg (macOS-12, pytorch, 3.8) success ✅
install-pkg (macOS-12, pytorch, 3.11) success ✅
install-pkg (macOS-12, lightning, 3.8) success ✅
install-pkg (macOS-12, lightning, 3.11) success ✅
install-pkg (macOS-12, notset, 3.8) success ✅
install-pkg (macOS-12, notset, 3.11) success ✅
install-pkg (windows-2022, app, 3.8) success ✅
install-pkg (windows-2022, app, 3.11) success ✅
install-pkg (windows-2022, fabric, 3.8) success ✅
install-pkg (windows-2022, fabric, 3.11) success ✅
install-pkg (windows-2022, pytorch, 3.8) success ✅
install-pkg (windows-2022, pytorch, 3.11) success ✅
install-pkg (windows-2022, lightning, 3.8) success ✅
install-pkg (windows-2022, lightning, 3.11) success ✅
install-pkg (windows-2022, notset, 3.8) success ✅
install-pkg (windows-2022, notset, 3.11) success ✅

These checks are required after the changes to src/lightning/fabric/strategies/xla_fsdp.py.

Thank you for your contribution! 💜

Note
This comment is automatically generated and updates for 60 minutes every 180 seconds. If you have any other questions, contact carmocca for help.

@codecov
Copy link

codecov bot commented Sep 6, 2023 •
edited
Loading

Codecov Report

Merging #18497 (2f50164) into master (ea0f6ba) will decrease coverage by 30%.
Report is 2 commits behind head on master.
The diff coverage is 83%.

Additional details and impacted files 
@@            Coverage Diff             @@
##           master   #18497      +/-   ##
==========================================
- Coverage      85%      55%     -30%     
==========================================
  Files         427      422       -5     
  Lines       33332    33314      -18     
==========================================
- Hits        28256    18183   -10073     
- Misses       5076    15131   +10055

@carmocca carmocca mentioned this pull request Sep 6, 2023
Merged
awaelchli
awaelchli approved these changes Sep 6, 2023
View reviewed changes
src/lightning/fabric/strategies/xla_fsdp.py Outdated Show resolved Hide resolved
@mergify mergify bot added the ready PRs ready to be merged label Sep 7, 2023
@carmocca carmocca merged commit e1c5c5a into master Sep 7, 2023
116 checks passed
@carmocca carmocca deleted the carmocca/compute-dtype-xlafsdp branch September 7, 2023 16:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Borda Borda Borda approved these changes

@awaelchli awaelchli awaelchli approved these changes

@justusschock justusschock Awaiting requested review from justusschock justusschock is a code owner

Assignees

@carmocca carmocca

Labels
bug Something isn't working fabric lightning.fabric.Fabric ready PRs ready to be merged strategy: xla
Projects
None yet
Milestone
2.1
Development

Successfully merging this pull request may close these issues.
3 participants
@carmocca @awaelchli @Borda