Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix pipline dataloader when batch elements contain tuple #565

Merged
merged 4 commits into from
Aug 25, 2023
Merged

Fix pipline dataloader when batch elements contain tuple #565

merged 4 commits into from
Aug 25, 2023

Conversation

ghosthamlet
Copy link
Contributor

No description provided.

jeffra
jeffra approved these changes Dec 2, 2020
View reviewed changes
Copy link
Collaborator

@jeffra jeffra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, /cc @ShadenSmith for another check though

@ShadenSmith
Copy link
Contributor

lgtm!

@ghosthamlet
Copy link
Contributor Author

Hi @jeffra, merge failed for test_pipe.py line 246, in _helper:

    assert rel_diff(base_avg, test_avg) < 0.03
AssertionError: assert 0.03816467364302331 < 0.03
 +  where 0.03816467364302331 = rel_diff(1.8875272107124328, 1.8154903507232667)

what can I do about it? it seems like not related to my simple commit.

@jeffra
Copy link
Collaborator

jeffra commented Dec 3, 2020

Hi @jeffra, merge failed for test_pipe.py line 246, in _helper:

    assert rel_diff(base_avg, test_avg) < 0.03
AssertionError: assert 0.03816467364302331 < 0.03
 +  where 0.03816467364302331 = rel_diff(1.8875272107124328, 1.8154903507232667)

what can I do about it? it seems like not related to my simple commit.

Yeah, this is pretty strange actually. I was just talking with @ShadenSmith about this as well. We need to investigate further since this shouldn't have anything to do with your PR changes.

@ghosthamlet
Copy link
Contributor Author

Ok, thanks.

@rocm-mici
Copy link

Can one of the admins verify this patch?

@jeffra jeffra requested a review from duli2012 as a code owner June 23, 2023 21:31
@awan-10 awan-10 self-assigned this Aug 18, 2023
@awan-10 awan-10 enabled auto-merge August 25, 2023 16:56
@awan-10 awan-10 added this pull request to the merge queue Aug 25, 2023
Merged via the queue into microsoft:master with commit c69bd1f Aug 25, 2023
conglongli pushed a commit to cctry/DeepSpeed that referenced this pull request Sep 14, 2023
@minjiaz
Updated the Megatron-DS section (microsoft#565)
15e7fd2 
* Updated the Megatron-DS section

* minor fix

* minor fix

* minor fix
conglongli added a commit that referenced this pull request Sep 18, 2023
@conglongli @cctry @minjiaz
DeepSpeed4Science (#569)
a5552a6 
* Integrating evoformer attention

* add cutlass version check

* Updaate error message

* add benchmark

* Update

* Update evoformer_attn.py

* Update run_evoformer_test.py

* Update evoformer_attn.py

* Update run_evoformer_test.py

* support more GPU archs

* add copyright

* add tests

* Fix bugs

* Update benchmark

* update

* Fix nvcc macro

* clean code

* fix formatting

* fix yaml import

* skip unit test when not compatible

* fix yaml requirement

* revert changes

* update tutorial

* update

* fix formatting

* fix format

* skip evoformer attn in pre-compile-ops

* revert changes

* update tutorial

* fix cutlass check

* update tutorial

* refactor tutorial

* revise

* Updated the Megatron-DS section (#565)

* Updated the Megatron-DS section

* minor fix

* minor fix

* minor fix

* separate evoformer tutorial

* Revised the ds4science landing page (#566)

* Updated the Megatron-DS section

* minor fix

* minor fix

* minor fix

* Revised the landing page

* Revised the landing page

* Removing unused file

* fix links image position

* modify main page

* fix doc

---------

Co-authored-by: Shiyang Chen <[email protected]>
Co-authored-by: Minjia Zhang <[email protected]>
github-merge-queue bot pushed a commit that referenced this pull request Sep 18, 2023
@conglongli @HeyangQin
@zhiruiluo @cli99 @tjruwase @CaffreyR @jeffra @stephen-youn @styoun @arashb @yidoe @GuanhuaWang @cmikeh2 @awan-10 @mrwyattii @tohtana @loadams @jomayeri @rraminen @cctry @minjiaz
DeepSpeed4Science (#4357)
f876d81 
* zero++ tutorial PR (#3783)

* [Fix] _conv_flops_compute when padding is a str and stride=1 (#3169)

* fix conv_flops_compute when padding is a str when stride=1

* fix error

* change type of paddings to tuple

* fix padding calculation

* apply formatting check

---------

Co-authored-by: Cheng Li <[email protected]>
Co-authored-by: Olatunji Ruwase <[email protected]>

* fix interpolate flops compute (#3782)

* use `Flops Profiler` to test `model.generate()` (#2515)

* Update profiler.py

* pre-commit run --all-files

* Delete .DS_Store

* Delete .DS_Store

* Delete .DS_Store

---------

Co-authored-by: Jeff Rasley <[email protected]>
Co-authored-by: Cheng Li <[email protected]>

* revert PR #3611 (#3786)

* bump to 0.9.6

* ZeRO++ chinese blog (#3793)

* zeropp chinese blog

* try better quality images

* make title larger

* even larger...

* various fix

* center captions

* more fixes

* fix format

* remove staging trigger (#3792)

* DeepSpeed-Triton for Inference (#3748)

Co-authored-by: Stephen Youn <[email protected]>
Co-authored-by: Arash Bakhtiari <[email protected]>
Co-authored-by: Cheng Li <[email protected]>
Co-authored-by: Ethan Doe <[email protected]>
Co-authored-by: yidoe <[email protected]>
Co-authored-by: Jeff Rasley <[email protected]>

* ZeRO++ (#3784)

Co-authored-by: HeyangQin <[email protected]>
Co-authored-by: GuanhuaWang <[email protected]>
Co-authored-by: cmikeh2 <[email protected]>
Co-authored-by: Ammar Ahmad Awan <[email protected]>
Co-authored-by: Jeff Rasley <[email protected]>
Co-authored-by: Michael Wyatt <[email protected]>
Co-authored-by: Olatunji Ruwase <[email protected]>
Co-authored-by: Reza Yazdani <[email protected]>

* adding zero++ to navigation panel of deepspeed.ai (#3796)

* Add ZeRO++ Japanese blog (#3797)

* zeropp chinese blog

* try better quality images

* make title larger

* even larger...

* various fix

* center captions

* more fixes

* fix format

* add ZeRO++ Japanese blog

* add links

---------

Co-authored-by: HeyangQin <[email protected]>
Co-authored-by: Conglong Li <[email protected]>

* Bug Fixes for autotuner and flops profiler (#1880)

* fix autotuner when backward is not called

* fix format

---------

Co-authored-by: Olatunji Ruwase <[email protected]>

* Missing strided copy for gated MLP (#3788)

Co-authored-by: Ammar Ahmad Awan <[email protected]>
Co-authored-by: Jeff Rasley <[email protected]>
Co-authored-by: Logan Adams <[email protected]>

* Requires grad checking. (#3789)

Co-authored-by: Jeff Rasley <[email protected]>

* bump to 0.10.0

* Fix Bug in transform.cu (#3534)

* Bug fix

* Fixed formatting error

---------

Co-authored-by: Logan Adams <[email protected]>

* bug fix: triton importing error (#3799)

Co-authored-by: Stephen Youn <[email protected]>
Co-authored-by: Jeff Rasley <[email protected]>

* DeepSpeed4Science (#569)

* Integrating evoformer attention

* add cutlass version check

* Updaate error message

* add benchmark

* Update

* Update evoformer_attn.py

* Update run_evoformer_test.py

* Update evoformer_attn.py

* Update run_evoformer_test.py

* support more GPU archs

* add copyright

* add tests

* Fix bugs

* Update benchmark

* update

* Fix nvcc macro

* clean code

* fix formatting

* fix yaml import

* skip unit test when not compatible

* fix yaml requirement

* revert changes

* update tutorial

* update

* fix formatting

* fix format

* skip evoformer attn in pre-compile-ops

* revert changes

* update tutorial

* fix cutlass check

* update tutorial

* refactor tutorial

* revise

* Updated the Megatron-DS section (#565)

* Updated the Megatron-DS section

* minor fix

* minor fix

* minor fix

* separate evoformer tutorial

* Revised the ds4science landing page (#566)

* Updated the Megatron-DS section

* minor fix

* minor fix

* minor fix

* Revised the landing page

* Revised the landing page

* Removing unused file

* fix links image position

* modify main page

* fix doc

---------

Co-authored-by: Shiyang Chen <[email protected]>
Co-authored-by: Minjia Zhang <[email protected]>

---------

Co-authored-by: Heyang Qin <[email protected]>
Co-authored-by: Bill Luo <[email protected]>
Co-authored-by: Cheng Li <[email protected]>
Co-authored-by: Olatunji Ruwase <[email protected]>
Co-authored-by: Guorun <[email protected]>
Co-authored-by: Jeff Rasley <[email protected]>
Co-authored-by: stephen youn <[email protected]>
Co-authored-by: Stephen Youn <[email protected]>
Co-authored-by: Arash Bakhtiari <[email protected]>
Co-authored-by: Ethan Doe <[email protected]>
Co-authored-by: yidoe <[email protected]>
Co-authored-by: GuanhuaWang <[email protected]>
Co-authored-by: cmikeh2 <[email protected]>
Co-authored-by: Ammar Ahmad Awan <[email protected]>
Co-authored-by: Michael Wyatt <[email protected]>
Co-authored-by: Reza Yazdani <[email protected]>
Co-authored-by: Masahiro Tanaka <[email protected]>
Co-authored-by: Logan Adams <[email protected]>
Co-authored-by: Joe Mayer <[email protected]>
Co-authored-by: Ramya Ramineni <[email protected]>
Co-authored-by: Shiyang Chen <[email protected]>
Co-authored-by: Minjia Zhang <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@raghavthind2005 raghavthind2005 raghavthind2005 approved these changes

@jeffra jeffra jeffra approved these changes

@ShadenSmith ShadenSmith ShadenSmith approved these changes

@arashashari arashashari Awaiting requested review from arashashari

@awan-10 awan-10 Awaiting requested review from awan-10

@cli99 cli99 Awaiting requested review from cli99

@conglongli conglongli Awaiting requested review from conglongli

@eltonzheng eltonzheng Awaiting requested review from eltonzheng

@minjiaz minjiaz Awaiting requested review from minjiaz

@niumanar niumanar Awaiting requested review from niumanar

@RezaYazdaniAminabadi RezaYazdaniAminabadi Awaiting requested review from RezaYazdaniAminabadi

@samyam samyam Awaiting requested review from samyam

@tjruwase tjruwase Awaiting requested review from tjruwase

@duli2012 duli2012 Awaiting requested review from duli2012

Assignees

@awan-10 awan-10

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@ghosthamlet @ShadenSmith @jeffra @rocm-mici @raghavthind2005 @awan-10