-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Enhance] Add extra dataloader settings in configs. #752
Merged
mzr1996
merged 3 commits into
open-mmlab:dev
from
mzr1996:different_samples_for_eval_train_test
Apr 1, 2022
Merged
[Enhance] Add extra dataloader settings in configs. #752
mzr1996
merged 3 commits into
open-mmlab:dev
from
mzr1996:different_samples_for_eval_train_test
Apr 1, 2022
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
in the `data` field to specify different arguments.
Codecov Report
@@ Coverage Diff @@
## dev #752 +/- ##
==========================================
+ Coverage 85.11% 86.67% +1.56%
==========================================
Files 124 126 +2
Lines 7690 7857 +167
Branches 1328 1356 +28
==========================================
+ Hits 6545 6810 +265
+ Misses 951 846 -105
- Partials 194 201 +7
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
6 tasks
mzr1996
force-pushed
the
different_samples_for_eval_train_test
branch
from
March 30, 2022 12:20
f19c002
to
e539a63
Compare
ayulockin
pushed a commit
to ayulockin/mmclassification
that referenced
this pull request
May 6, 2022
* [Enhance] Add extra dataloader settings in configs. (open-mmlab#752) * Use `train_dataloader`, `val_dataloader` and `test_dataloader` settings in the `data` field to specify different arguments. * Fix bug * Fix bug * [Enhance] Improve CPE performance by reduce memory copy. (open-mmlab#762) * [Feature] Support resize relative position embedding in `SwinTransformer`. (open-mmlab#749) * [Feature]: Add resize rel pos embed * [Refactor]: Create a separated resize_rel_pos_bias_table func * [Refactor]: Refactor rel pos embed bias * [Refactor]: Move interpolate into func * Remove index buffer only when window_size changes Co-authored-by: mzr1996 <[email protected]> * [Feature] Add PoolFormer backbone and checkpoints. (open-mmlab#746) * add PoolFormer * fix some typos in PoolFormer * fix lint error * modify out_indices and gap * fix typo * fix lint * fix typo * fix typo in poolforemr README * fix lint * Update some paths * Refactor freeze_stages method * Add unit tests * Fix lint Co-authored-by: mzr1996 <[email protected]> * Bump version to v0.22.1 (open-mmlab#785) * [Docs] Refine API reference. (open-mmlab#774) * [Docs] Refine API reference * Add PoolFormer * [Docs] Fix docs. * [Enhance] Reduce the memory usage of unit tests for Swin-Transformer. (open-mmlab#759) * [Feature] Support VAN. (open-mmlab#739) * add van * fix config * add metafile * add test * model convert script * fix review * fix lint * fix the configs and improve docs * rm debug lines * add VAN into api Co-authored-by: Yu Zhaohui <[email protected]> * [Feature] Support DenseNet. (open-mmlab#750) * init add densenet implementation * Add config and converted models * update meta * add test for memory efficient * Add docs * add doc for jit * Update checkpoint path * Update readthedocs Co-authored-by: mzr1996 <[email protected]> * [Fix] Use symbolic link in the API reference of Chinese docs. * [Enhance] Support training on IPU and add fine-tuning configs of ViT. (open-mmlab#723) * implement training and evaluation on IPU * fp16 SOTA * Tput reaches 5600 * 123 * add poptorch dataloder * change ipu_replicas to ipu-replicas * add noqa to config long line(website) * remove ipu dataloder test code * del one blank line in test_builder * refine the dataloder initialization * fix a typo * refine args for dataloder * remove an annoted line * process one more conflict * adjust code structure in mmcv.ipu * adjust ipu code structure in mmcv * IPUDataloader to IPUDataLoader * align with mmcv * adjust according to mmcv * mmcv code structre fixed Co-authored-by: hudi <[email protected]> * [Fix] Fix lint and mmcv version requirement for IPU. * Bump version to v0.23.0 (open-mmlab#809) * Refacoter Wandb hook and refine docstring Co-authored-by: XiaobingZhang <[email protected]> Co-authored-by: Yuan Liu <[email protected]> Co-authored-by: Weihao Yu <[email protected]> Co-authored-by: takuoko <[email protected]> Co-authored-by: Yu Zhaohui <[email protected]> Co-authored-by: Hubert <[email protected]> Co-authored-by: Hu Di <[email protected]> Co-authored-by: hudi <[email protected]>
mzr1996
added a commit
that referenced
this pull request
Jun 2, 2022
…Biases Integration) (#764) * wandb integration * visualize using wandb tables * wandb tables enhanced * Refactor MMClsWandbHook (#1) * [Enhance] Add extra dataloader settings in configs. (#752) * Use `train_dataloader`, `val_dataloader` and `test_dataloader` settings in the `data` field to specify different arguments. * Fix bug * Fix bug * [Enhance] Improve CPE performance by reduce memory copy. (#762) * [Feature] Support resize relative position embedding in `SwinTransformer`. (#749) * [Feature]: Add resize rel pos embed * [Refactor]: Create a separated resize_rel_pos_bias_table func * [Refactor]: Refactor rel pos embed bias * [Refactor]: Move interpolate into func * Remove index buffer only when window_size changes Co-authored-by: mzr1996 <[email protected]> * [Feature] Add PoolFormer backbone and checkpoints. (#746) * add PoolFormer * fix some typos in PoolFormer * fix lint error * modify out_indices and gap * fix typo * fix lint * fix typo * fix typo in poolforemr README * fix lint * Update some paths * Refactor freeze_stages method * Add unit tests * Fix lint Co-authored-by: mzr1996 <[email protected]> * Bump version to v0.22.1 (#785) * [Docs] Refine API reference. (#774) * [Docs] Refine API reference * Add PoolFormer * [Docs] Fix docs. * [Enhance] Reduce the memory usage of unit tests for Swin-Transformer. (#759) * [Feature] Support VAN. (#739) * add van * fix config * add metafile * add test * model convert script * fix review * fix lint * fix the configs and improve docs * rm debug lines * add VAN into api Co-authored-by: Yu Zhaohui <[email protected]> * [Feature] Support DenseNet. (#750) * init add densenet implementation * Add config and converted models * update meta * add test for memory efficient * Add docs * add doc for jit * Update checkpoint path * Update readthedocs Co-authored-by: mzr1996 <[email protected]> * [Fix] Use symbolic link in the API reference of Chinese docs. * [Enhance] Support training on IPU and add fine-tuning configs of ViT. (#723) * implement training and evaluation on IPU * fp16 SOTA * Tput reaches 5600 * 123 * add poptorch dataloder * change ipu_replicas to ipu-replicas * add noqa to config long line(website) * remove ipu dataloder test code * del one blank line in test_builder * refine the dataloder initialization * fix a typo * refine args for dataloder * remove an annoted line * process one more conflict * adjust code structure in mmcv.ipu * adjust ipu code structure in mmcv * IPUDataloader to IPUDataLoader * align with mmcv * adjust according to mmcv * mmcv code structre fixed Co-authored-by: hudi <[email protected]> * [Fix] Fix lint and mmcv version requirement for IPU. * Bump version to v0.23.0 (#809) * Refacoter Wandb hook and refine docstring Co-authored-by: XiaobingZhang <[email protected]> Co-authored-by: Yuan Liu <[email protected]> Co-authored-by: Weihao Yu <[email protected]> Co-authored-by: takuoko <[email protected]> Co-authored-by: Yu Zhaohui <[email protected]> Co-authored-by: Hubert <[email protected]> Co-authored-by: Hu Di <[email protected]> Co-authored-by: hudi <[email protected]> * shuffle val data * minor updates * minor fix Co-authored-by: Ma Zerun <[email protected]> Co-authored-by: XiaobingZhang <[email protected]> Co-authored-by: Yuan Liu <[email protected]> Co-authored-by: Weihao Yu <[email protected]> Co-authored-by: takuoko <[email protected]> Co-authored-by: Yu Zhaohui <[email protected]> Co-authored-by: Hubert <[email protected]> Co-authored-by: Hu Di <[email protected]> Co-authored-by: hudi <[email protected]>
mzr1996
added a commit
to mzr1996/mmpretrain
that referenced
this pull request
Nov 24, 2022
* Use `train_dataloader`, `val_dataloader` and `test_dataloader` settings in the `data` field to specify different arguments. * Fix bug * Fix bug
mzr1996
added a commit
to mzr1996/mmpretrain
that referenced
this pull request
Nov 24, 2022
…Biases Integration) (open-mmlab#764) * wandb integration * visualize using wandb tables * wandb tables enhanced * Refactor MMClsWandbHook (open-mmlab#1) * [Enhance] Add extra dataloader settings in configs. (open-mmlab#752) * Use `train_dataloader`, `val_dataloader` and `test_dataloader` settings in the `data` field to specify different arguments. * Fix bug * Fix bug * [Enhance] Improve CPE performance by reduce memory copy. (open-mmlab#762) * [Feature] Support resize relative position embedding in `SwinTransformer`. (open-mmlab#749) * [Feature]: Add resize rel pos embed * [Refactor]: Create a separated resize_rel_pos_bias_table func * [Refactor]: Refactor rel pos embed bias * [Refactor]: Move interpolate into func * Remove index buffer only when window_size changes Co-authored-by: mzr1996 <[email protected]> * [Feature] Add PoolFormer backbone and checkpoints. (open-mmlab#746) * add PoolFormer * fix some typos in PoolFormer * fix lint error * modify out_indices and gap * fix typo * fix lint * fix typo * fix typo in poolforemr README * fix lint * Update some paths * Refactor freeze_stages method * Add unit tests * Fix lint Co-authored-by: mzr1996 <[email protected]> * Bump version to v0.22.1 (open-mmlab#785) * [Docs] Refine API reference. (open-mmlab#774) * [Docs] Refine API reference * Add PoolFormer * [Docs] Fix docs. * [Enhance] Reduce the memory usage of unit tests for Swin-Transformer. (open-mmlab#759) * [Feature] Support VAN. (open-mmlab#739) * add van * fix config * add metafile * add test * model convert script * fix review * fix lint * fix the configs and improve docs * rm debug lines * add VAN into api Co-authored-by: Yu Zhaohui <[email protected]> * [Feature] Support DenseNet. (open-mmlab#750) * init add densenet implementation * Add config and converted models * update meta * add test for memory efficient * Add docs * add doc for jit * Update checkpoint path * Update readthedocs Co-authored-by: mzr1996 <[email protected]> * [Fix] Use symbolic link in the API reference of Chinese docs. * [Enhance] Support training on IPU and add fine-tuning configs of ViT. (open-mmlab#723) * implement training and evaluation on IPU * fp16 SOTA * Tput reaches 5600 * 123 * add poptorch dataloder * change ipu_replicas to ipu-replicas * add noqa to config long line(website) * remove ipu dataloder test code * del one blank line in test_builder * refine the dataloder initialization * fix a typo * refine args for dataloder * remove an annoted line * process one more conflict * adjust code structure in mmcv.ipu * adjust ipu code structure in mmcv * IPUDataloader to IPUDataLoader * align with mmcv * adjust according to mmcv * mmcv code structre fixed Co-authored-by: hudi <[email protected]> * [Fix] Fix lint and mmcv version requirement for IPU. * Bump version to v0.23.0 (open-mmlab#809) * Refacoter Wandb hook and refine docstring Co-authored-by: XiaobingZhang <[email protected]> Co-authored-by: Yuan Liu <[email protected]> Co-authored-by: Weihao Yu <[email protected]> Co-authored-by: takuoko <[email protected]> Co-authored-by: Yu Zhaohui <[email protected]> Co-authored-by: Hubert <[email protected]> Co-authored-by: Hu Di <[email protected]> Co-authored-by: hudi <[email protected]> * shuffle val data * minor updates * minor fix Co-authored-by: Ma Zerun <[email protected]> Co-authored-by: XiaobingZhang <[email protected]> Co-authored-by: Yuan Liu <[email protected]> Co-authored-by: Weihao Yu <[email protected]> Co-authored-by: takuoko <[email protected]> Co-authored-by: Yu Zhaohui <[email protected]> Co-authored-by: Hubert <[email protected]> Co-authored-by: Hu Di <[email protected]> Co-authored-by: hudi <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
In some situation, we need to use different dataloader settings like
samples_per_gpu
in train/val/test dataloader.Modification
Use
train_dataloader
,val_dataloader
andtest_dataloader
settings in thedata
field to specify different arguments.Use cases (Optional)
Checklist
Before PR:
After PR: