docs: add doc for multitask fine-tuning #3717
Conversation
WalkthroughWalkthroughThe recent modifications enhance the documentation on fine-tuning models in TensorFlow and PyTorch. It now includes detailed sections on implementation strategies and methods for fine-tuning, offering specific commands and configurations for PyTorch. Users can explore single-task and multi-task fine-tuning approaches extensively through this updated guide. Changes
Recent Review DetailsConfiguration used: CodeRabbit UI Files selected for processing (3)
Additional Context UsedLanguageTool (48)
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configration File (
|
iProzd
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
Out of diff range and nitpick comments (1)
doc/train/finetuning.md (1)
Line range hint
7-40: Consider revising the section to enhance clarity and correct minor grammatical issues.- Pretraining-and-finetuning is a widely used approach in other fields such as Computer Vision (CV) or Natural Language Processing (NLP) + Pretraining-and-finetuning is a widely used approach in fields such as Computer Vision (CV) and Natural Language Processing (NLP), - to vastly reduce the training cost, while it's not trivial in potential models. + which significantly reduces training costs, although it's not trivial in potential models. - Recently the emerging of methods such as [DPA-1](https://arxiv.org/abs/2208.08236) has brought us to a new stage where we can + Recently, the emergence of methods such as [DPA-1](https://arxiv.org/abs/2208.08236) has brought us to a new stage where we can - perform similar pretraining-finetuning approaches. + perform similar pretraining-and-finetuning approaches. - DPA-1 can hopefully learn the common knowledge in the pretrained dataset (especially the `force` information) + DPA-1 aims to learn the common knowledge in the pretrained dataset (especially the `force` information) - and thus reduce the computational cost in downstream training tasks. + thereby reducing the computational costs in downstream training tasks.
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## devel #3717 +/- ##
==========================================
+ Coverage 82.12% 82.19% +0.06%
==========================================
Files 511 513 +2
Lines 47363 47642 +279
Branches 2952 2980 +28
==========================================
+ Hits 38898 39159 +261
Misses 7572 7572
- Partials 893 911 +18 ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Actionable comments posted: 14
Out of diff range and nitpick comments (13)
doc/train/multi-task-training-pt.md (10)
Line range hint
11-11: Consider merging "multi-task" into "multitask" for consistency throughout the document.
Line range hint
19-19: Consider merging "multi-task" into "multitask" for consistency throughout the document.
Line range hint
25-25: Consider merging "multi-task" into "multitask" for consistency throughout the document.
Line range hint
31-31: Consider merging "multi-task" into "multitask" for consistency throughout the document.
Line range hint
33-33: Consider merging "multi-task" into "multitask" for consistency throughout the document.
Line range hint
35-35: Consider merging "multi-task" into "multitask" for consistency throughout the document.
Line range hint
39-39: Consider merging "multi-task" into "multitask" for consistency throughout the document.
Line range hint
70-70: Consider merging "multi-task" into "multitask" for consistency throughout the document.
77-77: Consider merging "multi-task" into "multitask" for consistency throughout the document.
79-79: Consider merging "multi-task" into "multitask" for consistency throughout the document.doc/train/finetuning.md (3)
88-88: Consider merging "multi-task" into "multitask" for consistency throughout the document.
90-90: Consider merging "multi-task" into "multitask" for consistency throughout the document.
112-112: Consider merging "multi-task" into "multitask" for consistency throughout the document.
There was a problem hiding this comment.
Actionable comments posted: 18
Out of diff range and nitpick comments (28)
doc/train/multi-task-training-pt.md (8)
Line range hint
11-11: Ensure consistency in terminology: use either "multi-task" or "multitask" throughout the document.- multi-task + multitask
Line range hint
19-19: Ensure consistency in terminology: use either "multi-task" or "multitask" throughout the document.- multi-task + multitask
Line range hint
25-25: Ensure consistency in terminology: use either "multi-task" or "multitask" throughout the document.- multi-task + multitask
Line range hint
31-31: Ensure consistency in terminology: use either "multi-task" or "multitask" throughout the document.- multi-task + multitask
Line range hint
33-33: Ensure consistency in terminology: use either "multi-task" or "multitask" throughout the document.- multi-task + multitask
Line range hint
35-35: Ensure consistency in terminology: use either "multi-task" or "multitask" throughout the document.- multi-task + multitask
Line range hint
39-39: Ensure consistency in terminology: use either "multi-task" or "multitask" throughout the document.- multi-task + multitask
Line range hint
70-70: Ensure consistency in terminology: use either "multi-task" or "multitask" throughout the document.- multi-task + multitaskdoc/train/multi-task-training-tf.md (19)
Line range hint
11-11: Ensure consistency in terminology: use either "multi-task" or "multitask" throughout the document.- multi-task + multitask
Line range hint
19-19: Ensure consistency in terminology: use either "multi-task" or "multitask" throughout the document.- multi-task + multitask
Line range hint
27-27: Ensure consistency in terminology: use either "multi-task" or "multitask" throughout the document.- multi-task + multitask
Line range hint
29-29: Ensure consistency in terminology: use either "multi-task" or "multitask" throughout the document.- multi-task + multitask
Line range hint
31-31: Ensure consistency in terminology: use either "multi-task" or "multitask" throughout the document.- multi-task + multitask
Line range hint
46-46: Ensure consistency in terminology: use either "multi-task" or "multitask" throughout the document.- multi-task + multitask
Line range hint
47-47: Ensure consistency in terminology: use either "multi-task" or "multitask" throughout the document.- multi-task + multitask
Line range hint
49-49: Ensure consistency in terminology: use either "multi-task" or "multitask" throughout the document.- multi-task + multitask
Line range hint
51-51: Ensure consistency in terminology: use either "multi-task" or "multitask" throughout the document.- multi-task + multitask
Line range hint
60-60: Ensure consistency in terminology: use either "multi-task" or "multitask" throughout the document.- multi-task + multitask
Line range hint
74-74: Ensure consistency in terminology: use either "multi-task" or "multitask" throughout the document.- multi-task + multitask
Line range hint
75-75: Ensure consistency in terminology: use either "multi-task" or "multitask" throughout the document.- multi-task + multitask
Line range hint
101-101: Ensure consistency in terminology: use either "multi-task" or "multitask" throughout the document.- multi-task + multitask
Line range hint
106-106: Consider adding a comma here for clarity.- Then if you want to transfer the trained descriptor and some fitting nets (take `water_ener` for example) to newly added datasets with new fitting net `water_ener_2`, + Then, if you want to transfer the trained descriptor and some fitting nets (take `water_ener` for example) to newly added datasets with new fitting net `water_ener_2`,
Line range hint
126-126: The preposition ‘of’ seems more likely in this position.- other parts in the input script, including {ref}`data_dict <training/data_dict>` and {ref}`loss_dict <loss_dict>` (optionally {ref}`fitting_weight <training/fitting_weight>`), + other parts of the input script, including {ref}`data_dict <training/data_dict>` and {ref}`loss_dict <loss_dict>` (optionally {ref}`fitting_weight <training/fitting_weight>`),
Line range hint
129-129: This word is normally spelled as one.- multi-task + multitask
Line range hint
137-137: This word is normally spelled as one.- multi-task + multitask
Line range hint
138-138: After ‘some of’, you should use ‘the’ (“some of the layers”) or simply say “some layers”.- if one want to share some of layers among fitting networks, + if one wants to share some of the layers among fitting networks,
Line range hint
141-141: Possible subject-verb agreement error.- For example, if one want to share the first and the third layers for two three-hidden-layer fitting networks, the following parameters should be set. + For example, if one wants to share the first and the third layers for two three-hidden-layer fitting networks, the following parameters should be set.doc/train/finetuning.md (1)
Line range hint
28-28: Do not mix variants of the same word (‘pretrain’ and ‘pre-train’) within a single text.- pre-trained + pretrained
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
Add docs for multitask fine-tuning. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Documentation** - Updated the fine-tuning guide with new sections on TensorFlow and PyTorch implementations. - Added detailed instructions for fine-tuning methods in PyTorch, including specific commands and configurations. - Modified the multi-task training guide to redirect users to the fine-tuning section for more comprehensive instructions. - Corrected a typo in the multi-task training TensorFlow documentation for improved clarity. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add docs for multitask fine-tuning.
Summary by CodeRabbit