Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Release v2.2.0 #404

Merged
merged 5 commits into from
Dec 6, 2024
Merged

chore: Release v2.2.0 #404

merged 5 commits into from
Dec 6, 2024

Conversation

Abhishek-TAMU
Copy link
Collaborator

Release v2.2.0 of fms-hf-tuning

dushyantbehl and others added 5 commits November 21, 2024 13:55

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Signed-off-by: Dushyant Behl <[email protected]>
Signed-off-by: Dushyant Behl <[email protected]>
…398)

Signed-off-by: Dushyant Behl <[email protected]>
* Add initial implementation of dataloader v1

Signed-off-by: Dushyant Behl <[email protected]>

* tests: reformat mock.patch to inside unit tests

Signed-off-by: Will Johnson <[email protected]>

fmt

Signed-off-by: Will Johnson <[email protected]>

* Add data config argument to data preprocessor

Signed-off-by: Dushyant Behl <[email protected]>

* fix: Changes to support current implementation

Signed-off-by: Abhishek <[email protected]>

* Ensure data handling is done within process dataargs
Removes unused dead code after adding the new framework and refactors
some test cases and files.

Signed-off-by: Dushyant Behl <[email protected]>

* Remove accelerator in favor of torch distributed check for multi node
data preprocessing

Signed-off-by: Dushyant Behl <[email protected]>

* Refactor data util tests as data handler tests.

Signed-off-by: Dushyant Behl <[email protected]>

* fix: add __init__.py to add tuning.data to python package

Signed-off-by: Will Johnson <[email protected]>

* fix: multi GPU prepare training dataset

Signed-off-by: Will Johnson <[email protected]>

* fix: lint

Signed-off-by: Will Johnson <[email protected]>

* fix: Add TODO

Signed-off-by: Will Johnson <[email protected]>

* test: add test for process_dataset_configs in HFBasedDataPreProcessor

Signed-off-by: Will Johnson <[email protected]>

* add: test cases for framework

Signed-off-by: Abhishek <[email protected]>

* fix: update function name get_dataprocessor->get_datapreprocessor

Signed-off-by: Will Johnson <[email protected]>

* Rename loader to processor

Signed-off-by: Dushyant Behl <[email protected]>

* data folders should be together

Signed-off-by: Dushyant Behl <[email protected]>

* Add code comments and make code path clearer.
Remove packing check as packing support for pretokenised data is merged
to trl. See huggingface/trl#2011

Signed-off-by: Dushyant Behl <[email protected]>

---------

Signed-off-by: Dushyant Behl <[email protected]>
Signed-off-by: Will Johnson <[email protected]>
Signed-off-by: Abhishek <[email protected]>
Co-authored-by: Will Johnson <[email protected]>
Co-authored-by: Abhishek <[email protected]>
Signed-off-by: Abhishek <[email protected]>
Copy link

github-actions bot commented Dec 5, 2024

Thanks for making a pull request! 😃
One of the maintainers will review and advise on the next steps.

@Abhishek-TAMU Abhishek-TAMU changed the title release: merge set of changes for v2.2.0 chore: Release v2.2.0 Dec 5, 2024
@github-actions github-actions bot added the chore label Dec 5, 2024
Copy link
Collaborator

@willmj willmj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The commits look correct to me!

@Abhishek-TAMU Abhishek-TAMU merged commit 419de55 into release Dec 6, 2024
9 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants