Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix type annotations for arguments in training_args #24550

Merged
merged 21 commits into from
Jul 20, 2023
Merged

fix type annotations for arguments in training_args #24550

Changes from 11 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
4cf0835
testing
shauray8 Jun 22, 2023
7e2f313
example script
shauray8 Jun 25, 2023
f9f3239
Merge branch 'main' into fix_trainingargs
shauray8 Jun 28, 2023
2a2b232
fix typehinting
shauray8 Jun 28, 2023
bb23735
some tests
shauray8 Jun 28, 2023
1569775
make test
shauray8 Jun 28, 2023
266d5b0
optional update
shauray8 Jun 28, 2023
8f6e382
Union of arguments
shauray8 Jun 28, 2023
056fd3d
does this fix the issue
shauray8 Jun 28, 2023
71049cb
remove reports
shauray8 Jun 28, 2023
7f9b4cb
set default to False
shauray8 Jun 28, 2023
878eb9b
documentation change
shauray8 Jun 29, 2023
4443814
None support
shauray8 Jun 29, 2023
07760be
does not need None
shauray8 Jun 29, 2023
5ce5f1f
Merge branch 'huggingface:main' into fix_trainingargs
shauray8 Jul 9, 2023
fbac8a7
Fix typing annotations for FSDP and DeepSpeed in TrainingArguments (#…
mryab Jun 28, 2023
ea1b714
Revert "Fix typing annotations for FSDP and DeepSpeed in TrainingArgu…
sgugger Jun 29, 2023
05cc09b
Fix typing annotations for FSDP and DeepSpeed in TrainingArguments (#…
mryab Jun 28, 2023
760f89c
merge
shauray8 Jul 9, 2023
20d6b84
hacky fix
shauray8 Jul 10, 2023
272f654
fixup
shauray8 Jul 20, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions src/transformers/training_args.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -944,8 +944,8 @@ class TrainingArguments:
)
},
)
sharded_ddp: str = field(
default="",
sharded_ddp: Optional[Union[str, bool, List[ShardedDDPOption]]] = field(
default=False,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default shouldn't be touched.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the documentation says that it defaults to False - sharded_ddp (bool, str or list of [~trainer_utils.ShardedDDPOption], optional, defaults to False). but if the default variable has to be " ", I'll try doing it some other way.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then let's fix the documentation as a change in default is potentially braking.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, changing default to "" in the documentation as well as in the code.

metadata={
"help": (
"Whether or not to use sharded DDP training (in distributed training only). The base option should be"
Expand All @@ -955,8 +955,8 @@ class TrainingArguments:
),
},
)
fsdp: str = field(
default="",
fsdp: Optional[Union[str, bool, List[FSDPOption]]] = field(
default=False,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same for this - fsdp (`bool`, `str` or list of [`~trainer_utils.FSDPOption`], *optional*, defaults to `False`)

metadata={
"help": (
"Whether or not to use PyTorch Fully Sharded Data Parallel (FSDP) training (in distributed training"
Expand All @@ -976,7 +976,7 @@ class TrainingArguments:
)
},
)
fsdp_config: Optional[str] = field(
fsdp_config: Optional[Union[str, Dict]] = field(
default=None,
metadata={
"help": (
Expand Down