Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

torch.load warnings #4143

Closed
njzjz opened this issue Sep 19, 2024 · 0 comments · Fixed by #4147
Closed

torch.load warnings #4143

njzjz opened this issue Sep 19, 2024 · 0 comments · Fixed by #4147
Assignees
@njzjz
Labels
bug
Milestone
v3.0.0

Comments

@njzjz
Copy link
Member

njzjz commented Sep 19, 2024 •
edited
Loading

/home/njzjz/codes/deepmd-kit/deepmd/pt/infer/inference.py:37: FutureWarning: You are using torch.load with weights_only=False (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for weights_only will be flipped to True. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via torch.serialization.add_safe_globals. We recommend you start setting weights_only=True for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
state_dict = torch.load(model_ckpt, map_location=DEVICE)
@njzjz njzjz added the bug label Sep 19, 2024
njzjz added a commit to njzjz/deepmd-kit that referenced this issue Sep 19, 2024
@njzjz
fix: set weights_only=True for torch.load
1c0f994 
Fix deepmodeling#4143.

Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz njzjz mentioned this issue Sep 19, 2024
Merged
@njzjz njzjz linked a pull request Sep 19, 2024 that will close this issue
fix(pt): set weights_only=True for torch.load #4147
Merged
@iProzd iProzd mentioned this issue Sep 20, 2024
Merged
github-merge-queue bot pushed a commit that referenced this issue Sep 21, 2024
@iProzd
fix(pt): make state_dict safe for weights_only (#4148)
532e309 
See #4147 and #4143.
We can first make `state_dict` safe for `weights_only`, then make a
breaking change when loading `state_dict` in the future.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Enhanced model saving functionality by ensuring learning rates are
consistently stored as floats, improving type consistency.
  
- **Bug Fixes**
- Updated model loading behavior in tests to focus solely on model
weights, which may resolve issues related to state dictionary loading.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
@iProzd iProzd moved this to mustfix in DeePMD-kit V3.0.0 RC Sep 26, 2024
@iProzd iProzd moved this from mustfix to enhance in DeePMD-kit V3.0.0 RC Sep 26, 2024
@iProzd iProzd removed the status in DeePMD-kit V3.0.0 RC Sep 26, 2024
@iProzd iProzd moved this to mustfix in DeePMD-kit V3.0.0 RC Sep 26, 2024
@njzjz njzjz added this to the v3.0.0 milestone Sep 26, 2024
@njzjz njzjz mentioned this issue Oct 18, 2024
Merged
github-merge-queue bot pushed a commit that referenced this issue Oct 23, 2024
@njzjz @pre-commit-ci
fix(pt): set weights_only=True for torch.load (#4147)
911f41b 
Fix #4143.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Enhanced model loading efficiency by only loading model weights, which
reduces memory usage and improves performance.

- **Bug Fixes**
- Streamlined the loading process across various components, ensuring
that only essential model parameters are loaded, thus optimizing the
overall functionality.

- **Tests**
- Updated tests to reflect the new loading behavior, ensuring that only
model weights are loaded in various test scenarios for improved clarity
and performance.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Jinzhe Zeng <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@github-project-automation github-project-automation bot moved this from mustfix to Done in DeePMD-kit V3.0.0 RC Oct 23, 2024
github-merge-queue bot pushed a commit that referenced this issue Oct 23, 2024
@njzjz
style: enable TorchFix in pre-commit (#4230)
0fa1b43 
Enable [TorchFix](https://github.com/pytorch-labs/torchfix).

Need to resolve the following issues before merging:
- #4229
- #4143

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
	- Integrated `flake8` for enhanced code quality checks.
	- Added `ruff-format` hook for improved linting.
- Implemented consistent license header insertion for various file
types.

- **Bug Fixes**
- Updated `check-added-large-files` hook to enforce a maximum file size
of 1024 KB.

- **Documentation**
	- Updated linting configuration with new rules under `[tool.flake8]`.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Jinzhe Zeng <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@njzjz njzjz

Labels
bug
Projects
Multiple backend support for DeePMD-kit
Archived in project
DeePMD-kit V3.0.0 RC
Status: Done
Milestone
v3.0.0
Development

Successfully merging a pull request may close this issue.

fix(pt): set weights_only=True for torch.load njzjz/deepmd-kit
2 participants
@njzjz @wanghan-iapcm