Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] set_prefix doesn't work for both TF and PT #3477

Closed
njzjz opened this issue Mar 17, 2024 · 1 comment · Fixed by #3753
Closed

[BUG] set_prefix doesn't work for both TF and PT #3477

njzjz opened this issue Mar 17, 2024 · 1 comment · Fixed by #3753
Assignees
Labels
Milestone

Comments

@njzjz
Copy link
Member

njzjz commented Mar 17, 2024

Bug summary

set_prefix cannot be changed to other values except set in both TensorFlow and PyTorch backends.

We should remove such an option, or allow it to exist for compatibility but disallow other options and update the documentation.

DeePMD-kit Version

v3.0.0a0-28-ged831c88

TensorFlow Version

PT v2.2.0+cu121-g8ac9b20d4b0

How did you download the software?

Built from source

Input Files, Running Commands, Error Log, etc.

The get_data method doesn't have this argument, as shown below.

Traceback (most recent call last):
  File "/home/jz748/anaconda3/bin/dp", line 8, in <module>
    sys.exit(main())
  File "/home/jz748/codes/deepmd-kit/deepmd/main.py", line 807, in main
    deepmd_main(args)
  File "/home/jz748/codes/deepmd-kit/deepmd/tf/entrypoints/main.py", line 72, in main
    train_dp(**dict_args)
  File "/home/jz748/codes/deepmd-kit/deepmd/tf/entrypoints/train.py", line 153, in train
    _do_work(jdata, run_opt, is_compress)
  File "/home/jz748/codes/deepmd-kit/deepmd/tf/entrypoints/train.py", line 205, in _do_work
    train_data = get_data(
  File "/home/jz748/codes/deepmd-kit/deepmd/utils/data_system.py", line 798, in get_data
    data = DeepmdDataSystem(
  File "/home/jz748/codes/deepmd-kit/deepmd/utils/data_system.py", line 112, in __init__
    DeepmdData(
  File "/home/jz748/codes/deepmd-kit/deepmd/utils/data.py", line 66, in __init__
    raise FileNotFoundError(f"No {set_prefix}.* is found in {sys_path}")
FileNotFoundError: No set.* is found in ../data/data_0/

Steps to Reproduce

cd examples/water/data

Change the prefix of all training data from set to haha.

Then go to

cd examples/water/se_atten

Add set_prefix option:

diff --git a/examples/water/se_atten/input.json b/examples/water/se_atten/input.json
index bfd6d48c..e3362fdd 100644
--- a/examples/water/se_atten/input.json
+++ b/examples/water/se_atten/input.json
@@ -66,6 +66,7 @@
         "../data/data_2/"
       ],
       "batch_size": "auto",
+         "set_prefix": "haha",
       "_comment7": "that's all"
     },
     "validation_data": {
@@ -74,6 +75,7 @@
       ],
       "batch_size": 1,
       "numb_btch": 3,
+         "set_prefix": "haha",
       "_comment8": "that's all"
     },
     "numb_steps": 1000000,
dp train input.json

Further Information, Files, and Links

No response

@iProzd
Copy link
Collaborator

iProzd commented May 1, 2024

We will remove support for set_prefix.

njzjz added a commit to njzjz/deepmd-kit that referenced this issue May 6, 2024
Fix deepmodeling#3477.

Deprecate `set_prefix` in the training parameter and `--set-prefix` in CLI. It has been broken since several years ago, and dpdata only supports `set`.

Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz njzjz linked a pull request May 6, 2024 that will close this issue
njzjz added a commit to njzjz/dpgen that referenced this issue May 6, 2024
njzjz added a commit to njzjz/dpgen2 that referenced this issue May 6, 2024
njzjz added a commit to njzjz/tutorials that referenced this issue May 6, 2024
wanghan-iapcm pushed a commit to deepmodeling/dpgen2 that referenced this issue May 7, 2024
See:
- deepmodeling/deepmd-kit#3477
- deepmodeling/deepmd-kit#3753

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **Refactor**
	- Removed an unused configuration parameter from the training settings.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
wanghan-iapcm added a commit to deepmodeling/dpgen that referenced this issue May 7, 2024
See:
- deepmodeling/deepmd-kit#3477
- deepmodeling/deepmd-kit#3753

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **Refactor**
- Removed the `"set_prefix": "set"` attribute across various JSON and
YAML configuration files, streamlining data handling and batch
processing setups.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Han Wang <[email protected]>
github-merge-queue bot pushed a commit that referenced this issue May 7, 2024
Fix #3477.

Deprecate `set_prefix` in the training parameter and `--set-prefix` in
CLI. It has been broken since several years ago, and dpdata only
supports `set`.

---------

Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz njzjz closed this as completed May 7, 2024
@github-project-automation github-project-automation bot moved this from Backlog to Done in DeePMD-3.0.0 beta release May 7, 2024
njzjz added a commit to deepmodeling/tutorials that referenced this issue Aug 9, 2024
mtaillefumier pushed a commit to mtaillefumier/deepmd-kit that referenced this issue Sep 18, 2024
Fix deepmodeling#3477.

Deprecate `set_prefix` in the training parameter and `--set-prefix` in
CLI. It has been broken since several years ago, and dpdata only
supports `set`.

---------

Signed-off-by: Jinzhe Zeng <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

2 participants