breaking: pt: remove data stat from model init #3233
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
amcadmus
changed the title
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## devel #3233 +/- ##
==========================================
+ Coverage 0 74.92% +74.92%
==========================================
Files 0 332 +332
Lines 0 28006 +28006
Branches 0 908 +908
==========================================
+ Hits 0 20983 +20983
- Misses 0 6486 +6486
- Partials 0 537 +537 ☔ View full report in Codecov by Sentry. |
njzjz
reviewed
Feb 5, 2024
Comment on lines
58
to
74
| @classmethod | ||
| def get_stat_name(cls, config): | ||
| """Get the name for the statistic file of the descriptor.""" | ||
| descrpt_type = config["type"] | ||
| return Descriptor.__plugins.plugins[descrpt_type].get_stat_name(config) | ||
|
|
||
| @classmethod | ||
| def get_data_stat_key(cls, config): | ||
| """Get the keys for the data statistic of the descriptor.""" | ||
| descrpt_type = config["type"] | ||
| return Descriptor.__plugins.plugins[descrpt_type].get_data_stat_key(config) | ||
|
|
||
| @classmethod | ||
| def get_data_process_key(cls, config): | ||
| """Get the keys for the data preprocess.""" | ||
| descrpt_type = config["type"] | ||
| return Descriptor.__plugins.plugins[descrpt_type].get_data_process_key(config) |
There was a problem hiding this comment.
If a subclass doesn't implement these subclasses, the program will stuck!
There was a problem hiding this comment.
I suggest adding the error message for this case:
Suggested change
| @classmethod | |
| def get_stat_name(cls, config): | |
| """Get the name for the statistic file of the descriptor.""" | |
| descrpt_type = config["type"] | |
| return Descriptor.__plugins.plugins[descrpt_type].get_stat_name(config) | |
| @classmethod | |
| def get_data_stat_key(cls, config): | |
| """Get the keys for the data statistic of the descriptor.""" | |
| descrpt_type = config["type"] | |
| return Descriptor.__plugins.plugins[descrpt_type].get_data_stat_key(config) | |
| @classmethod | |
| def get_data_process_key(cls, config): | |
| """Get the keys for the data preprocess.""" | |
| descrpt_type = config["type"] | |
| return Descriptor.__plugins.plugins[descrpt_type].get_data_process_key(config) | |
| @classmethod | |
| def get_stat_name(cls, config): | |
| """Get the name for the statistic file of the descriptor.""" | |
| if cls is not Descriptor: | |
| raise NotImplementedError("get_stat_name is not implemented!") | |
| descrpt_type = config["type"] | |
| return Descriptor.__plugins.plugins[descrpt_type].get_stat_name(config) | |
| @classmethod | |
| def get_data_stat_key(cls, config): | |
| """Get the keys for the data statistic of the descriptor.""" | |
| if cls is not Descriptor: | |
| raise NotImplementedError("get_data_stat_key is not implemented!") | |
| descrpt_type = config["type"] | |
| return Descriptor.__plugins.plugins[descrpt_type].get_data_stat_key(config) | |
| @classmethod | |
| def get_data_process_key(cls, config): | |
| """Get the keys for the data preprocess.""" | |
| if cls is not Descriptor: | |
| raise NotImplementedError("get_data_process_key is not implemented!") | |
| descrpt_type = config["type"] | |
| return Descriptor.__plugins.plugins[descrpt_type].get_data_process_key(config) |
wanghan-iapcm
requested changes
Feb 6, 2024
Comment on lines
+37
to
+42
| "EnvMat", | ||
| "NativeLayer", | ||
| "NativeNet", | ||
| "EmbeddingNet", | ||
| "FittingNet", | ||
| "DescrptSeA", |
There was a problem hiding this comment.
Any reason of providing all the classes here?
| descrpt_type = config["type"] | ||
| return Descriptor.__plugins.plugins[descrpt_type].get_stat_name(config) | ||
|
|
||
| @classmethod | ||
| def get_data_stat_key(cls, config): |
There was a problem hiding this comment.
not a good idea to pass a dict at interface. clearly write what does the method need.
| @classmethod | ||
| def get_data_process_key(cls, config): | ||
| """Get the keys for the data preprocess.""" |
There was a problem hiding this comment.
one would not understand the method from such a doc str.
| descrpt_type = config["type"] | ||
| return Descriptor.__plugins.plugins[descrpt_type].get_stat_name(config) | ||
|
|
||
| @classmethod | ||
| def get_data_stat_key(cls, config): | ||
| """Get the keys for the data statistic of the descriptor.""" |
There was a problem hiding this comment.
one would not understand the method from such a doc str.
| if sampled is not None: # compute stat | ||
| for sys in sampled: | ||
| dict_to_device(sys) | ||
| sumr, suma, sumn, sumr2, suma2 = self.descriptor.compute_input_stats( |
There was a problem hiding this comment.
not all the models have the data member self.descriptor, e.g. the pair_table atomic model.
we should mv the method out of the base model.
The same is for fitting. but not that the atomic_model may need the output stat.
There was a problem hiding this comment.
you have a lot of changes on the file. please tell what have you done
iProzd
added a commit
to iProzd/deepmd-kit
that referenced
this pull request
Feb 7, 2024
Restore deepmodeling#3233 with resolved conflicts and conversations.
|
See #3245 for further information. |
wanghan-iapcm
pushed a commit
that referenced
this pull request
Feb 10, 2024
Restore #3233 with resolved conflicts and conversations. This PR clean up the data stat process from model init. Please note that this code PR is just an initial cleanup and refinement of the data stat, and a more detailed design of the data stat will be completed in the next PR: - independent data stat from dataloader - data stat support for hybrid descriptors --------- Signed-off-by: Duo <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
njzjz
pushed a commit
to njzjz/deepmd-kit
that referenced
this pull request
Feb 10, 2024
Restore deepmodeling#3233 with resolved conflicts and conversations. This PR clean up the data stat process from model init. Please note that this code PR is just an initial cleanup and refinement of the data stat, and a more detailed design of the data stat will be completed in the next PR: - independent data stat from dataloader - data stat support for hybrid descriptors --------- Signed-off-by: Duo <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR:
Please note that this code PR is just an initial cleanup and refinement of the data stat, and a more detailed design of the data stat will be completed in the next PR.