Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add utility methods #965

Merged
merged 3 commits into from
Aug 31, 2022
Merged

Add utility methods #965

merged 3 commits into from
Aug 31, 2022

Conversation

fealho
Copy link
Member

@fealho fealho commented Aug 19, 2022

Resolve #948.

@fealho fealho force-pushed the issue-948-add-utilities branch from 860ac28 to 4882d0f Compare August 20, 2022 21:02
@codecov-commenter
Copy link

codecov-commenter commented Aug 20, 2022

Codecov Report

Merging #965 (bf920e3) into V1.0.0.dev (7414df2) will increase coverage by 0.12%.
The diff coverage is 90.00%.

@@              Coverage Diff               @@
##           V1.0.0.dev     #965      +/-   ##
==============================================
+ Coverage       74.51%   74.64%   +0.12%     
==============================================
  Files              42       42              
  Lines            3547     3577      +30     
==============================================
+ Hits             2643     2670      +27     
- Misses            904      907       +3     
Impacted Files Coverage Δ
sdv/data_processing/data_processor.py 92.50% <86.36%> (-7.50%) ⬇️
sdv/metadata/single_table.py 99.14% <100.00%> (+0.03%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@fealho fealho marked this pull request as ready for review August 20, 2022 22:37
@fealho fealho requested a review from a team as a code owner August 20, 2022 22:37
@fealho fealho requested review from amontanez24 and removed request for a team August 20, 2022 22:37
"""
return {
'metadata': copy.deepcopy(self.metadata.to_dict()),
'constraints_to_reverse': copy.deepcopy(self._constraints_to_reverse),
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This implementation assumes constraints_to_reverse is a list of constraint_dicts. If we decide not to do so, this needs to be revised.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_constraints_to_reverse will actually be a list of constraint instances. When this class gets initialized it creates all the constraint instances from the metadata and then adds specific ones to this list when needed

dict:
Dict representation of this DataProcessor.
"""
return {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm only passing metadata, constraints_to_reverse, model_kwargs because:

  1. _constraints are derived from metadata during the init (so they are not needed).
  2. _transformers_by_sdtype cannot be fully transformed into dict, since RDT transformers don't have to_dict/from_dict methods.

}

@classmethod
def from_dict(cls, metadata_dict, learn_rounding_scheme=True, enforce_min_max_values=True):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since to_dict doesn't provide _transformers_by_sdtype, we have to pass learn_rounding_scheme and enforce_min_max_values to derive it from scratch.

Copy link
Contributor

@amontanez24 amontanez24 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a comment about the _constraints_to_reverse. Everything else looks good

"""
return {
'metadata': copy.deepcopy(self.metadata.to_dict()),
'constraints_to_reverse': copy.deepcopy(self._constraints_to_reverse),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_constraints_to_reverse will actually be a list of constraint instances. When this class gets initialized it creates all the constraint instances from the metadata and then adds specific ones to this list when needed

enforce_min_max_values=enforce_min_max_values,
model_kwargs=metadata_dict.get('model_kwargs')
)
instance._constraints_to_reverse = metadata_dict.get('constraints_to_reverse', [])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will have to change as well

@fealho fealho requested a review from amontanez24 August 30, 2022 12:10
Copy link
Contributor

@amontanez24 amontanez24 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One small comment, but not a big deal. :shipit:

@@ -97,9 +97,10 @@ def to_dict(self):
dict:
Dict representation of this DataProcessor.
"""
constraints_to_reverse = [cnt.to_dict() for cnt in self._constraints_to_reverse]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor: does it not fit to do constraint instead of cnt. In general I think we should try to use full descriptive names

@fealho fealho merged commit e08fc54 into V1.0.0.dev Aug 31, 2022
@fealho fealho deleted the issue-948-add-utilities branch August 31, 2022 11:39
amontanez24 pushed a commit that referenced this pull request Sep 13, 2022
* Add tests + methods

* Update to_json/from_json + test cases

* Address feedback
amontanez24 pushed a commit that referenced this pull request Nov 22, 2022
* Add tests + methods

* Update to_json/from_json + test cases

* Address feedback
amontanez24 pushed a commit that referenced this pull request Dec 21, 2022
* Add tests + methods

* Update to_json/from_json + test cases

* Address feedback
amontanez24 pushed a commit that referenced this pull request Feb 17, 2023
* Add tests + methods

* Update to_json/from_json + test cases

* Address feedback
amontanez24 pushed a commit that referenced this pull request Mar 23, 2023
* Add tests + methods

* Update to_json/from_json + test cases

* Address feedback
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants