Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Single transforms config #109

Merged
merged 9 commits into from
May 18, 2023
Merged

Single transforms config #109

merged 9 commits into from
May 18, 2023

Conversation

mikeknep
Copy link
Contributor

Adds a new method, train_transforms, that accepts a single Transforms model config and applies it to all tables. This method accepts an optional list of tables passed as either only or ignore to limit which tables are trained. (These arguments match the optional list arguments on the Relational Trainer Connector's extract method.)

Users can include the name of a table as it appears in their db in the only|ignore argument and we will automatically extend that "request" to tables we invented from that source table (due to JSON). For example, if a purchases table had nested JSON and we ended up inventing 3 tables when we normalized that JSON data, the user can just pass the name purchases to explicitly include/exclude that table—they don't need to know the exact names of the tables we created under the hood.

This method will eventually fully replace train_transform_models, which is marked for deprecation here.

While here, updated a few of the type hints to the new style.

Copy link
Contributor

@pimlock pimlock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

tests/relational/test_train_transforms.py Show resolved Hide resolved
@mikeknep mikeknep force-pushed the single-transforms-config branch from 8572c71 to ced662f Compare May 18, 2023 19:44
@mikeknep mikeknep merged commit 8ac8209 into main May 18, 2023
@mikeknep mikeknep deleted the single-transforms-config branch May 18, 2023 19:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants