Catch mistake in structured dataset #2834
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Yee Hing Tong <[email protected]>
wild-endeavor
requested review from
kumare3,
eapolinario,
pingsutw,
cosmicBboy,
samhita-alla,
thomasjpfan and
Future-Outlier
as code owners
wild-endeavor
changed the title
Signed-off-by: Yee Hing Tong <[email protected]>
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #2834 +/- ##
===========================================
- Coverage 95.61% 74.96% -20.66%
===========================================
Files 31 196 +165
Lines 1094 20380 +19286
Branches 0 2626 +2626
===========================================
+ Hits 1046 15277 +14231
- Misses 48 4397 +4349
- Partials 0 706 +706 ☔ View full report in Codecov by Sentry. |
granthamtaylor
added a commit
that referenced
this pull request
Oct 21, 2024
* Catch mistake in structured dataset (#2834) Signed-off-by: Yee Hing Tong <[email protected]> * Small change to clean up unit test. (#2835) * delete=true Signed-off-by: Yee Hing Tong <[email protected]> * close but don't delete on close Signed-off-by: Yee Hing Tong <[email protected]> * skip for 3.9 Signed-off-by: Yee Hing Tong <[email protected]> * 10 Signed-off-by: Yee Hing Tong <[email protected]> --------- Signed-off-by: Yee Hing Tong <[email protected]> * Fix tree printing (#2837) Signed-off-by: Yee Hing Tong <[email protected]> * handle case where error may not have args (#2831) Signed-off-by: Blake Jackson <[email protected]> Co-authored-by: Blake Jackson <[email protected]> * Bump pyspark from 3.3.1 to 3.3.2 in /plugins/flytekit-greatexpectations (#2818) Bumps [pyspark](https://github.com/apache/spark) from 3.3.1 to 3.3.2. - [Commits](apache/spark@v3.3.1...v3.3.2) --- updated-dependencies: - dependency-name: pyspark dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Pull secrets from environment when running locally (#2800) Signed-off-by: Thomas J. Fan <[email protected]> * Support executing launchplans from CLI (#2839) Signed-off-by: Ketan Umare <[email protected]> Co-authored-by: Ketan Umare <[email protected]> * Add top-level access to FlyteRemote, FlyteFile, and FlyteDirectory and convenience class methods for FlyteRemote (#2836) --------- Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: Blake Jackson <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Thomas J. Fan <[email protected]> Signed-off-by: Ketan Umare <[email protected]> Co-authored-by: Yee Hing Tong <[email protected]> Co-authored-by: Blake Jackson <[email protected]> Co-authored-by: Blake Jackson <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Thomas J. Fan <[email protected]> Co-authored-by: Ketan Umare <[email protected]> Co-authored-by: Ketan Umare <[email protected]>
granthamtaylor
added a commit
that referenced
this pull request
Oct 22, 2024
* Catch mistake in structured dataset (#2834) Signed-off-by: Yee Hing Tong <[email protected]> * Small change to clean up unit test. (#2835) * delete=true Signed-off-by: Yee Hing Tong <[email protected]> * close but don't delete on close Signed-off-by: Yee Hing Tong <[email protected]> * skip for 3.9 Signed-off-by: Yee Hing Tong <[email protected]> * 10 Signed-off-by: Yee Hing Tong <[email protected]> --------- Signed-off-by: Yee Hing Tong <[email protected]> * Fix tree printing (#2837) Signed-off-by: Yee Hing Tong <[email protected]> * handle case where error may not have args (#2831) Signed-off-by: Blake Jackson <[email protected]> Co-authored-by: Blake Jackson <[email protected]> * Bump pyspark from 3.3.1 to 3.3.2 in /plugins/flytekit-greatexpectations (#2818) Bumps [pyspark](https://github.com/apache/spark) from 3.3.1 to 3.3.2. - [Commits](apache/spark@v3.3.1...v3.3.2) --- updated-dependencies: - dependency-name: pyspark dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Pull secrets from environment when running locally (#2800) Signed-off-by: Thomas J. Fan <[email protected]> * Support executing launchplans from CLI (#2839) Signed-off-by: Ketan Umare <[email protected]> Co-authored-by: Ketan Umare <[email protected]> * Add top-level access to FlyteRemote, FlyteFile, and FlyteDirectory and convenience class methods for FlyteRemote (#2836) * Config for_endpoint doesn't respect config file (#2843) Signed-off-by: Yee Hing Tong <[email protected]> * Union/enum handling (#2845) * temp Signed-off-by: Yee Hing Tong <[email protected]> * delete test Signed-off-by: Yee Hing Tong <[email protected]> * allow strings Signed-off-by: Yee Hing Tong <[email protected]> * remove enum transformer change and tests Signed-off-by: Yee Hing Tong <[email protected]> * reset a test Signed-off-by: Yee Hing Tong <[email protected]> * lint Signed-off-by: Yee Hing Tong <[email protected]> * use .value for enums to handle dynamic case Signed-off-by: Yee Hing Tong <[email protected]> * use prior logic for 3.9 Signed-off-by: Yee Hing Tong <[email protected]> * add 3.9 testing Signed-off-by: Yee Hing Tong <[email protected]> * should always be a union type Signed-off-by: Yee Hing Tong <[email protected]> --------- Signed-off-by: Yee Hing Tong <[email protected]> * update docs for FlyteRemote (#2847) --------- Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: Blake Jackson <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Thomas J. Fan <[email protected]> Signed-off-by: Ketan Umare <[email protected]> Co-authored-by: Yee Hing Tong <[email protected]> Co-authored-by: Blake Jackson <[email protected]> Co-authored-by: Blake Jackson <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Thomas J. Fan <[email protected]> Co-authored-by: Ketan Umare <[email protected]> Co-authored-by: Ketan Umare <[email protected]>
kumare3
pushed a commit
that referenced
this pull request
Nov 8, 2024
Signed-off-by: Yee Hing Tong <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds an error message to try to catch a common mistake users make, when they request a transformer for the
StructuredDatasettype, but just pass in the raw dataframe.