Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix union transformer #2024

Merged
merged 4 commits into from
Dec 5, 2023
Merged

Fix union transformer #2024

merged 4 commits into from
Dec 5, 2023

Conversation

pingsutw
Copy link
Member

@pingsutw pingsutw commented Dec 5, 2023

Why are the changes needed?

Failed to convert a literal to python valueunion[dataclass, dataclass].

from dataclasses import dataclass
from typing import Union

from mashumaro.mixins.json import DataClassJSONMixin

from flytekitplugins.flyin import vscode
from flytekit import task


@dataclass
class Cat(DataClassJSONMixin):
    name: str
    age: int


@dataclass
class Yarn(DataClassJSONMixin):
    color: str
    length: int

@task
def t1(num: int) -> Union[Cat, Yarn]:
    return Cat(name="Mittens", age=num)


if __name__ == '__main__':
    print(t1(num=1))

What changes were proposed in this pull request?

When converting literal to python value, union transformer should catch any kind of exception.

except (TypeTransformerFailedError, AttributeError) as e:
logger.debug(f"Failed to convert from {lv} to {v}", e)
. Therefore, if one of transformers in the union fails, it will try next one.

How was this patch tested?

Run above workflow locally

Setup process

Screenshots

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Copy link

codecov bot commented Dec 5, 2023

Codecov Report

Attention: 3 lines in your changes are missing coverage. Please review.

Comparison is base (5c6802c) 86.22% compared to head (ba8205f) 86.21%.

Files Patch % Lines
flytekit/core/type_engine.py 83.33% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2024      +/-   ##
==========================================
- Coverage   86.22%   86.21%   -0.01%     
==========================================
  Files         320      320              
  Lines       23531    23540       +9     
  Branches     3464     3466       +2     
==========================================
+ Hits        20289    20296       +7     
- Misses       2650     2652       +2     
  Partials      592      592              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Kevin Su <[email protected]>
@@ -1455,12 +1456,15 @@ def to_literal(self, ctx: FlyteContext, python_val: T, python_type: Type[T], exp
res_type = _add_tag_to_type(trans.get_literal_type(t), trans.name)
if found_res:
# Should really never happen, sanity check
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you remove the comment?

kumare3
kumare3 previously approved these changes Dec 5, 2023
Signed-off-by: Kevin Su <[email protected]>
@eapolinario eapolinario merged commit 1e30eb1 into master Dec 5, 2023
74 of 76 checks passed
RRap0so pushed a commit to RRap0so/flytekit that referenced this pull request Dec 15, 2023
* Fix union transformer

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* fix test

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

---------

Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Rafael Raposo <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants