Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Return None type instead of exception #1907

Merged
merged 1 commit into from
Nov 27, 2023
Merged

Return None type instead of exception #1907

merged 1 commit into from
Nov 27, 2023

Conversation

honnix
Copy link
Member

@honnix honnix commented Oct 21, 2023 •
edited
Loading

TL;DR

Return None type instead of exception for list subtype

Type

  • Bug Fix
  • Feature
  • Plugin

Are all requirements met?

  • Code completed
  • Smoke tested
  • Unit tests added
  • Code documentation added
  • Any pending items have an associated Issue

Complete description

More details in flyteorg/flyte#4270

Tracking Issue

Fixes flyteorg/flyte#4270

Follow-up issue

NA

@honnix honnix mentioned this pull request Oct 21, 2023
Closed
2 tasks
@codecov
Copy link

codecov bot commented Oct 21, 2023 •
edited
Loading

Codecov Report

Attention: 4 lines in your changes are missing coverage. Please review.

Comparison is base (4d5e1b8) 54.71% compared to head (30a25f1) 54.71%.

Additional details and impacted files 
@@           Coverage Diff           @@
##           master    #1907   +/-   ##
=======================================
  Coverage   54.71%   54.71%           
=======================================
  Files         305      305           
  Lines       22732    22737    +5     
  Branches     3453     3455    +2     
=======================================
+ Hits        12438    12441    +3     
- Misses      10122    10123    +1     
- Partials      172      173    +1
Files Coverage Δ
flytekit/core/promise.py 27.94% <0.00%> (ø)
flytekit/core/type_engine.py 27.55% <50.00%> (+0.16%) ⬆️

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@honnix honnix force-pushed the none-instead-of-failure branch from 3a35541 to 06f46b2 Compare October 22, 2023 14:41
honnix
honnix commented Oct 22, 2023
View reviewed changes
flytekit/core/type_engine.py Outdated
@@ -1090,6 +1090,16 @@ def get_sub_type(t: Type[T]) -> Type[T]:

raise ValueError("Only generic univariate typing.List[T] type is supported.")

@staticmethod
def get_sub_type_or_none(t: Type[T]) -> Optional[Type[T]]:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I chose to add a new method because get_sub_type is used in many places where they assume it never returns None.

@honnix
Copy link
Member Author

honnix commented Oct 22, 2023

Not sure what happened to CI. The failure is not related to the change and it also happens to another PR.

@honnix honnix marked this pull request as ready for review October 22, 2023 15:38
kumare3
kumare3 reviewed Oct 22, 2023
View reviewed changes
Copy link
Contributor

@kumare3 kumare3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do not have support for list of unknown types- what does none mean?

@honnix honnix force-pushed the none-instead-of-failure branch from 06f46b2 to b076da1 Compare October 22, 2023 17:41
@honnix
Copy link
Member Author

honnix commented Oct 22, 2023

We do not have support for list of unknown types- what does none mean?

It is sort of unknown due to lack of python interface. The use case is

flytekit/flytekit/remote/remote_callable.py

Line 34 in 1b5692a

return create_and_link_node_from_remote(ctx, entity=self, **kwargs) # noqa
. DictTransformer handles this by returning None, so the type will be inferred by looking at the elements themselves

flytekit/flytekit/core/promise.py

Line 615 in 1b5692a

scalar = TypeEngine.to_literal(ctx, t_value, t_value_type or type(t_value), expected_literal_type).scalar

As flyteorg/flyte#4270 describes, binding currently does not work for list type in a remote entity.

@honnix honnix force-pushed the none-instead-of-failure branch from b076da1 to 30a25f1 Compare October 28, 2023 07:49
@honnix honnix requested a review from kumare3 October 28, 2023 16:00
@honnix
Copy link
Member Author

honnix commented Nov 17, 2023

Hi. Can someone take a look at this PR again? Thank you.

@pingsutw pingsutw merged commit dc32ab8 into master Nov 27, 2023
69 of 70 checks passed
@adrianloy
Copy link

Thanks for fixing this, was about to open a PR to address it. Nice to see its already on main!

RRap0so pushed a commit to RRap0so/flytekit that referenced this pull request Dec 15, 2023
@honnix @RRap0so
Handle list subtype that cannot be inferred (flyteorg#1907)
32042e7 
Signed-off-by: Hongxin Liang <[email protected]>
Signed-off-by: Rafael Raposo <[email protected]>
@honnix honnix deleted the none-instead-of-failure branch December 22, 2023 09:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pingsutw pingsutw pingsutw approved these changes

@wild-endeavor wild-endeavor Awaiting requested review from wild-endeavor wild-endeavor is a code owner

@eapolinario eapolinario Awaiting requested review from eapolinario eapolinario is a code owner

@cosmicBboy cosmicBboy Awaiting requested review from cosmicBboy cosmicBboy is a code owner

@kumare3 kumare3 Awaiting requested review from kumare3 kumare3 is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[BUG] List subtype check for remote task compilation does not work
4 participants
@honnix @adrianloy @kumare3 @pingsutw