Improve error message when omitting task kwarg in workflows

[fg91]

TL;DR

Improve error message when, in a workflow, one omits to pass an optional kwarg to a task.

---

In python, the following is perfectly valid:

def foo(bar: int=1) -> int:
    return bar

def wf() -> int:
    return foo()

In Flyte, the following workflow (which only adds task/workflow decorators) is not:

@task
def foo(bar: int=1) -> int:
    return bar


@workflow
def wf() -> int:
    return foo()

The user is confronted with this error message:

flytekit.exceptions.user.FlyteAssertion: Error encountered while executing 'wf':
  Input was not specified for: bar of type <FlyteLiteral simple: INTEGER>

Many Flyte users are ML engineers that are not necessarily aware of the following (source):

Although Flyte workflow syntax looks like Python code, it’s actually a domain-specific language (DSL) for building execution graphs where tasks – and other workflows – serve as the building blocks. This means that the workflow function body only supports a subset of Python’s semantics

For these users, the above error message is very confusing and does not give a hint why something that works in python wouldn't work in a flyte workflow.

---

Since I was asked about this particular limitation by several of our platform users, I propose that we improve the error message so that the error explains itself and users are informed about flyte workflows being a DSL, not python.

Type

Neither of the following but improved error message.

• Bug Fix
• Feature
• Plugin

Are all requirements met?

• Code completed
• Smoke tested
• Unit tests added
• Code documentation added
• Any pending items have an associated Issue

Complete description

NA

Tracking Issue

NA

Follow-up issue

NA

[codecov]

Codecov Report

Attention: 6 lines in your changes are missing coverage. Please review.

Comparison is base (f16ac49) 54.71% compared to head (c725eff) 54.78%.
Report is 3 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1921      +/-   ##
==========================================
+ Coverage   54.71%   54.78%   +0.06%     
==========================================
  Files         305      306       +1     
  Lines       22732    22782      +50     
  Branches     3453     3454       +1     
==========================================
+ Hits        12438    12481      +43     
- Misses      10122    10129       +7     
  Partials      172      172              

Files	Coverage Δ
flytekit/core/promise.py	27.68% <0.00%> (-0.27%)	⬇️

... and 3 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[fg91]

In the review, it would be good to think whether the if statement if isinstance(entity, Task) and _default is not None: is not covering cases where this error message should be added or does cover cases where it shouldn't be added.

I tested the following:

@task
def foo(bar: int = 1) -> int:  # default value that doesn't work
    return bar

@workflow
def wf() -> int:
    return foo()

flytekit.exceptions.user.FlyteAssertion: Error encountered while executing 'wf':
  Input bar of type <FlyteLiteral simple: INTEGER> was not specified for function test.foo.
  Flyte workflow syntax is a domain-specific language (DSL) for building execution graphs which supports a subset of Python’s semantics. When calling tasks, all kwargs have to be provided.

@task
def foo(bar: int) -> int:  # No default value
    return bar

@workflow
def wf() -> int:
    return foo()

flytekit.exceptions.user.FlyteAssertion: Error encountered while executing 'wf':
  Input bar of type <FlyteLiteral simple: INTEGER> was not specified for function test.foo

The issue is not a limitation of DSL vs python but actually no value provided for the kwarg.

@task
def foo(bar: int) -> int:
    return bar

@workflow
def sub_wf(bar: int=1) -> int:
    return foo(bar=bar)

@workflow
def wf() -> int:
    return sub_wf()

No error, as expected.

@task
def foo(bar: int) -> int:
    return bar

@workflow
def sub_wf(bar: int) -> int:   # no default value
    return foo(bar=bar)

@workflow
def wf() -> int:
    return sub_wf()

flytekit.exceptions.user.FlyteAssertion: Error encountered while executing 'wf':
  Encountered error while executing workflow 'sub_wf':
  Input bar of type <FlyteLiteral simple: INTEGER> was not specified for function sub_wf

5. Now with dynamic

@task
def foo(bar: int) -> int:
    return bar

@dynamic
def sub_wf(bar: int) -> int:  # no default value
    return foo(bar=bar)

@workflow
def wf() -> int:
    return sub_wf()

flytekit.exceptions.user.FlyteAssertion: Error encountered while executing 'wf':
  Input bar of type <FlyteLiteral simple: INTEGER> was not specified for function test.sub_wf

@task
def foo(bar: int) -> int:
    return bar

@dynamic
def sub_wf(bar: int=1) -> int:   # default value that doesn't work
    return foo(bar=bar)

@workflow
def wf() -> int:
    return sub_wf()

flytekit.exceptions.user.FlyteAssertion: Error encountered while executing 'wf':
  Input bar of type <FlyteLiteral simple: INTEGER> was not specified for function test.sub_wf. 
  Flyte workflow syntax is a domain-specific language (DSL) for building execution graphs which supports a subset of Python’s semantics. When calling tasks, all kwargs have to be provided.

[pingsutw]

would it be better to show the python type in the error message? like

Input bar of type [int] was not specified for function test.

we can get the python type from interface.inputs[k]

[fg91]

Good call, thanks!

c725eff

[pingsutw]

Thanks, LGTM