Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[flytepropeller] Support attribute access on promise #4232

Merged
merged 4 commits into from
Oct 19, 2023

Conversation

ByronHsu
Copy link
Contributor

@ByronHsu ByronHsu commented Oct 15, 2023

Tracking issue

#3864

Describe your changes

See flyteorg/flyteidl#439 (comment)

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Screenshots

Note to reviewers

@codecov
Copy link

codecov bot commented Oct 15, 2023

Codecov Report

Attention: 35 lines in your changes are missing coverage. Please review.

Comparison is base (a114769) 59.06% compared to head (21bb73f) 60.02%.
Report is 1 commits behind head on master.

❗ Current head 21bb73f differs from pull request most recent head 1b5fe18. Consider uploading reports for the commit 1b5fe18 to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4232      +/-   ##
==========================================
+ Coverage   59.06%   60.02%   +0.96%     
==========================================
  Files         621      571      -50     
  Lines       53105    41337   -11768     
==========================================
- Hits        31365    24812    -6553     
+ Misses      19240    14123    -5117     
+ Partials     2500     2402      -98     
Flag Coverage Δ
unittests ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
...ugins/go/tasks/plugins/webapi/databricks/plugin.go 65.82% <100.00%> (+4.48%) ⬆️
flytepropeller/pkg/controller/nodes/resolve.go 88.05% <100.00%> (+0.71%) ⬆️
flytepropeller/pkg/compiler/validators/bindings.go 71.87% <76.92%> (+1.08%) ⬆️
...epropeller/pkg/controller/nodes/output_resolver.go 47.54% <60.00%> (+1.82%) ⬆️
...tepropeller/pkg/compiler/errors/compiler_errors.go 65.81% <0.00%> (-2.58%) ⬇️
...opeller/pkg/controller/nodes/attr_path_resolver.go 76.84% <76.84%> (ø)

... and 571 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ByronHsu ByronHsu changed the title Byhsu/promise attr path/3 [flytepropeller] Support attribute access on promise - Add dataclass_type proto Oct 15, 2023
@ByronHsu ByronHsu changed the title [flytepropeller] Support attribute access on promise - Add dataclass_type proto [flytepropeller] Support attribute access on promise Oct 15, 2023
Copy link
Contributor

@hamersaw hamersaw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few nit comments and failing checks on lint and generate. Otherwise this looks great! This is going to be so powerful.

flytepropeller/pkg/compiler/validators/bindings.go Outdated Show resolved Hide resolved
@ByronHsu ByronHsu force-pushed the byhsu/promise-attr-path/3 branch from fa23167 to 1b5fe18 Compare October 17, 2023 22:31
Copy link
Contributor

@hamersaw hamersaw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Will wait for one more approve before merging to try and stick with old idl requirements.


switch obj := obj.(type) {
case map[string]interface{}:
newSt, err := structpb.NewStruct(obj)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just one question. if you just have a typing.Dict[str, DataClass], and you bind to one of the dataclasses. you end up in this condition right?

What is this line for? newSt, err := structpb.NewStruct(obj)

In fact in this case - line 22 above:
tmpVal, exist = currVal.GetMap().GetLiterals()[attr.GetStringValue()]
won't tmpVal already be a Literal? can't you just return it? it won't get returned right now, it'll fall into the condition on line 41.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i guess you are talking about the case that where bindAttrPath[count:] is 0 and the currVal on line41 is a datatclass. Currently we convert convert struct to map, and convert it back to struct without doing other things because line 63 the loop will be skipped.

adding a check about len(bindAttrPath[count:]) == 0 on line 41 can circumvent the unnecessary work, but I think it's ok to keep it as-is because it should not be a large overhead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants