Pass additional fields to agent create

[noahjax]

Tracking issue

https://github.com/flyteorg/flyte/issues/

Why are the changes needed?

With the update to flytekit 1.11.0 (specifically this PR) and creation of AsyncAgentBase, output_prefix was removed from the create function for agents. This is problematic for our use case, and in general seems like it is moving things towards less flexible agents.

We would also like to be able to consume the task execution metadata in our create function if possible, and I may tack that on to this PR later.

What changes were proposed in this pull request?

How was this patch tested?

Setup process

Screenshots

Check all the applicable boxes

• I updated the documentation accordingly.
• All new and existing tests passed.
• All commits are signed-off.

Related PRs

Docs link

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 83.36%. Comparing base (3642ec6) to head (38447ae).
Report is 1 commits behind head on master.

Files	Patch %	Lines
flytekit/extend/backend/agent_service.py	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2272      +/-   ##
==========================================
- Coverage   83.76%   83.36%   -0.41%     
==========================================
  Files         329      311      -18     
  Lines       25002    24193     -809     
  Branches     3708     3496     -212     
==========================================
- Hits        20943    20168     -775     
+ Misses       3432     3398      -34     
  Partials      627      627              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[noahjax]

@pingsutw At some point I'd like to also add TaskExecutionMetadata to agent create (more context here). Should I add that to this PR or address it in a followup?

[ddl-ebrown]

@pingsutw At some point I'd like to also add TaskExecutionMetadata to agent create (more context here). Should I add that to this PR or address it in a followup?

I would work on landing one thing at a time to make it easier to review / merge

[pingsutw]

@noahjax yes, you can add that to this PR.

[noahjax]

@pingsutw I'm actually not sure if there is an easy way to add TaskExecutionMetadata here...I don't think I can simply add another arg to AsyncAgentBase without updating all subclasses the way I did for output_prefix. Can we merge this smaller change and followup later on how to add this additional field?

[pingsutw]

Sure, either works for me

[noahjax]

This is what I initially tried for adding TaskExecutionMetadata.

I don't see a way to add it without making a breaking change to AsyncAgentBase.create. Is this ok if I update all of its implementations?

[noahjax]

@pingsutw Everything look ok to you?

[pingsutw]

LGTM, thank you.

[ddl-ebrown]

Approving based on the prior review from @pingsutw

[pingsutw]

LGTM, thank you @ddl-ebrown @noahjax