Add ImageConfig to the serialization context for dynamic task

[pingsutw]

Tracking issue

NA

Why are the changes needed?

Dynamic task tries to build the ImageSpec at runtime.

What changes were proposed in this pull request?

• Build the ImageSpec for the task entities that are inside the dynamic task
• Add ImageConfig to the serialization context for dynamic tasks at compile time.
• At runtime, the dynamic task will look up the image URL in the ImageConfig instead of rebuilding the image spec.

How was this patch tested?

remote cluster

Setup process

from click.testing import CliRunner

from flytekit import ImageSpec, task, dynamic, workflow
from flytekit.clis.sdk_in_container import pyflyte


new_flytekit = "git+https://github.com/flyteorg/flytekit.git@28d4da3a49c98a9f8dc93fef0945e22ca3cd208a"

repro_img = ImageSpec(
    builder="envd",
    packages=["mypy", new_flytekit],
    apt_packages=["git"],
    registry="pingsutw",
    source_root="."
)

image_foo = repro_img.with_apt_packages(["curl"])
image_d1 = repro_img.with_apt_packages(["wget"])
image_bar = repro_img.with_apt_packages(["vim"])


# hello
@task(container_image=image_foo)
def foo():
    print("foo")


@task(container_image=image_bar)
def bar():
    print("bar")


@dynamic(container_image=image_d1)
def d1():
    bar()


@workflow
def wf():
    foo()
    d1()

Screenshots

Also tested it on serverless

Check all the applicable boxes

• I updated the documentation accordingly.
• All new and existing tests passed.
• All commits are signed-off.

Related PRs

NA

Docs link

NA

[codecov]

Codecov Report

Attention: Patch coverage is 21.05263% with 15 lines in your changes missing coverage. Please review.

Project coverage is 41.91%. Comparing base (6d40413) to head (375db5c).
Report is 3 commits behind head on master.

❗ Current head 375db5c differs from pull request most recent head 6095194

Please upload reports for the commit 6095194 to get more accurate results.

Files	Patch %	Lines
flytekit/core/python_auto_container.py	12.50%	7 Missing ⚠️
flytekit/tools/translator.py	33.33%	6 Missing ⚠️
flytekit/core/container_task.py	0.00%	1 Missing ⚠️
flytekit/core/python_customized_container_task.py	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           master    #2456       +/-   ##
===========================================
- Coverage   76.07%   41.91%   -34.17%     
===========================================
  Files         182      182               
  Lines       18473    18484       +11     
  Branches     3825     3614      -211     
===========================================
- Hits        14053     7747     -6306     
- Misses       3809    10644     +6835     
+ Partials      611       93      -518     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[pingsutw]

cc @kumare3 @EngHabu

[wild-endeavor]

deduped