Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add image transformer #1901

Merged
merged 8 commits into from
Oct 19, 2023
Merged

Add image transformer #1901

merged 8 commits into from
Oct 19, 2023

Conversation

pingsutw
Copy link
Member

TL;DR

Add an image transformer, so the task can return PIL.Image.Image as an output,

Type

  • Bug Fix
  • Feature
  • Plugin

Are all requirements met?

  • Code completed
  • Smoke tested
  • Unit tests added
  • Code documentation added
  • Any pending items have an associated Issue

Complete description

My example:

import io
from typing_extensions import Annotated

import requests

from flytekit import task, workflow, ImageSpec
import PIL.Image

new_flytekit = "git+https://github.com/flyteorg/flytekit.git@da9f6645bb5e99580e38d89ef6824b81a1054279"
image_spec = ImageSpec(base_image="python:3.8-slim-buster", packages=[new_flytekit, "pillow"], apt_packages=["git"], registry="pingsutw")


@task(enable_deck=True, container_image=image_spec)
def t1() -> PIL.Image.Image:
    url = "https://miro.medium.com/v2/resize:fit:1400/1*0T9PjBnJB9H0Y4qrllkJtQ.png"
    response = requests.get(url)
    image_bytes = io.BytesIO(response.content)
    im = PIL.Image.open(image_bytes)
    return im


@workflow
def wf():
    t1()


if __name__ == "__main__":
    wf()
image

Tracking Issue

flyteorg/flyte#4193

Follow-up issue

NA

Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
flytekit/types/file/image.py Outdated Show resolved Hide resolved
flytekit/types/file/image.py Outdated Show resolved Hide resolved
flytekit/types/file/image.py Show resolved Hide resolved
@codecov
Copy link

codecov bot commented Oct 19, 2023

Codecov Report

Attention: 47 lines in your changes are missing coverage. Please review.

Comparison is base (8dc26d2) 55.20% compared to head (bfa4597) 55.09%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1901      +/-   ##
==========================================
- Coverage   55.20%   55.09%   -0.12%     
==========================================
  Files         300      301       +1     
  Lines       22399    22445      +46     
  Branches     3368     3370       +2     
==========================================
  Hits        12366    12366              
- Misses       9869     9914      +45     
- Partials      164      165       +1     
Files Coverage Δ
flytekit/experimental/eager_function.py 22.94% <ø> (ø)
...ins/flytekit-kf-pytorch/tests/test_elastic_task.py 100.00% <ø> (ø)
...gins/flytekit-mlflow/tests/test_mlflow_tracking.py 100.00% <100.00%> (ø)
...kit-deck-standard/flytekitplugins/deck/renderer.py 88.88% <0.00%> (ø)
flytekit/core/type_engine.py 27.77% <0.00%> (-0.06%) ⬇️
flytekit/types/file/image.py 0.00% <0.00%> (ø)

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Copy link
Collaborator

@eapolinario eapolinario left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think it's worth adding a test where we use Annotated[PIL.Image]?

from flytekit import task, workflow


@task(disable_deck=False)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: can you use enable_deck?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't find any other place using PIL.Image in flytkeit

Signed-off-by: Kevin Su <[email protected]>
@pingsutw pingsutw merged commit 85b8d29 into master Oct 19, 2023
68 of 70 checks passed
troychiu pushed a commit to troychiu/flytekit that referenced this pull request Oct 20, 2023
Future-Outlier pushed a commit to Future-Outlier/flytekit that referenced this pull request Oct 20, 2023
ringohoffman pushed a commit to ringohoffman/flytekit that referenced this pull request Nov 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants