Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve error message for ImageSpec #2498

Merged
merged 5 commits into from
Jul 2, 2024
Merged

Conversation

pingsutw
Copy link
Member

Tracking issue

NA

Why are the changes needed?

To improve the error message

What changes were proposed in this pull request?

How was this patch tested?

Setup process

from flytekit import task, workflow, ImageSpec

scraper_image_spec3 = ImageSpec(
    base_image="python:3.11-slim-bookworm",
    packages=["pandas", "flytekit"],
    apt_packages=["vim"],
    registry="pingsutw",
)

@task(container_image=scraper_image_spec3)
def t3(a: int) -> int:
    return a + 2


@workflow
def wf(a: int = 3):
    t3(a=a)

Screenshots

Before:
image

After:

Screenshot 2024-06-18 at 2 22 33 PM
Screenshot 2024-06-18 at 2 02 33 PM
Screenshot 2024-06-18 at 2 03 06 PM

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Related PRs

NA

Docs link

NA

pingsutw added 2 commits June 18, 2024 14:07
Signed-off-by: Kevin Su <[email protected]>
pingsutw added 2 commits June 18, 2024 14:31
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
click.secho("Flytekit assumes that the image already exists.", fg="blue")
return True
click.secho(f"Failed to check if the image exists with error:\n {e}", fg="red")
click.secho(f"Flytekit assumes the image {self.image_name()} already exists.", fg="blue")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this print statement belongs in should_build instead.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense, I updated it, thanks!

return True

click.secho(f"Failed to check if the image exists with error:\n {e}", fg="red")
click.secho(f"Flytekit assumes the image {self.image_name()} already exists.", fg="blue")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@@ -107,6 +107,8 @@ def is_container(self) -> bool:
def exist(self) -> bool:
"""
Check if the image exists in the registry.
Return True if the image exists in the registry, False otherwise.
Return None if failed to check if the image exists due to the permission issue or other reasons.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That Python typing for exist needs to be updated to include None.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

@@ -153,9 +158,9 @@ def exist(self) -> bool:
f" pip install --upgrade docker"
)

click.secho(f"Failed to check if the image exists with error : {e}", fg="red")
click.secho("Flytekit assumes that the image already exists.", fg="blue")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we ensure we handle runtime error nicely in the clean exception in pyflyte

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

@@ -68,7 +80,7 @@ def execute_command(command: str):

if p.returncode != 0:
_, stderr = p.communicate()
raise Exception(f"failed to run command {command} with error {stderr}")
raise Exception(f"failed to run command {command} with error:\n {stderr.decode()}")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change this to raise assertionerror
Or runtime or specific error

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

Signed-off-by: Kevin Su <[email protected]>
Copy link

codecov bot commented Jun 28, 2024

Codecov Report

Attention: Patch coverage is 56.52174% with 10 lines in your changes missing coverage. Please review.

Project coverage is 78.14%. Comparing base (29662e7) to head (e146d88).
Report is 20 commits behind head on master.

Files Patch % Lines
flytekit/image_spec/image_spec.py 46.15% 6 Missing and 1 partial ⚠️
...lytekit-envd/flytekitplugins/envd/image_builder.py 70.00% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2498      +/-   ##
==========================================
+ Coverage   71.79%   78.14%   +6.34%     
==========================================
  Files         182      185       +3     
  Lines       18561    18737     +176     
  Branches     3654     3662       +8     
==========================================
+ Hits        13326    14642    +1316     
+ Misses       4592     3471    -1121     
+ Partials      643      624      -19     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@pingsutw pingsutw merged commit bd01c3d into master Jul 2, 2024
46 of 48 checks passed
bgedik pushed a commit to bgedik/flytekit that referenced this pull request Jul 3, 2024
fiedlerNr9 pushed a commit that referenced this pull request Jul 25, 2024
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Jan Fiedler <[email protected]>
mao3267 pushed a commit to mao3267/flytekit that referenced this pull request Jul 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants