Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check for None before lazy loading #2435

Merged
merged 1 commit into from
May 22, 2024

Conversation

thomasjpfan
Copy link
Member

Tracking issue

Related to flyteorg/flyte#4877

Why are the changes needed?

From time to time, I get this error locally or remotely:

Traceback (most recent call last):
  File "/opt/conda/envs/envd/bin/pyflyte-execute", line 5, in <module>
    from flytekit.bin.entrypoint import execute_task_cmd
  File "/opt/conda/envs/envd/lib/python3.11/site-packages/flytekit/__init__.py", line 242, in <module>
    from flytekit.deck import Deck
  File "/opt/conda/envs/envd/lib/python3.11/site-packages/flytekit/deck/__init__.py", line 22, in <module>
    from .renderer import MarkdownRenderer, SourceCodeRenderer, TopFrameRenderer
  File "/opt/conda/envs/envd/lib/python3.11/site-packages/flytekit/deck/renderer.py", line 13, in <module>
    pandas = lazy_module("pandas")
             ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/envd/lib/python3.11/site-packages/flytekit/lazy_import/lazy_module.py", line 41, in lazy_module
    loader = importlib.util.LazyLoader(spec.loader)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib.util>", line 302, in __init__
  File "<frozen importlib.util>", line 293, in __check_eager_loader
TypeError: loader must define exec_module()

What changes were proposed in this pull request?

In some environments, importlib.util.find_spec returns a spec that does not have a loader.

Signed-off-by: Thomas J. Fan <[email protected]>
Copy link
Collaborator

@eapolinario eapolinario left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very interesting that this is non-deterministic (for a given package version). I wonder if it's worth logging that case also.

@thomasjpfan thomasjpfan merged commit d89181f into flyteorg:master May 22, 2024
45 of 46 checks passed
fiedlerNr9 pushed a commit that referenced this pull request Jul 25, 2024
Signed-off-by: Thomas J. Fan <[email protected]>
Signed-off-by: Jan Fiedler <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants