Skip to content

Commit

Permalink
[Refactor] Rename flyin to flyteinteractive (#2157)
Browse files Browse the repository at this point in the history
Signed-off-by: Chi-Sheng Liu <[email protected]>
Signed-off-by: Future-Outlier <[email protected]>
Co-authored-by: Han-Ru Chen <[email protected]>
  • Loading branch information
MortalHappiness and Han-Ru Chen authored Feb 5, 2024
1 parent 95fb8fa commit b744a11
Show file tree
Hide file tree
Showing 22 changed files with 45 additions and 45 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/pythonbuild.yml
Original file line number Diff line number Diff line change
Expand Up @@ -221,7 +221,7 @@ jobs:
- flytekit-spark
- flytekit-sqlalchemy
- flytekit-vaex
- flytekit-flyin
- flytekit-flyteinteractive
- flytekit-whylogs
exclude:
# flytekit-modin depends on ray which does not have a 3.11 wheel yet.
Expand Down
18 changes: 9 additions & 9 deletions .github/workflows/pythonpublish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -227,7 +227,7 @@ jobs:
cache-from: type=gha
cache-to: type=gha,mode=max

build-and-push-flyin-images:
build-and-push-flyteinteractive-images:
runs-on: ubuntu-latest
needs: deploy
steps:
Expand All @@ -246,25 +246,25 @@ jobs:
registry: ghcr.io
username: "${{ secrets.FLYTE_BOT_USERNAME }}"
password: "${{ secrets.FLYTE_BOT_PAT }}"
- name: Prepare flyin Image Names
id: flyin-names
- name: Prepare flyteinteractive Image Names
id: flyteinteractive-names
uses: docker/metadata-action@v3
with:
images: |
ghcr.io/${{ github.repository_owner }}/flytekit
tags: |
flyin-latest
flyin-${{ github.sha }}
flyin-${{ needs.deploy.outputs.version }}
flyteinteractive-latest
flyteinteractive-${{ github.sha }}
flyteinteractive-${{ needs.deploy.outputs.version }}
- name: Push Flyin Image to GitHub Registry
uses: docker/build-push-action@v2
with:
context: "./plugins/flytekit-flyin/"
context: "./plugins/flytekit-flyteinteractive/"
platforms: linux/arm64, linux/amd64
push: ${{ github.event_name == 'release' }}
tags: ${{ steps.flyin-names.outputs.tags }}
tags: ${{ steps.flyteinteractive-names.outputs.tags }}
build-args: |
VERSION=${{ needs.deploy.outputs.version }}
file: ./plugins/flytekit-flyin/Dockerfile
file: ./plugins/flytekit-flyteinteractive/Dockerfile
cache-from: type=gha
cache-to: type=gha,mode=max
2 changes: 1 addition & 1 deletion Dockerfile.dev
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ RUN apt-get update && apt-get install build-essential vim libmagic1 git -y \
-e /flytekit \
-e /flytekit/plugins/flytekit-k8s-pod \
-e /flytekit/plugins/flytekit-deck-standard \
-e /flytekit/plugins/flytekit-flyin \
-e /flytekit/plugins/flytekit-flyteinteractive \
scikit-learn \
&& apt-get clean autoclean \
&& apt-get autoremove --yes \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ ARG TARGETARCH
# 1. Update the necessary packages for flytekit
# 2. Install code-server
# 3. Download code-server extensions for Python and Jupyter via wget
# 4. Install flytekit and flytekit-flyin with no cache
# 4. Install flytekit and flytekit-flyteinteractive with no cache
# 5. Delete apt cache. Reference: https://gist.github.com/marvell/7c812736565928e602c4
# 6. Some packages will create config file under /home by default, so we need to make sure it's writable
# 7. Change the permission of /tmp, so that others can run command on it
Expand All @@ -22,7 +22,7 @@ RUN apt-get update \
&& tar -xzf /tmp/code-server/code-server-4.19.0-linux-${TARGETARCH}.tar.gz -C /tmp/code-server/ \
&& wget --no-check-certificate https://open-vsx.org/api/ms-python/python/2023.20.0/file/ms-python.python-2023.20.0.vsix -P /tmp/code-server \
&& wget --no-check-certificate https://open-vsx.org/api/ms-toolsai/jupyter/2023.9.100/file/ms-toolsai.jupyter-2023.9.100.vsix -P /tmp/code-server \
&& pip install --no-cache-dir -U flytekitplugins-flyin==$VERSION flytekit==$VERSION \
&& pip install --no-cache-dir -U flytekitplugins-flyteinteractive==$VERSION flytekit==$VERSION \
&& apt-get clean autoclean \
&& apt-get autoremove --yes \
&& rm -rf /var/lib/{apt,dpkg,cache,log}/ \
Expand Down
Original file line number Diff line number Diff line change
@@ -1,26 +1,26 @@
# Flytekit FlyIn Plugin
# Flytekit FlyteInteractive Plugin

> Flyin = Flyte Interactive = Develop Flyte with the speed of flying
> FlyteInteractive = Develop Flyte with the speed of flying
FlyIn plugin provides users' favorite interface to develop and debug a flyte task interactively. We support vscode, jupyter (WIP), and neovim (WIP).
FlyteInteractive plugin provides users' favorite interface to develop and debug a flyte task interactively. We support vscode, jupyter (WIP), and neovim (WIP).

## Installation

To install the plugin, run the following command:

```bash
pip install flytekitplugins-flyin
pip install flytekitplugins-flyteinteractive
```

## Vscode

FlyIn Vscode offers an easy solution for users to run Python tasks within an interactive VSCode server, compatible with any image. `@vscode` is a decorator which users can put within @task and user function. With `@vscode`, the task will install vscode dependencies (skip if they already exist) and run a vscode server instead of the user defined functions.
FlyteInteractive Vscode offers an easy solution for users to run Python tasks within an interactive VSCode server, compatible with any image. `@vscode` is a decorator which users can put within @task and user function. With `@vscode`, the task will install vscode dependencies (skip if they already exist) and run a vscode server instead of the user defined functions.


## Starter Example
```python
from flytekit import task
from flytekitplugins.flyin import vscode
from flytekitplugins.flyteinteractive import vscode

@task
@vscode
Expand Down Expand Up @@ -53,7 +53,7 @@ If users want to skip the vscode downloading process at runtime, they have the o

```python
from flytekit import task, workflow
from flytekitplugins.flyin import vscode, VscodeConfig, DEFAULT_CODE_SERVER_EXTENSIONS
from flytekitplugins.flyteinteractive import vscode, VscodeConfig, DEFAULT_CODE_SERVER_EXTENSIONS

@task(
container_image="localhost:30000/flytekit-vscode:0.0.2",
Expand Down
File renamed without changes
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
"""
.. currentmodule:: flytekitplugins.flyin
.. currentmodule:: flytekitplugins.flyteinteractive
This package contains flyin plugin for Flytekit.
This package contains flyteinteractive plugin for Flytekit.
.. autosummary::
:template: custom.rst
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,11 @@
# https://coder.com/docs/code-server/latest/FAQ#what-is-the-heartbeat-file
HEARTBEAT_PATH = os.path.expanduser("~/.local/share/code-server/heartbeat")

INTERACTIVE_DEBUGGING_FILE_NAME = "flyin_interactive_entrypoint.py"
RESUME_TASK_FILE_NAME = "flyin_resume_task.py"
INTERACTIVE_DEBUGGING_FILE_NAME = "flyteinteractive_interactive_entrypoint.py"
RESUME_TASK_FILE_NAME = "flyteinteractive_resume_task.py"
# Config keys to store in task template
VSCODE_TYPE_KEY = "flyin_type"
VSCODE_PORT_KEY = "flyin_port"
VSCODE_TYPE_KEY = "flyteinteractive_type"
VSCODE_PORT_KEY = "flyteinteractive_port"

# Context attribute name of the task function's source file path
TASK_FUNCTION_SOURCE_PATH = "TASK_FUNCTION_SOURCE_PATH"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
from typing import Callable, List, Optional

import fsspec
from flytekitplugins.flyin.utils import load_module_from_path
from flytekitplugins.flyteinteractive.utils import load_module_from_path

import flytekit
from flytekit.core.context_manager import FlyteContextManager
Expand Down Expand Up @@ -267,10 +267,10 @@ def prepare_interactive_python(task_function):

# Generate a Python script
task_module_name, task_name = task_function.__module__, task_function.__name__
python_script = f"""# This file is auto-generated by flyin
python_script = f"""# This file is auto-generated by flyteinteractive
from {task_module_name} import {task_name}
from flytekitplugins.flyin import get_task_inputs
from flytekitplugins.flyteinteractive import get_task_inputs
if __name__ == "__main__":
inputs = get_task_inputs(
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
from setuptools import setup

PLUGIN_NAME = "flyin"
PLUGIN_NAME = "flyteinteractive"

microlib_name = f"flytekitplugins-{PLUGIN_NAME}"

Expand All @@ -13,7 +13,7 @@
version=__version__,
author="flyteorg",
author_email="[email protected]",
description="This package holds the flyin plugins for flytekit",
description="This package holds the flyteinteractive plugins for flytekit",
namespace_packages=["flytekitplugins"],
packages=[
f"flytekitplugins.{PLUGIN_NAME}",
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

import mock
import pytest
from flytekitplugins.flyin import (
from flytekitplugins.flyteinteractive import (
CODE_TOGETHER_CONFIG,
CODE_TOGETHER_EXTENSION,
COPILOT_CONFIG,
Expand All @@ -16,8 +16,8 @@
jupyter,
vscode,
)
from flytekitplugins.flyin.vscode_lib.constants import EXIT_CODE_SUCCESS
from flytekitplugins.flyin.vscode_lib.decorator import (
from flytekitplugins.flyteinteractive.vscode_lib.constants import EXIT_CODE_SUCCESS
from flytekitplugins.flyteinteractive.vscode_lib.decorator import (
get_code_server_info,
get_installed_extensions,
is_extension_installed,
Expand Down Expand Up @@ -49,15 +49,15 @@ def mock_code_server_info_dict():
@pytest.fixture
def vscode_patches():
with mock.patch("multiprocessing.Process") as mock_process, mock.patch(
"flytekitplugins.flyin.vscode_lib.decorator.prepare_interactive_python"
"flytekitplugins.flyteinteractive.vscode_lib.decorator.prepare_interactive_python"
) as mock_prepare_interactive_python, mock.patch(
"flytekitplugins.flyin.vscode_lib.decorator.exit_handler"
"flytekitplugins.flyteinteractive.vscode_lib.decorator.exit_handler"
) as mock_exit_handler, mock.patch(
"flytekitplugins.flyin.vscode_lib.decorator.download_vscode"
"flytekitplugins.flyteinteractive.vscode_lib.decorator.download_vscode"
) as mock_download_vscode, mock.patch("signal.signal") as mock_signal, mock.patch(
"flytekitplugins.flyin.vscode_lib.decorator.prepare_resume_task_python"
"flytekitplugins.flyteinteractive.vscode_lib.decorator.prepare_resume_task_python"
) as mock_prepare_resume_task_python, mock.patch(
"flytekitplugins.flyin.vscode_lib.decorator.prepare_launch_json"
"flytekitplugins.flyteinteractive.vscode_lib.decorator.prepare_launch_json"
) as mock_prepare_launch_json:
yield (
mock_process,
Expand Down Expand Up @@ -206,8 +206,8 @@ def wf(a: int, b: int):
mock_prepare_launch_json.assert_called_once()


@mock.patch("flytekitplugins.flyin.jupyter_lib.decorator.subprocess.Popen")
@mock.patch("flytekitplugins.flyin.jupyter_lib.decorator.sys.exit")
@mock.patch("flytekitplugins.flyteinteractive.jupyter_lib.decorator.subprocess.Popen")
@mock.patch("flytekitplugins.flyteinteractive.jupyter_lib.decorator.sys.exit")
def test_jupyter(mock_exit, mock_popen):
@task
@jupyter
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import os

from flytekitplugins.flyin import get_task_inputs
from flytekitplugins.flyin.utils import load_module_from_path
from flytekitplugins.flyteinteractive import get_task_inputs
from flytekitplugins.flyteinteractive.utils import load_module_from_path


def test_load_module_from_path():
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from flytekitplugins.flyin import vscode
from flytekitplugins.flyteinteractive import vscode

from flytekit import task

Expand Down
2 changes: 1 addition & 1 deletion plugins/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@
"flytekitplugins-sqlalchemy": "flytekit-sqlalchemy",
"flytekitplugins-vaex": "flytekit-vaex",
"flytekitplugins-whylogs": "flytekit-whylogs",
"flytekitplugins-flyin": "flytekit-flyin",
"flytekitplugins-flyteinteractive": "flytekit-flyteinteractive",
}


Expand Down

0 comments on commit b744a11

Please sign in to comment.