v1.10.3b5

What's Changed

• Plugins lazy module loading by @cosmicBboy in #2049
• Add support pandas 2 in flytekit by @pingsutw in #1818
• Enable the flytekit to be pluggable by @thomasjpfan in #2039
• async s3fs plugin by @troychiu in #1936
• Enable python 3.12 by @eapolinario in #2005
• Fixes type hint for configure_pyflyte_cli in plugin by @thomasjpfan in #2065
• Bump apache-airflow from 2.7.3 to 2.8.0 in /plugins/flytekit-airflow by @dependabot in #2066
• Fix: Handle SIGTERM in kubeflow pytorch elastic training plugin by @fg91 in #2064
• unpin envd and add extra_path to imagespec by @samhita-alla in #2063
• Change Flyin VScode Default Extensions by @Future-Outlier in #2058
• Allows Secret groups to be optional and configurable by @thomasjpfan in #2062
• [Core feature] Allow dictionaries to be passed to execute workflows that take dataclasses by @jasonlai1218 in #2013
• Flyin - fix the wrong usage of context. by @troychiu in #2070
• Error when multiple configuration plugins are installed by @thomasjpfan in #2069
• Fix Get the Secret Bug from OS Environment Variable by @Future-Outlier in #2077
• Generalized subproc_execute and added tests by @pryce-turner in #2072
• Improve error handling for local execution by @pingsutw in #2068
• Download specific files in FlyteDirectory by @pingsutw in #2059
• Constraint the full fsspec family by @honnix in #2087
• Update default template used by pyflyte init by @neverett in #2041
• Support PodTemplate in ArrayNode by @hamersaw in #2088
• Remove unnecessary newline in plugin README by @douenergy in #2090
• Fix file not found error by @austin362667 in #2079
• add monodocs redirect banner by @cosmicBboy in #2091
• Accelerator docs by @kumare3 in #2076
• Remote workflow & task execution by @kumare3 in #2094
• Print log links in local execution by @pingsutw in #2097
• Bump fonttools from 4.41.1 to 4.43.0 in /tests/flytekit/integration/remote/mock_flyte_repo/workflows by @dependabot in #2092
• add monodocs index page in flytekit by @cosmicBboy in #2100
• add monodocs redirects to the flytekit readthedocs project by @cosmicBboy in #2095
• Local Sync Agent by @Future-Outlier in #2085
• Bump jinja2 from 3.1.2 to 3.1.3 in /plugins/flytekit-airflow by @dependabot in #2103
• Bump jinja2 from 3.1.2 to 3.1.3 by @dependabot in #2104
• Enhance Airflow Plugin by @Future-Outlier in #2099
• Bump jinja2 from 3.1.2 to 3.1.3 in /plugins/flytekit-onnx-pytorch by @dependabot in #2102
• Bump jinja2 from 3.1.2 to 3.1.3 in /tests/flytekit/integration/remote/mock_flyte_repo/workflows by @dependabot in #2101
• Remove storage as a task resource option by @Future-Outlier in #2078
• Fix Flyin Plugin VSCode download bug by @Future-Outlier in #1991
• Adds conda support to envd plugin by @thomasjpfan in #2020
• Fix is_deferrable in airflow agent by @pingsutw in #2109

New Contributors

• @neverett made their first contribution in #2041
• @douenergy made their first contribution in #2090
• @austin362667 made their first contribution in #2079

Full Changelog: v1.10.3b4...v1.10.3b5

v1.10.3b4

What's Changed

• Fix warnings in package building by @thomasjpfan in #2051
• Inject version into the build environment by @thomasjpfan in #2052
• Run commands in imageSpec by @pingsutw in #1909
• Add log_links to GetTaskResponse by @pingsutw in #2021
• feat: add source code deck by @jasonlai1218 in #2038
• Logger improvements by @kumare3 in #2053
• Preserve stacktrace by @RRap0so in #2056
• Update vscode decorator by @pingsutw in #2018
• Agent uploads input.pb even if task input is None by @pingsutw in #2054
• Fix Base Agent Bug by @Future-Outlier in #2057

New Contributors

• @RRap0so made their first contribution in #2056

Full Changelog: v1.10.3b3...v1.10.3b4

v1.10.3b3

What's Changed

• Add support failure node by @kumare3 in #840
• Add various executions phases which are available in flyteidl, but not in flytekit by @mdjong1 in #2022
• Flyin - VSCode default directory by @troychiu in #2046
• Monodocs dev build by @cosmicBboy in #2048

New Contributors

• @mdjong1 made their first contribution in #2022

Full Changelog: v1.10.3b2...v1.10.3b3

v1.10.3b2

What's Changed

• Configures ruff to also run isort by @thomasjpfan in #2043
• Adds setuptools and wheel back in the CI for plugins by @thomasjpfan in #2044

Full Changelog: v1.10.3b1...v1.10.3b2

v1.10.3b1

What's Changed

• Fix Flyin Dockerfile by @Future-Outlier in #2040
• Flyin - fix vscode task path by @troychiu in #2042
• add monodocs ci build in flytekit by @cosmicBboy in #2035
• Support Databricks Agent API 2.1 and Console URL by @Future-Outlier in #1935

Full Changelog: v1.10.3b0...v1.10.3b1

v1.10.3b0

What's Changed

• Flyin - Enhance code server installation based on system architecture by @jasonlai1218 in #2016
• changing permissions of /tmp by @pingsutw in #2026
• removes references to unused plugins by @cosmicBboy in #2025
• Separate agent from pyflyte serve by @chaohengstudent in #2019
• Flyin - vscode resume task by @troychiu in #2017
• Run databricks task locally by @pingsutw in #1951
• Fix flyin image by setting py to 3.10 by @ByronHsu in #2031
• Generate version with setuptools_scm and migrate to pyproject.toml by @thomasjpfan in #2028

Full Changelog: v1.10.2...v1.10.3b0

v1.10.2

What's Changed

• Add support spark plugin in python-3.11 and remove requirement.txt by @pingsutw in #1940
• Support imagespec in V1Container (pod_template) by @ByronHsu in #1952
• feat: add file type validation and mime type extraction by @jasonlai1218 in #1893
• Fix bug in polars parquet decoding handler by @danpf in #1899
• Added subproc_raise public facing function for calling shell commands by @pryce-turner in #1949
• Bump werkzeug from 2.3.7 to 2.3.8 in /plugins/flytekit-dbt by @dependabot in #1960
• MAINT Adds support for pytest-xdist and enables it in the CI by @thomasjpfan in #1974
• MAINT Updates github actions to their latest versions by @thomasjpfan in #1975
• MAINT Remove natsort dependency by @thomasjpfan in #1969
• MAINT Removes GitPython dependency by @thomasjpfan in #1966
• add support for gate nodes in sync_node_execution by @samhita-alla in #1971
• Adds deprecation warning for disable_deck and point to enable_deck by @thomasjpfan in #1964
• Bump aiohttp from 3.8.5 to 3.8.6 in /plugins/flytekit-spark by @dependabot in #1959
• MAINT Removes pyopenssl dependency by @thomasjpfan in #1965
• MAINT Remove pytz dependency by @thomasjpfan in #1968
• Removes docker-image-py dependency by @thomasjpfan in #1978
• FIX Fixes codecov upload by @thomasjpfan in #1979
• Add Sections in the PR Template by @Future-Outlier in #1939
• Migrates developer tooling to use ruff by @thomasjpfan in #1982
• MAINT Adds more timeout for flakey test by @thomasjpfan in #1981
• MAINT Removes sortedcontainers, deprecated, and dateutil dependencies by @thomasjpfan in #1970
• New Features in VSCode Plugin: Auto server termination after max idle time and default installation of python + jupyter notebook extensions by @JasonZhu1313 in #1977
• Do not check if image builder exists in the dynamic task by @pingsutw in #1967
• Rename vscode as flyin plugin by @ByronHsu in #1986
• Bump aiohttp from 3.8.5 to 3.8.6 in /tests/flytekit/integration/remote/mock_flyte_repo/workflows by @dependabot in #1958
• Agent Get Pending Message by @Future-Outlier in #1834
• Run serialization test individually by @pingsutw in #1985
• Use ImageSpec inside raw container task by @pingsutw in #1944
• MAINT Remove wrapt dependency by @thomasjpfan in #1973
• [flyin] Use current context logger instead of root logger to show info level logs by @ByronHsu in #1998
• feat: add jupyter notebook decorator by @jasonlai1218 in #1976
• convert datetime to date when the type is specified as date by @samhita-alla in #1999
• Add codecov exclude_lines configuration by @ringohoffman in #1995
• Run task first and detect local execution by @troychiu in #1997
• Return None type instead of exception by @honnix in #1907
• Bump aiohttp from 3.8.6 to 3.9.0 in /plugins/flytekit-spark by @dependabot in #2001
• Bump aiohttp from 3.8.6 to 3.9.0 in /tests/flytekit/integration/remote/mock_flyte_repo/workflows by @dependabot in #2002
• Remove upper bound on pyarrow by @wild-endeavor in #1988
• Fix: Restore of corrupt checkpoints by @fellhorn in #1996
• Fixed CI job error (aiohttp) by @Future-Outlier in #2007
• Bump cryptography from 41.0.4 to 41.0.6 in /tests/flytekit/integration/remote/mock_flyte_repo/workflows by @dependabot in #2006
• Fix: Detection of wrong type transformer for multi-inheritance enum by @fg91 in #2004
• Update Flytekit Dockerfile by @Future-Outlier in #1993
• change the wait by @wild-endeavor in #1961
• Airflow Agent Refactor by @pingsutw in #1929
• Bump cryptography from 41.0.5 to 41.0.6 in /plugins/flytekit-airflow by @dependabot in #2008
• Remove wheel dependency by @thomasjpfan in #2010
• Bump aiohttp from 3.8.6 to 3.9.0 in /plugins/flytekit-airflow by @dependabot in #2009
• Flyin - interactive debugging by @troychiu in #2000
• Flyin Dockerfile by @Future-Outlier in #1990
• Add Flyin Config for VSCode Extension by @Future-Outlier in #1989
• Fix default paramaters when launching launchplans from within other workflows by @Tom-Newton in #1972
• Add Why, What, Tests Sections in PR Template by @Future-Outlier in #2003
• Override the container image of map task by @pingsutw in #2011
• Introduce class decorator to flyin vscode for url exposure by @ByronHsu in #1994
• use original file/directory with local pyflyte run by @cosmicBboy in #1957
• Fix union transformer by @pingsutw in #2024
• Simplify CLI implementation for build and run by @thomasjpfan in #2023

New Contributors

• @jasonlai1218 made their first contribution in #1893
• @danpf made their first contribution in #1899
• @pryce-turner made their first contribution in #1949
• @thomasjpfan made their first contribution in #1974
• @JasonZhu1313 made their first contribution in #1977
• @fellhorn made their first contribution in #1996

Full Changelog: v1.10.1...v1.10.2

v1.10.1

What's Changed

• Fix flyteagent image build by @eapolinario in #1906
• Flyte flyte:// file system and improve remote file handling by @wild-endeavor in #1674
• Improved rendering of output literals by @kumare3 in #1916
• Use Airflow task inside dynamic workflow by @pingsutw in #1912
• Add more tests for sensor by @pingsutw in #1892
• [WIP] Fix: Allow elastic tasks to be recoverable by @fg91 in #1846
• Make sure $HOME is writable in dev image by @ByronHsu in #1924
• Use communicate in shell task by @eapolinario in #1923
• Improve error message when omitting task kwarg in workflows by @fg91 in #1921
• Add support for min_success_ratio for local map_task execution by @chaohengstudent in #1884
• Do not cache packages in images by @eapolinario in #1927
• [flytekit] Support attribute access on promises by @ByronHsu in #1825
• Pyflyte --env Option for Register and Serialize by @MinuraPunchihewa in #1880
• Feat: Add Audience + Scopes Param to PKCE and DeviceCode Auth Flows by @PudgyPigeon in #1876
• [Extended Resources] GPU Accelerators by @jeevb in #1843
• Fix remote integration tests by @troychiu in #1896
• Set protobuf to <4.25.0 by @pingsutw in #1934
• Auto download for fetch by @kumare3 in #1918
• Get launchplans by @kumare3 in #1930
• Cancel a currently running workflow from the same PR when a new workflow is triggered by @pingsutw in #1886
• Add hadoop aws/gcp jar to the spark default image by @pingsutw in #1908
• Sort install_requires dependencies by @bstadlbauer in #1931
• Add vscode plugin to enable interactive debugging by @troychiu in #1922
• Kubeflow TensorFlow Training Operator Add Evaluator by @Future-Outlier in #1870
• Support pip install for vaex on python3.10 by @ericwudayi in #1942
• Expose -r in pyflyte run by @eapolinario in #1933
• Adds download command to remote. by @kumare3 in #1946
• Feat: If PKCE AuthorizationClient cannot open browser, ask user to do so manually by @fg91 in #1947
• Bump pyarrow from 10.0.1 to 14.0.1 in /tests/flytekit/integration/remote/mock_flyte_repo/workflows by @dependabot in #1950
• Fix lint test by @pingsutw in #1956
• Fix remote file system (get/put) by @pingsutw in #1955

New Contributors

• @MinuraPunchihewa made their first contribution in #1880
• @ericwudayi made their first contribution in #1942

Full Changelog: v1.10.0...v1.10.1

v1.10.1b0

What's Changed

• Fix flyteagent image build by @eapolinario in #1906
• Flyte flyte:// file system and improve remote file handling by @wild-endeavor in #1674
• Improved rendering of output literals by @kumare3 in #1916
• Use Airflow task insdie dynamic workflow by @pingsutw in #1912

Full Changelog: v1.10.0...v1.10.1b0

v1.10.0

What's Changed

• Better error messaging for overrides by @kumare3 in #1807
• Run remote Launchplan from pyflyte run by @kumare3 in #1785
• Add is none function by @pingsutw in #1757
• Dynamic workflow should not throw nested task warning by @oliverhu in #1812
• Add a manual image building GH action by @wild-endeavor in #1816
• Enable Azure Workload Identity for fsspec in flytekit by @fiedlerNr9 in #1813
• Fix list of annotated structured dataset by @wild-endeavor in #1817
• Support the flytectl config.yaml admin.clientSecretEnvVar option in flytekit by @chaohengstudent in #1819
• Async agent delete function for while loop case by @Future-Outlier in #1802
• fix docs warnings by @samhita-alla in #1827
• Fix extract_task_module by @pingsutw in #1829
• Feat: Add type support for pydantic BaseModels by @ArthurBook in #1660
• Make FlyteRemote example slightly more copy/pastable by @katrogan in #1830
• Pyflyte meta inputs by @kumare3 in #1823
• Use mashumaro to serialize/deserialize dataclass by @hhcs9527 in #1735
• Databricks Agent by @Future-Outlier in #1797
• Prometheus metrics by @pingsutw in #1815
• Pyflyte register optionally activates schedule by @kumare3 in #1832
• Remove versions 3.9 and 3.10 by @wild-endeavor in #1831
• Snowflake agent by @hhcs9527 in #1799
• Update agent metric name by @pingsutw in #1835
• MemVerge MMCloud Agent by @edwinyyyu in #1821
• Add download badges to the readme by @pingsutw in #1836
• Eager local entrypoint and support for offloaded types by @cosmicBboy in #1833
• update requirements and add snowflake agent to api reference by @samhita-alla in #1838
• Fix: Make sure decks created in elastic task workers are transferred to parent process by @fg91 in #1837
• add accept grpc by @wild-endeavor in #1841
• Feat: Enable flytekit to authenticate with proxy in front of FlyteAdmin by @fg91 in #1787
• Backfill command now supports failure-policy by @kumare3 in #1840
• Pass cluster pool when creating executions by @iaroslav-ciupin in #1208
• Add more clear error message when fetching secrets by @ysysys3074 in #1847
• Pyflyte run workflows correctly handles Optional[TYPE] = None by @cosmicBboy in #1849
• Bump gitpython from 3.1.32 to 3.1.35 in /tests/flytekit/integration/remote/mock_flyte_repo/workflows by @dependabot in #1828
• Bump cryptography from 41.0.3 to 41.0.4 in /tests/flytekit/integration/remote/mock_flyte_repo/workflows by @dependabot in #1844
• update codecov config in pythonbuild ci by @cosmicBboy in #1852
• Improved gate node handling by @kumare3 in #1850
• update codecov yaml, make eager wf test more stable by @cosmicBboy in #1854
• more codecov updates by @cosmicBboy in #1856
• make mlflow plugin work with python 3.11 by @bcvanmeurs in #1855
• Agent get secret function by @Future-Outlier in #1851
• Better Azure blob storage support by @Tom-Newton in #1842
• Remove Envd from dockerfile by @pingsutw in #1814
• Update agent uri format by @hhcs9527 in #1861
• modified setup.py in flytekit-mmcloud by @helenzhangyc in #1864
• Label request_failure metric with error_code by @pingsutw in #1862
• add back Edwin's name by @helenzhangyc in #1868
• Fix py38 aync agent service and add async agent test by @Future-Outlier in #1866
• Pin because not installing by @wild-endeavor in #1872
• Bump pillow from 10.0.0 to 10.0.1 in /tests/flytekit/integration/remote/mock_flyte_repo/workflows by @dependabot in #1867
• Prune hosted github runner to free up space for sandbox by @jeevb in #1875
• Fixes Auth issue, where refresh token is expired by @kumare3 in #1873
• Name field is incorrect assigned by @kumare3 in #1874
• [Docs]Updated examples for customizing resources by @LunarMarathon in #1871
• change key path to SNOWFLAKE_PRIVATE_KEY by @hhcs9527 in #1879
• Run notebook task on aws batch by @pingsutw in #1859
• Automatically Convert name Argument and registry Argument to Lowercase in ImageSpec by @Future-Outlier in #1881
• Dep: Pin min required flytekit version in IAP plugin by @fg91 in #1878
• Remove duplicate click option by @wild-endeavor in #1883
• Bump urllib3 from 1.26.16 to 1.26.17 in /tests/flytekit/integration/remote/mock_flyte_repo/workflows by @dependabot in #1865
• Improve error message in ListTransformer when not passing a list by @fg91 in #1882
• Update typing_extension Any import to typing due to typing_extension version compatibility by @hfurkanvural in #1877
• Bump gitpython from 3.1.35 to 3.1.37 in /tests/flytekit/integration/remote/mock_flyte_repo/workflows by @dependabot in #1885
• Airflow agent by @pingsutw in #1725
• Add mmcloud plugin into the agent image by @pingsutw in #1891
• Fix map_task sensitive to argument order by @chaohengstudent in #1888
• Bump urllib3 from 1.26.17 to 1.26.18 in /tests/flytekit/integration/remote/mock_flyte_repo/workflows by @dependabot in #1897
• Add enable_deck to task decorator by @eapolinario in #1898
• Add image transformer by @pingsutw in #1901
• add default project domain in execute launch plan by @troychiu in #1894
• Add Databricks and Snowflake agent to Dockerfile by @Future-Outlier in #1905
• Fix module loading bug in instance tracker class by @cosmicBboy in #1902

New Contributors

• @fiedlerNr9 made their first contribution in #1813
• @chaohengstudent made their first contribution in #1819
• @ArthurBook made their first contribution in #1660
• @hhcs9527 made their first contribution in #1735
• @edwinyyyu made their first contribution in #1821
• @iaroslav-ciupin made their first contribution in #1208
• @bcvanmeurs made their first contribution in #1855
• @Tom-Newton made their first contribution in #1842
• @helenzhangyc made their first contribution in #1864
• @LunarMarathon made their first contribution in #1871

Full Changelog: v1.9.1...v1.10.0