v1.11.0

Breaking change

As of this version, overriding container image via task node overrides requires flytepropeller 1.11.0.

What's Changed

• Extends config plugin API to customize the default OAuth success html by @thomasjpfan in #2189
• Fix Airflow Agent kwargs bug by @Future-Outlier in #2195
• Container Image should be an input into computing the task version by @pingsutw in #2194
• Bump cryptography from 42.0.0 to 42.0.2 in /tests/flytekit/integration/remote/mock_flyte_repo/workflows by @dependabot in #2197
• add executable to shell task by @samhita-alla in #2084
• feat(flyteinteractive): Disable workspace trust for vscode server by @MortalHappiness in #2202
• Fix local overwrite cache by @ChungYujoyce in #2198
• docs(flyteinteractive): Change docker build to docker buildx build by @MortalHappiness in #2200
• Distribute tests by file by @eapolinario in #2199
• Support overridding cache configuration by @hamersaw in #2154
• [caching] Add overwrite_cache flag when creating launch plan by @ysysys3074 in #2029
• pin great expectations version and run local cache tests serially by @samhita-alla in #2205
• Move agent (extend) tests to extras by @wild-endeavor in #2218
• Fix CI error by @pingsutw in #2220
• upload files/directories when using remote-workflow by @samhita-alla in #2222
• CI Uses GITHUB_OUTPUT in github actions by @thomasjpfan in #2219
• Add launch condition column by @wild-endeavor in #2204
• Add SyncAgentBase by @pingsutw in #2146
• Use disk-space-reclaimer only when running envd tests by @pingsutw in #2225
• Dep: Pin grpcio in flytekit-identity-aware-proxy to version that allo… by @fg91 in #2212
• Fix: Handle overriding of container image in backend by @fg91 in #2176
• Fixes failing test related to TaskNodeOverrides by @thomasjpfan in #2230
• ci(build-image): Fix incomplete double quote in yaml file by @MortalHappiness in #2209
• Add agent slim image by @pingsutw in #2231
• Add health check servicer for agent by @pingsutw in #2232
• Bump orjson from 3.9.10 to 3.9.15 in /plugins/flytekit-airflow by @dependabot in #2216
• ChatGPT Agent by @Future-Outlier in #2226
• Add --overwrite-cache in pyflyte backfill cli by @ChungYujoyce in #2214
• Add default_project in register_launch_plan by @ChungYujoyce in #2215
• Fix Monodocs build by @pingsutw in #2235
• Enhance default image to check FLYTE_INTERNAL_IMAGE by @ddl-rliu in #2223
• Define a str method for FlyteException by @noahjax in #2203
• Remove trailing '=' character from generated image tag hash value by @jasonlai1218 in #2240
• Simpler dev images by @eapolinario in #2243
• Only have one flyteidl entry in pyproject toml by @thomasjpfan in #2245
• DOCS-301 update flytekit links to monodocs by @ppiegaze in #2210
• Output metadata tracking by @wild-endeavor in #2221
• Define python versions a priori and schedule CI runs once a day by @eapolinario in #2237
• Simplify version calculation by @eapolinario in #2247
• Adds a FlyteDeck markdown renderer to core by @thomasjpfan in #2246
• DeleteTask should return DeleteTaskResponse by @pingsutw in #2251
• Promote source code renderer and use it by default by @thomasjpfan in #2248
• Refactor Databricks Agent Phase by @Future-Outlier in #2244
• feat(resources): support serialization of Resources dataclasses by @cameronraysmith in #2250
• Fix sql alchemy plugin error module 'pandas.io' has no attribute 'common' by @Future-Outlier in #2249
• ShellTask: stores process return code, stdout and stderr for later use. by @benoistlaurent in #2229
• Bump flyteidl for spark plugin by @pingsutw in #2253
• Rewrite test to work better on all platforms by @eapolinario in #2255
• Support pydantic plugin in 2.xx version by @Future-Outlier in #2217
• Fix FlyteFS by @pingsutw in #2208

New Contributors

• @noahjax made their first contribution in #2203
• @cameronraysmith made their first contribution in #2250
• @benoistlaurent made their first contribution in #2229

Full Changelog: v1.10.7...v1.11.0

[Beta] Ability to track output metadata

Output metadata tracking (#2221)

Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Ketan Umare <[email protected]>

v1.10.8b3

What's Changed

• Remove trailing '=' character from generated image tag hash value by @jasonlai1218 in #2240
• Simpler dev images by @eapolinario in #2243
• Only have one flyteidl entry in pyproject toml by @thomasjpfan in #2245

Full Changelog: v1.10.8b2...v1.10.8b3

v1.10.8b2

What's Changed

• Add SyncAgentBase by @pingsutw in #2146
• Use disk-space-reclaimer only when running envd tests by @pingsutw in #2225
• Dep: Pin grpcio in flytekit-identity-aware-proxy to version that allo… by @fg91 in #2212
• Fix: Handle overriding of container image in backend by @fg91 in #2176
• Fixes failing test related to TaskNodeOverrides by @thomasjpfan in #2230
• ci(build-image): Fix incomplete double quote in yaml file by @MortalHappiness in #2209
• Add agent slim image by @pingsutw in #2231
• Add health check servicer for agent by @pingsutw in #2232
• Bump orjson from 3.9.10 to 3.9.15 in /plugins/flytekit-airflow by @dependabot in #2216
• ChatGPT Agent by @Future-Outlier in #2226
• Add --overwrite-cache in pyflyte backfill cli by @ChungYujoyce in #2214
• Add default_project in register_launch_plan by @ChungYujoyce in #2215
• Fix Monodocs build by @pingsutw in #2235
• Enhance default image to check FLYTE_INTERNAL_IMAGE by @ddl-rliu in #2223
• Define a str method for FlyteException by @noahjax in #2203

New Contributors

• @noahjax made their first contribution in #2203

Full Changelog: v1.10.8b1...v1.10.8b2

[Beta] Add launch condition column

different column for trigger

v1.10.8b0

What's Changed

• Extends config plugin API to customize the default OAuth success html by @thomasjpfan in #2189

Full Changelog: v1.10.7...v1.10.8b0

v1.10.7

What's Changed

• Extends ImageSpec to accept image names from plugin and have priority for plugins by @thomasjpfan in #2119
• Use logger in data_persistence.py by @eapolinario in #2129
• Adjust tar method to iterate over files/dirs in dir rather than strip… by @neilisaur in #2131
• Include exception type in error messages by @Tom-Newton in #2130
• Adds get_default_image into configuration plugin by @thomasjpfan in #2133
• Add 3.12 as classifier by @honnix in #2135
• Fixing copy-all version of tar file creation as well by @neilisaur in #2134
• Bump pillow from 10.1.0 to 10.2.0 in /plugins/flytekit-onnx-pytorch by @dependabot in #2127
• Bump aiohttp from 3.8.6 to 3.9.2 by @dependabot in #2137
• Bump aiohttp from 3.9.1 to 3.9.2 in /plugins/flytekit-spark by @dependabot in #2140
• Bump aiohttp from 3.9.0 to 3.9.2 in /plugins/flytekit-airflow by @dependabot in #2139
• Bump aiohttp from 3.9.1 to 3.9.2 in /tests/flytekit/integration/remote/mock_flyte_repo/workflows by @dependabot in #2138
• Bump pillow from 10.1.0 to 10.2.0 in /plugins/flytekit-onnx-tensorflow by @dependabot in #2126
• Envvars local execution by @eapolinario in #2132
• node_dependency_hints for dynamic tasks by @Tom-Newton in #2015
• Warn user when overriding requests but not limits by @fg91 in #2151
• Improve error message for pyflyte run by @pingsutw in #2142
• Remove upper version bound from protobuf by @pingsutw in #2144
• Agent Metadata Servicer by @Future-Outlier in #2012
• Fix: Improve error handling in workflow compilation when output binding fails by @fg91 in #2047
• Add metadata to literal by @pingsutw in #2147
• Add _literal_map_to_python_input to base task by @pingsutw in #2150
• Fix: Allow both '*test.py' and '*test.py' test module naming convention for nested tasks by @fg91 in #2155
• Fix: Limit grpcio version in flytekit-identity-aware-proxy due to regression by @fg91 in #2156
• [Refactor] Rename flyin to flyteinteractive by @MortalHappiness in #2157
• install latest flyteidl with monodocs build by @cosmicBboy in #2162
• Bump fastapi from 0.108.0 to 0.109.1 by @dependabot in #2161
• Replace Agent State with Agent Phase by @Future-Outlier in #2123
• install latest flyteidl with monodocs build by @cosmicBboy in #2163
• Bump pillow from 10.0.1 to 10.2.0 in /tests/flytekit/integration/remote/mock_flyte_repo/workflows by @dependabot in #2125
• Remove docs gh workflows by @eapolinario in #2164
• Remove kubernetes from dependencies by @pingsutw in #2148
• Fix lint error caused by #2164 by @eapolinario in #2166
• Add support Union[FlyteDirectory, FlyteFile] by @pingsutw in #2149
• Bump cryptography from 41.0.6 to 42.0.0 in /tests/flytekit/integration/remote/mock_flyte_repo/workflows by @dependabot in #2165
• Remove upper version bound from fsspec by @pingsutw in #2143
• Use python3.8 to generate dev-requirements.txt for Great Expectations by @pingsutw in #2168
• Modify recursive paths by @wild-endeavor in #2121
• Force tests in a module to be run by the same worker by @eapolinario in #2177
• art id proto by @wild-endeavor in #1928
• add env vars option in pyflyte package by @fiedlerNr9 in #2171
• Build python 3.12 default flytekit image by @eapolinario in #2181
• Fix airflow sensor by @pingsutw in #2169
• Remove asynchronous flag from base agent by @pingsutw in #2141
• Update Base Agent Logs by Rich Progress Feature by @Future-Outlier in #2159
• Fix Base Agent Input Bug by @Future-Outlier in #2186
• Bump grpcio from 1.53.0 to 1.53.2 in /tests/flytekit/integration/remote/mock_flyte_repo/workflows by @dependabot in #2187
• Disable rich logger in the default image by @pingsutw in #2185
• Enhance error logging in pyflyte by @ddl-rliu in #2190
• Do not lazy-load pyspark.ml by @eapolinario in #2184
• Bump flyteidl to 1.10.7 by @eapolinario in #2192

New Contributors

• @neilisaur made their first contribution in #2131
• @MortalHappiness made their first contribution in #2157
• @ddl-rliu made their first contribution in #2190

Full Changelog: v1.10.3...v1.10.7

[Beta] Buf & new protos

Re-releasing the same release to a 1.10.7 number so as to match the coming main flyte release.

Below are the original release notes:

What's Changed

• Extends ImageSpec to accept image names from plugin and have priority for plugins by @thomasjpfan in #2119
• Use logger in data_persistence.py by @eapolinario in #2129
• Adjust tar method to iterate over files/dirs in dir rather than strip… by @neilisaur in #2131
• Include exception type in error messages by @Tom-Newton in #2130
• Adds get_default_image into configuration plugin by @thomasjpfan in #2133
• Add 3.12 as classifier by @honnix in #2135
• Fixing copy-all version of tar file creation as well by @neilisaur in #2134
• Bump pillow from 10.1.0 to 10.2.0 in /plugins/flytekit-onnx-pytorch by @dependabot in #2127
• Bump aiohttp from 3.8.6 to 3.9.2 by @dependabot in #2137
• Bump aiohttp from 3.9.1 to 3.9.2 in /plugins/flytekit-spark by @dependabot in #2140
• Bump aiohttp from 3.9.0 to 3.9.2 in /plugins/flytekit-airflow by @dependabot in #2139
• Bump aiohttp from 3.9.1 to 3.9.2 in /tests/flytekit/integration/remote/mock_flyte_repo/workflows by @dependabot in #2138
• Bump pillow from 10.1.0 to 10.2.0 in /plugins/flytekit-onnx-tensorflow by @dependabot in #2126
• Envvars local execution by @eapolinario in #2132
• node_dependency_hints for dynamic tasks by @Tom-Newton in #2015
• Warn user when overriding requests but not limits by @fg91 in #2151
• Improve error message for pyflyte run by @pingsutw in #2142
• Remove upper version bound from protobuf by @pingsutw in #2144
• Agent Metadata Servicer by @Future-Outlier in #2012
• Fix: Improve error handling in workflow compilation when output binding fails by @fg91 in #2047
• Add metadata to literal by @pingsutw in #2147
• Add _literal_map_to_python_input to base task by @pingsutw in #2150
• Fix: Allow both '*test.py' and '*test.py' test module naming convention for nested tasks by @fg91 in #2155
• Fix: Limit grpcio version in flytekit-identity-aware-proxy due to regression by @fg91 in #2156
• [Refactor] Rename flyin to flyteinteractive by @MortalHappiness in #2157
• install latest flyteidl with monodocs build by @cosmicBboy in #2162
• Bump fastapi from 0.108.0 to 0.109.1 by @dependabot in #2161
• Replace Agent State with Agent Phase by @Future-Outlier in #2123
• install latest flyteidl with monodocs build by @cosmicBboy in #2163
• Bump pillow from 10.0.1 to 10.2.0 in /tests/flytekit/integration/remote/mock_flyte_repo/workflows by @dependabot in #2125
• Remove docs gh workflows by @eapolinario in #2164
• Remove kubernetes from dependencies by @pingsutw in #2148
• Fix lint error caused by #2164 by @eapolinario in #2166
• Add support Union[FlyteDirectory, FlyteFile] by @pingsutw in #2149
• Bump cryptography from 41.0.6 to 42.0.0 in /tests/flytekit/integration/remote/mock_flyte_repo/workflows by @dependabot in #2165
• Remove upper version bound from fsspec by @pingsutw in #2143
• Use python3.8 to generate dev-requirements.txt for Great Expectations by @pingsutw in #2168
• Modify recursive paths by @wild-endeavor in #2121
• Force tests in a module to be run by the same worker by @eapolinario in #2177
• art id proto by @wild-endeavor in #1928
• add env vars option in pyflyte package by @fiedlerNr9 in #2171

New Contributors

• @neilisaur made their first contribution in #2131
• @MortalHappiness made their first contribution in #2157

Full Changelog: v1.10.3...v1.10.3b7

v1.10.3

What's Changed

• Flyin - Enhance code server installation based on system architecture by @jasonlai1218 in #2016
• changing permissions of /tmp by @pingsutw in #2026
• removes references to unused plugins by @cosmicBboy in #2025
• Separate agent from pyflyte serve by @chaohengstudent in #2019
• Flyin - vscode resume task by @troychiu in #2017
• Run databricks task locally by @pingsutw in #1951
• Fix flyin image by setting py to 3.10 by @ByronHsu in #2031
• Generate version with setuptools_scm and migrate to pyproject.toml by @thomasjpfan in #2028
• Fix Flyin Dockerfile by @Future-Outlier in #2040
• Flyin - fix vscode task path by @troychiu in #2042
• add monodocs ci build in flytekit by @cosmicBboy in #2035
• Support Databricks Agent API 2.1 and Console URL by @Future-Outlier in #1935
• Configures ruff to also run isort by @thomasjpfan in #2043
• Adds setuptools and wheel back in the CI for plugins by @thomasjpfan in #2044
• Add support failure node by @kumare3 in #840
• Add various executions phases which are available in flyteidl, but not in flytekit by @mdjong1 in #2022
• Flyin - VSCode default directory by @troychiu in #2046
• Monodocs dev build by @cosmicBboy in #2048
• Fix warnings in package building by @thomasjpfan in #2051
• Inject version into the build environment by @thomasjpfan in #2052
• Run commands in imageSpec by @pingsutw in #1909
• Add log_links to GetTaskResponse by @pingsutw in #2021
• feat: add source code deck by @jasonlai1218 in #2038
• Logger improvements by @kumare3 in #2053
• Preserve stacktrace by @RRap0so in #2056
• Update vscode decorator by @pingsutw in #2018
• Agent uploads input.pb even if task input is None by @pingsutw in #2054
• Fix Base Agent Bug by @Future-Outlier in #2057
• Plugins lazy module loading by @cosmicBboy in #2049
• Add support pandas 2 in flytekit by @pingsutw in #1818
• Enable the flytekit to be pluggable by @thomasjpfan in #2039
• async s3fs plugin by @troychiu in #1936
• Enable python 3.12 by @eapolinario in #2005
• Fixes type hint for configure_pyflyte_cli in plugin by @thomasjpfan in #2065
• Bump apache-airflow from 2.7.3 to 2.8.0 in /plugins/flytekit-airflow by @dependabot in #2066
• Fix: Handle SIGTERM in kubeflow pytorch elastic training plugin by @fg91 in #2064
• unpin envd and add extra_path to imagespec by @samhita-alla in #2063
• Change Flyin VScode Default Extensions by @Future-Outlier in #2058
• Allows Secret groups to be optional and configurable by @thomasjpfan in #2062
• [Core feature] Allow dictionaries to be passed to execute workflows that take dataclasses by @jasonlai1218 in #2013
• Flyin - fix the wrong usage of context. by @troychiu in #2070
• Error when multiple configuration plugins are installed by @thomasjpfan in #2069
• Fix Get the Secret Bug from OS Environment Variable by @Future-Outlier in #2077
• Generalized subproc_execute and added tests by @pryce-turner in #2072
• Improve error handling for local execution by @pingsutw in #2068
• Download specific files in FlyteDirectory by @pingsutw in #2059
• Constraint the full fsspec family by @honnix in #2087
• Update default template used by pyflyte init by @neverett in #2041
• Support PodTemplate in ArrayNode by @hamersaw in #2088
• Remove unnecessary newline in plugin README by @douenergy in #2090
• Fix file not found error by @austin362667 in #2079
• add monodocs redirect banner by @cosmicBboy in #2091
• Accelerator docs by @kumare3 in #2076
• Remote workflow & task execution by @kumare3 in #2094
• Print log links in local execution by @pingsutw in #2097
• Bump fonttools from 4.41.1 to 4.43.0 in /tests/flytekit/integration/remote/mock_flyte_repo/workflows by @dependabot in #2092
• add monodocs index page in flytekit by @cosmicBboy in #2100
• add monodocs redirects to the flytekit readthedocs project by @cosmicBboy in #2095
• Local Sync Agent by @Future-Outlier in #2085
• Bump jinja2 from 3.1.2 to 3.1.3 in /plugins/flytekit-airflow by @dependabot in #2103
• Bump jinja2 from 3.1.2 to 3.1.3 by @dependabot in #2104
• Enhance Airflow Plugin by @Future-Outlier in #2099
• Bump jinja2 from 3.1.2 to 3.1.3 in /plugins/flytekit-onnx-pytorch by @dependabot in #2102
• Bump jinja2 from 3.1.2 to 3.1.3 in /tests/flytekit/integration/remote/mock_flyte_repo/workflows by @dependabot in #2101
• Remove storage as a task resource option by @Future-Outlier in #2078
• Fix Flyin Plugin VSCode download bug by @Future-Outlier in #1991
• Adds conda support to envd plugin by @thomasjpfan in #2020
• Fix is_deferrable in airflow agent by @pingsutw in #2109
• Use Latest FlyteIdl in Integration Test by @Future-Outlier in #2105
• Changes default destination dir to be "working dir" by @kumare3 in #2108
• Fix typos and run spellcheck in CI by @eapolinario in #2074
• Add @lru_cache to ImageBuildEngine.build by @pingsutw in #2112
• Skip errors when running pyflyte register by @pingsutw in #2111
• Option to disable local caching by @bstadlbauer in #2106
• Omit '=' from auto-generated version by @eapolinario in #2117
• Pass in env vars in FlyteRemote register_script by @knordstrom-muon in #2120
• [Bug] Map task caching failures by @pvditt in #2113
• Check envd version before building images by @pingsutw in #2122
• Bump pillow from 10.1.0 to 10.2.0 by @dependabot in #2128
• Support mashumaro DataClassORJSONMixin by @quinten-flwls in #2080

New Contributors

• @mdjong1 made their first contribution in #2022
• @RRap0so made their first contribution in #2056
• @neverett made their first contribution in #2041
• @douenergy made their first contribution in #2090
• @austin362667 made their first contribution in #2079
• @knordstrom-muon made their first contribution in #2120
• @pvditt made their first contribution in #2113
• @quinten-flwls made their first contribution in #2080

Full Changelog: v1.10.2...v1.10.3

v1.10.3b6

What's Changed

• Use Latest FlyteIdl in Integration Test by @Future-Outlier in #2105
• Changes default destination dir to be "working dir" by @kumare3 in #2108
• Fix typos and run spellcheck in CI by @eapolinario in #2074
• Add @lru_cache to ImageBuildEngine.build by @pingsutw in #2112
• Skip errors when running pyflyte register by @pingsutw in #2111

Full Changelog: v1.10.3b5...v1.10.3b6