Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Creating Whylogs transformer + renderers #1100

Closed
wants to merge 64 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
fb86d26
Add support deck in jupyter notebook (#955)
pingsutw May 19, 2022
97126b9
Got type transformer and unit tests for it working.
naddeoa May 19, 2022
99c0b4d
Fix upload/download logic
naddeoa May 24, 2022
f42743e
Removed TODOs
naddeoa May 25, 2022
76435c0
Updated to use a real version of whylogs
naddeoa May 25, 2022
5291a70
Add constraints example and unit test
naddeoa May 31, 2022
ac5431d
Remove unused imports
naddeoa May 31, 2022
fbea188
refact tests + adding renderers
murilommen Jul 11, 2022
01424d4
removing rc from whylogs' setup
murilommen Jul 12, 2022
b1a8108
making renderers accept pandas dataframes
murilommen Jul 13, 2022
0bc5b8b
fix: doc-requirements.txt to reduce vulnerabilities (#1012)
EngHabu May 19, 2022
9552a0d
Update get_serializable_workflow to consistently generate the same su…
vvasavada-fn May 20, 2022
0e81bc5
Fix the tests on windows (#1013)
pingsutw May 20, 2022
0cb725b
Changed 'design' to 'designed' (#1010)
SmritiSatyanV May 20, 2022
6ab35bd
Skip config checking if no ini config file present (#1016)
wild-endeavor May 20, 2022
b0e1d9f
Fix lint error in data.extend.rst (#1017)
eapolinario May 20, 2022
ba54f82
Bump requirements 1653070910 (#1014)
eapolinario May 20, 2022
a020736
Fix importing code in script mode (#1021)
eapolinario May 26, 2022
7d50da8
Use interface instead of python_interface to support remote entities …
wild-endeavor May 26, 2022
74bb514
pyflyte register (#1019)
wild-endeavor May 27, 2022
1885ca2
Added support for insecureSkipVerify on pyflyte (#1020)
RobertoRRW May 27, 2022
2a15184
Allow struct/dataclass types to be used as default arguments (#1024)
RobertoRRW May 27, 2022
b98ef08
nit (#1029)
wild-endeavor May 27, 2022
7421bd3
Add task_id field to ExecutionParameters (#1022)
wild-endeavor May 27, 2022
82f0548
TypeTransformer for Numpy (#1027)
samhita-alla May 28, 2022
2372b7e
Bump githb codecov action to 1.5.2 (#1031)
eapolinario Jun 1, 2022
f6c2bb5
capture default list/dict inputs in Usage: pyflyte run [OPTIONS] COMM…
samhita-alla Jun 1, 2022
00c5de4
Allow using underscores and numeric in the image name (#1030)
pingsutw Jun 1, 2022
20c1e0b
Bump codecov gh action and pin plugins requirements in tests (#1026)
eapolinario Jun 2, 2022
578891d
Fix wrong version help string (#1056)
fg91 Jun 6, 2022
92e7bc7
Add deck_uri to NodeExecution (#1034)
pingsutw Jun 9, 2022
9f12480
Add support list of flytetype in dataclass (#1057)
pingsutw Jun 9, 2022
af0aca1
fix: plugins/flytekit-aws-sagemaker/requirements.txt to reduce vulner…
eapolinario Jun 10, 2022
c405d39
Bump cookiecutter (#1058)
dependabot[bot] Jun 10, 2022
284fc18
Bump flyteidl and other requirements (#1060)
eapolinario Jun 13, 2022
1a13324
Fix get client in flyte-cli commands (#1063)
eapolinario Jun 14, 2022
e778b2b
Fix port rewrite in the case of sandbox/demo cluster (#1064)
eapolinario Jun 15, 2022
56a5049
Add polars plugin (#1061)
RobinKa Jun 15, 2022
2c89a6f
Fix read authorization_metadata_key from public_client_config in _ref…
sonjaer Jun 16, 2022
b14cb9c
Fix serialization of tasks with environment variables set (#1066)
jeevb Jun 16, 2022
a796a57
Add support union type for pyflyte run (#1062)
pingsutw Jun 18, 2022
0972a42
Support optional input (#989)
pingsutw Jun 20, 2022
b73e7f0
Alternate yaml extension okay (#1075)
wild-endeavor Jun 22, 2022
06dd9a5
Unit tests for promise.py (#1076)
pingsutw Jun 23, 2022
8608af2
Fixed doc for cron schedule (#1081)
pmahindrakar-oss Jun 24, 2022
818ff2a
Return deck html string when calling get_deck() (#1074)
pingsutw Jun 28, 2022
7efec66
Add details on fsspec data plugin (#1082)
SmritiSatyanV Jun 29, 2022
b227886
Bump grpcio to 1.47.0 (#1077)
pingsutw Jun 29, 2022
39ac057
Remove html tags in README markdown (#1083)
cosmicBboy Jun 29, 2022
71aa318
Add support insert in SQLAlchemyTask (#1070)
pingsutw Jun 30, 2022
1755cf2
Add importlib-metadata to setup.py (#1086)
wild-endeavor Jul 1, 2022
e364043
Housekeeping for pyflyte package and register commands (#1084)
madhur-tandon Jul 5, 2022
2a73398
fix: doc-requirements.txt to reduce vulnerabilities (#1095)
eapolinario Jul 6, 2022
606e717
Bump requirements 1657041111 (#1094)
eapolinario Jul 6, 2022
cc9704a
[Snyk] Security upgrade mistune from 0.8.4 to 2.0.3 (#1090)
EngHabu Jul 6, 2022
77994ba
[Snyk] Security upgrade mistune from 0.8.4 to 2.0.3 (#1091)
snyk-bot Jul 6, 2022
2fe939c
TypeTransformers for PyTorch Tensor, Module, and Checkpoint (#1032)
samhita-alla Jul 7, 2022
4b227c5
Correctly fetch `typing.Annotated` metadata in `typing.NamedTuple` (#…
samhita-alla Jul 11, 2022
e630c18
Update README.md (#1098)
cosmicBboy Jul 11, 2022
3451b32
ONNX Plugin (#804)
samhita-alla Jul 13, 2022
95860ff
adding .env directory to gitignore
murilommen Jul 13, 2022
33b6778
fixing project structure and style
murilommen Jul 14, 2022
e07c923
fix setup.py
murilommen Jul 14, 2022
e9a40b0
adding correct author email
murilommen Jul 14, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 19 additions & 2 deletions .github/workflows/pythonbuild.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ jobs:
run: |
coverage run -m pytest tests/flytekit/unit -m "not sandbox_test"
- name: Codecov
uses: codecov/codecov-action@v1
uses: codecov/codecov-action@v1.5.2
with:
fail_ci_if_error: true # optional (default = false)

Expand All @@ -77,16 +77,33 @@ jobs:
- flytekit-kf-pytorch
- flytekit-kf-tensorflow
- flytekit-modin
- flytekit-onnx-pytorch
- flytekit-onnx-scikitlearn
- flytekit-onnx-tensorflow
- flytekit-pandera
- flytekit-papermill
- flytekit-polars
- flytekit-snowflake
- flytekit-spark
- flytekit-sqlalchemy
- flytekit-whylogs
exclude:
# flytekit-modin depends on ray which does not have a 3.10 wheel yet.
# Issue tracked in https://github.com/ray-project/ray/issues/19116.
- python-version: 3.10
plugin-names: "flytekit-modin"
# Great-expectations does not support python 3.10 yet
# https://github.com/great-expectations/great_expectations/blob/develop/setup.py#L87-L89
- python-version: 3.10
plugin-names: "flytekit-greatexpectations"
# onnxruntime does not support python 3.10 yet
# https://github.com/microsoft/onnxruntime/issues/9782
- python-version: 3.10
plugin-names: "flytekit-onnx-pytorch"
- python-version: 3.10
plugin-names: "flytekit-onnx-scikitlearn"
- python-version: 3.10
plugin-names: "flytekit-onnx-tensorflow"
steps:
- uses: actions/checkout@v2
- name: Set up Python ${{ matrix.python-version }}
Expand All @@ -104,7 +121,7 @@ jobs:
run: |
make setup
cd plugins/${{ matrix.plugin-names }}
pip install -e .
pip install -r requirements.txt
if [ -f dev-requirements.txt ]; then pip install -r dev-requirements.txt; fi
pip install --no-deps -U https://github.com/flyteorg/flytekit/archive/${{ github.sha }}.zip#egg=flytekit
pip freeze
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -30,3 +30,6 @@ docs/source/plugins/generated/
.pytest_flyte
htmlcov
*.ipynb
.env
*dat
.env/
8 changes: 4 additions & 4 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,24 +1,24 @@
repos:
- repo: https://github.com/PyCQA/flake8
rev: 3.9.2
rev: 4.0.1
hooks:
- id: flake8
- repo: https://github.com/psf/black
rev: 22.3.0
hooks:
- id: black
- repo: https://github.com/PyCQA/isort
rev: 5.9.3
rev: 5.10.1
hooks:
- id: isort
args: ["--profile", "black"]
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.0.1
rev: v4.2.0
hooks:
- id: check-yaml
- id: end-of-file-fixer
- id: trailing-whitespace
- repo: https://github.com/shellcheck-py/shellcheck-py
rev: v0.7.2.1
rev: v0.8.0.4
hooks:
- id: shellcheck
2 changes: 1 addition & 1 deletion CODEOWNERS
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
# These owners will be the default owners for everything in
# the repo. Unless a later match takes precedence.
* @wild-endeavor @kumare3 @eapolinario
* @wild-endeavor @kumare3 @eapolinario @pingsutw
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ help:
.PHONY: install-piptools
install-piptools:
# pip 22.1 broke pip-tools: https://github.com/jazzband/pip-tools/issues/1617
pip install -U pip-tools setuptools wheel "pip>=22.0.3,!=22.1"
python -m pip install -U pip-tools setuptools wheel "pip>=22.0.3,!=22.1"

.PHONY: update_boilerplate
update_boilerplate:
Expand Down
30 changes: 14 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,17 @@
<html>
<p align="center">
<img src="https://raw.githubusercontent.com/flyteorg/static-resources/main/common/flyte_circle_gradient_1_4x4.png" alt="Flyte Logo" width="100">
</p>
<h1 align="center">
Flytekit Python
</h1>
<p align="center">
Flytekit Python is the Python SDK built on top of Flyte
</p>
<h3 align="center">
<a href="plugins/README.md">Plugins</a>
<span> · </span>
<a href="https://docs.flyte.org/projects/flytekit/en/latest/contributing.html">Contribution Guide</a>
</h3>
</html>
<p align="center">
<img src="https://raw.githubusercontent.com/flyteorg/static-resources/main/common/flyte_circle_gradient_1_4x4.png" alt="Flyte Logo" width="100">
</p>
<h1 align="center">
Flytekit Python
</h1>
<p align="center">
Flytekit Python is the Python SDK built on top of Flyte
</p>
<h3 align="center">
<a href="plugins/README.md">Plugins</a>
<span> · </span>
<a href="https://docs.flyte.org/projects/flytekit/en/latest/contributing.html">Contribution Guide</a>
</h3>

[![PyPI version fury.io](https://badge.fury.io/py/flytekit.svg)](https://pypi.python.org/pypi/flytekit/)
[![PyPI download day](https://img.shields.io/pypi/dd/flytekit.svg)](https://pypi.python.org/pypi/flytekit/)
Expand Down
2 changes: 2 additions & 0 deletions dev-requirements.in
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,5 @@ pre-commit
codespell
google-cloud-bigquery
google-cloud-bigquery-storage
IPython
torch
Loading