chore: pin visualization server python dependencies. Fixes kubeflow#3078

(kubeflow#4310) * build: fix visualization server build failure by adding missing licenses for new deps * wip * chore: pin visualization server python dependencies * updates * update licenses
Jeffwan · Dec 9, 2020 · 941ae35 · 941ae35
1 parent 13c9785
commit 941ae35
Show file tree

Hide file tree

Showing 7 changed files with 442 additions and 19 deletions.
diff --git a/backend/src/apiserver/visualization/README.md b/backend/src/apiserver/visualization/README.md
@@ -170,4 +170,9 @@ within your cluster.
 		grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(maxCallRecvMsgSize)),
 		grpc.WithInsecure(),
 	}
-    ```
+    ```
+
+## Update python dependencies
+1. Edit `requirements.in` with additional changes.
+1. Run `./update_requirements.sh` to re-resolve dependencies.
+1. Pinned dependencies are in `requirements.txt`.
diff --git a/backend/src/apiserver/visualization/requirements.in b/backend/src/apiserver/visualization/requirements.in
@@ -0,0 +1,18 @@
+bokeh==1.2.0
+gcsfs==0.2.3
+google-api-python-client==1.7.*
+itables==0.1.0
+ipykernel==5.1.1
+ipython==7.12.0
+jupyter_client==5.3.*
+nbconvert==5.5.0
+nbformat==4.4.0
+pandas==0.24.2
+pyarrow==0.16.*
+scikit_learn==0.21.2
+tensorflow-metadata==0.22.*
+tensorflow-model-analysis==0.22.*
+tensorflow-data-validation==0.22.*
+tornado==6.0.2
+# Increase google-api-core to some release AFTER 1.21.0 when it becomes available
+google-api-core==1.16.0
diff --git a/backend/src/apiserver/visualization/requirements.txt b/backend/src/apiserver/visualization/requirements.txt
@@ -1,18 +1,145 @@
-bokeh==1.2.0
-gcsfs==0.2.3
-google-api-python-client==1.7.9
-itables==0.1.0
-ipykernel==5.1.1
-ipython==7.12.0
-jupyter_client==5.2.4
-nbconvert==5.5.0
-nbformat==4.4.0
-pandas==0.24.2
-pyarrow==0.15.1
-scikit_learn==0.21.2
-tensorflow-metadata==0.21.1
-tensorflow-model-analysis==0.21.5
-tensorflow-data-validation==0.21.1
-tornado==6.0.2
-# Increase google-api-core to some release AFTER 1.21.0 when it becomes available
-google-api-core==1.16.0
+#
+# This file is autogenerated by pip-compile
+# To update, run:
+#
+#    pip-compile --output-file=- -
+#
+absl-py==0.8.1            # via tensorboard, tensorflow, tensorflow-data-validation, tensorflow-model-analysis, tensorflow-transform, tfx-bsl
+apache-beam[gcp]==2.23.0  # via tensorflow-data-validation, tensorflow-model-analysis, tensorflow-transform, tfx-bsl
+argon2-cffi==20.1.0       # via notebook
+astunparse==1.6.3         # via tensorflow
+attrs==19.3.0             # via jsonschema
+avro-python3==1.9.2.1     # via apache-beam
+backcall==0.2.0           # via ipython
+bleach==3.1.5             # via nbconvert
+bokeh==1.2.0              # via -r -
+cachetools==3.1.1         # via apache-beam, google-auth
+certifi==2020.6.20        # via requests
+cffi==1.14.1              # via argon2-cffi
+chardet==3.0.4            # via requests
+crcmod==1.7               # via apache-beam
+decorator==4.4.2          # via gcsfs, ipython, traitlets
+defusedxml==0.6.0         # via nbconvert
+dill==0.3.1.1             # via apache-beam
+docopt==0.6.2             # via hdfs
+entrypoints==0.3          # via nbconvert
+fastavro==0.23.6          # via apache-beam
+fasteners==0.15           # via google-apitools
+future==0.18.2            # via apache-beam
+gast==0.3.3               # via tensorflow
+gcsfs==0.2.3              # via -r -
+google-api-core[grpc,grpcgcp]==1.16.0  # via -r -, google-cloud-bigquery, google-cloud-bigtable, google-cloud-core, google-cloud-datastore, google-cloud-dlp, google-cloud-language, google-cloud-pubsub, google-cloud-spanner, google-cloud-videointelligence, google-cloud-vision
+google-api-python-client==1.7.12  # via -r -, tfx-bsl
+google-apitools==0.5.31   # via apache-beam
+google-auth-httplib2==0.0.4  # via google-api-python-client
+google-auth-oauthlib==0.4.1  # via gcsfs, tensorboard
+google-auth==1.20.0       # via gcsfs, google-api-core, google-api-python-client, google-auth-httplib2, google-auth-oauthlib, google-cloud-bigquery, tensorboard
+google-cloud-bigquery==1.24.0  # via apache-beam
+google-cloud-bigtable==1.0.0  # via apache-beam
+google-cloud-core==1.3.0  # via apache-beam, google-cloud-bigquery, google-cloud-bigtable, google-cloud-datastore, google-cloud-spanner
+google-cloud-datastore==1.7.4  # via apache-beam
+google-cloud-dlp==0.13.0  # via apache-beam
+google-cloud-language==1.3.0  # via apache-beam
+google-cloud-pubsub==1.0.2  # via apache-beam
+google-cloud-spanner==1.13.0  # via apache-beam
+google-cloud-videointelligence==1.13.0  # via apache-beam
+google-cloud-vision==0.42.0  # via apache-beam
+google-pasta==0.2.0       # via tensorflow
+google-resumable-media==0.5.1  # via google-cloud-bigquery
+googleapis-common-protos[grpc]==1.52.0  # via google-api-core, grpc-google-iam-v1, tensorflow-metadata
+grpc-google-iam-v1==0.12.3  # via google-cloud-bigtable, google-cloud-pubsub, google-cloud-spanner
+grpcio-gcp==0.2.2         # via apache-beam, google-api-core
+grpcio==1.30.0            # via apache-beam, google-api-core, googleapis-common-protos, grpc-google-iam-v1, grpcio-gcp, tensorboard, tensorflow, tensorflow-serving-api
+h5py==2.10.0              # via tensorflow
+hdfs==2.5.8               # via apache-beam
+httplib2==0.17.4          # via apache-beam, google-api-python-client, google-apitools, google-auth-httplib2, oauth2client
+idna==2.10                # via requests
+importlib-metadata==1.7.0  # via jsonschema, markdown
+ipykernel==5.1.1          # via -r -, ipywidgets, jupyter, jupyter-console, notebook, qtconsole
+ipython-genutils==0.2.0   # via nbformat, notebook, qtconsole, traitlets
+ipython==7.12.0           # via -r -, ipykernel, ipywidgets, jupyter-console
+ipywidgets==7.5.1         # via jupyter, tensorflow-model-analysis
+itables==0.1.0            # via -r -
+jedi==0.17.2              # via ipython
+jinja2==2.11.2            # via bokeh, nbconvert, notebook
+joblib==0.14.1            # via scikit-learn, tensorflow-data-validation
+jsonschema==3.2.0         # via nbformat
+jupyter-client==5.3.5     # via -r -, ipykernel, jupyter-console, notebook, qtconsole
+jupyter-console==6.1.0    # via jupyter
+jupyter-core==4.6.3       # via jupyter-client, nbconvert, nbformat, notebook, qtconsole
+jupyter==1.0.0            # via tensorflow-model-analysis
+keras-preprocessing==1.1.2  # via tensorflow
+markdown==3.2.2           # via tensorboard
+markupsafe==1.1.1         # via jinja2
+mistune==0.8.4            # via nbconvert
+mock==2.0.0               # via apache-beam
+monotonic==1.5            # via fasteners
+nbconvert==5.5.0          # via -r -, jupyter, notebook
+nbformat==4.4.0           # via -r -, ipywidgets, nbconvert, notebook
+notebook==6.1.0           # via jupyter, widgetsnbextension
+numpy==1.19.1             # via apache-beam, bokeh, h5py, keras-preprocessing, opt-einsum, pandas, pyarrow, scikit-learn, scipy, tensorboard, tensorflow, tensorflow-data-validation, tensorflow-model-analysis, tensorflow-transform, tfx-bsl
+oauth2client==3.0.0       # via apache-beam, google-apitools
+oauthlib==3.1.0           # via requests-oauthlib
+opt-einsum==3.3.0         # via tensorflow
+packaging==20.4           # via bleach, bokeh
+pandas==0.24.2            # via -r -, itables, tensorflow-data-validation, tensorflow-model-analysis, tfx-bsl
+pandocfilters==1.4.2      # via nbconvert
+parso==0.7.1              # via jedi
+pbr==5.4.5                # via mock
+pexpect==4.8.0            # via ipython
+pickleshare==0.7.5        # via ipython
+pillow==7.2.0             # via bokeh
+prometheus-client==0.8.0  # via notebook
+prompt-toolkit==3.0.5     # via ipython, jupyter-console
+protobuf==3.12.4          # via apache-beam, google-api-core, google-cloud-bigquery, googleapis-common-protos, tensorboard, tensorflow, tensorflow-data-validation, tensorflow-metadata, tensorflow-model-analysis, tensorflow-serving-api, tensorflow-transform, tfx-bsl
+ptyprocess==0.6.0         # via pexpect, terminado
+pyarrow==0.16.0           # via -r -, apache-beam, tensorflow-data-validation, tensorflow-model-analysis, tfx-bsl
+pyasn1-modules==0.2.8     # via google-auth, oauth2client
+pyasn1==0.4.8             # via oauth2client, pyasn1-modules, rsa
+pycparser==2.20           # via cffi
+pydot==1.4.1              # via apache-beam, tensorflow-transform
+pygments==2.6.1           # via ipython, jupyter-console, nbconvert, qtconsole
+pymongo==3.11.0           # via apache-beam
+pyparsing==2.4.7          # via packaging, pydot
+pyrsistent==0.16.0        # via jsonschema
+python-dateutil==2.8.1    # via apache-beam, bokeh, jupyter-client, pandas
+pytz==2020.1              # via apache-beam, fastavro, google-api-core, pandas
+pyyaml==5.3.1             # via bokeh
+pyzmq==19.0.2             # via jupyter-client, notebook, qtconsole
+qtconsole==4.7.5          # via jupyter
+qtpy==1.9.0               # via qtconsole
+requests-oauthlib==1.3.0  # via google-auth-oauthlib
+requests==2.24.0          # via gcsfs, google-api-core, hdfs, requests-oauthlib, tensorboard
+rsa==4.6                  # via google-auth, oauth2client
+scikit_learn==0.21.2      # via -r -
+scipy==1.4.1              # via scikit-learn, tensorflow, tensorflow-model-analysis
+send2trash==1.5.0         # via notebook
+six==1.15.0               # via absl-py, argon2-cffi, astunparse, bleach, bokeh, fasteners, google-api-core, google-api-python-client, google-apitools, google-auth, google-auth-httplib2, google-cloud-bigquery, google-pasta, google-resumable-media, grpcio, h5py, hdfs, jsonschema, keras-preprocessing, mock, oauth2client, packaging, protobuf, pyarrow, pyrsistent, python-dateutil, tensorboard, tensorflow, tensorflow-data-validation, tensorflow-model-analysis, tensorflow-transform, tfx-bsl, traitlets
+tensorboard-plugin-wit==1.7.0  # via tensorboard
+tensorboard==2.2.2        # via tensorflow
+tensorflow-data-validation==0.22.2  # via -r -
+tensorflow-estimator==2.2.0  # via tensorflow
+tensorflow-metadata==0.22.2  # via -r -, tensorflow-data-validation, tensorflow-model-analysis, tensorflow-transform, tfx-bsl
+tensorflow-model-analysis==0.22.2  # via -r -
+tensorflow-serving-api==2.2.0  # via tfx-bsl
+tensorflow-transform==0.22.0  # via tensorflow-data-validation
+tensorflow==2.2.0         # via tensorflow-data-validation, tensorflow-model-analysis, tensorflow-serving-api, tensorflow-transform, tfx-bsl
+termcolor==1.1.0          # via tensorflow
+terminado==0.8.3          # via notebook
+testpath==0.4.4           # via nbconvert
+tfx-bsl==0.22.1           # via tensorflow-data-validation, tensorflow-model-analysis, tensorflow-transform
+tornado==6.0.2            # via -r -, bokeh, ipykernel, jupyter-client, notebook, terminado
+traitlets==4.3.3          # via ipykernel, ipython, ipywidgets, jupyter-client, jupyter-core, nbconvert, nbformat, notebook, qtconsole
+typing-extensions==3.7.4.2  # via apache-beam
+uritemplate==3.0.1        # via google-api-python-client
+urllib3==1.25.10          # via requests
+wcwidth==0.2.5            # via prompt-toolkit
+webencodings==0.5.1       # via bleach
+werkzeug==1.0.1           # via tensorboard
+wheel==0.34.2             # via astunparse, tensorboard, tensorflow
+widgetsnbextension==3.5.1  # via ipywidgets
+wrapt==1.12.1             # via tensorflow
+zipp==3.1.0               # via importlib-metadata
+
+# The following packages are considered to be unsafe in a requirements file:
+# setuptools
diff --git a/backend/src/apiserver/visualization/third_party_licenses.csv b/backend/src/apiserver/visualization/third_party_licenses.csv
@@ -143,3 +143,5 @@ QtPy,https://raw.githubusercontent.com/spyder-ide/qtpy/master/LICENSE.txt,MIT
 argon2-cffi,https://raw.githubusercontent.com/hynek/argon2-cffi/master/LICENSE,MIT
 pycparser,https://raw.githubusercontent.com/eliben/pycparser/master/LICENSE,BSD
 cffi,https://raw.githubusercontent.com/cffi/cffi/master/COPYRIGHT,MIT
+astunparse,https://raw.githubusercontent.com/simonpercivall/astunparse/master/LICENSE,PSF
+tensorboard-plugin-wit,https://raw.githubusercontent.com/PAIR-code/what-if-tool/master/LICENSE,Apache 2.0
diff --git a/backend/src/apiserver/visualization/third_party_licenses/astunparse.LICENSE b/backend/src/apiserver/visualization/third_party_licenses/astunparse.LICENSE
@@ -0,0 +1,64 @@
+LICENSE
+=======
+
+Copyright (c) 2014, Simon Percivall
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
+
+* Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
+
+* Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
+
+* Neither the name of AST Unparser nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+
+PYTHON SOFTWARE FOUNDATION LICENSE VERSION 2
+--------------------------------------------
+
+1. This LICENSE AGREEMENT is between the Python Software Foundation
+("PSF"), and the Individual or Organization ("Licensee") accessing and
+otherwise using this software ("Python") in source or binary form and
+its associated documentation.
+
+2. Subject to the terms and conditions of this License Agreement, PSF hereby
+grants Licensee a nonexclusive, royalty-free, world-wide license to reproduce,
+analyze, test, perform and/or display publicly, prepare derivative works,
+distribute, and otherwise use Python alone or in any derivative version,
+provided, however, that PSF's License Agreement and PSF's notice of copyright,
+i.e., "Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010,
+2011, 2012, 2013, 2014 Python Software Foundation; All Rights Reserved" are retained
+in Python alone or in any derivative version prepared by Licensee.
+
+3. In the event Licensee prepares a derivative work that is based on
+or incorporates Python or any part thereof, and wants to make
+the derivative work available to others as provided herein, then
+Licensee hereby agrees to include in any such work a brief summary of
+the changes made to Python.
+
+4. PSF is making Python available to Licensee on an "AS IS"
+basis.  PSF MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
+IMPLIED.  BY WAY OF EXAMPLE, BUT NOT LIMITATION, PSF MAKES NO AND
+DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
+FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON WILL NOT
+INFRINGE ANY THIRD PARTY RIGHTS.
+
+5. PSF SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON
+FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS
+A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON,
+OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.
+
+6. This License Agreement will automatically terminate upon a material
+breach of its terms and conditions.
+
+7. Nothing in this License Agreement shall be deemed to create any
+relationship of agency, partnership, or joint venture between PSF and
+Licensee.  This License Agreement does not grant permission to use PSF
+trademarks or trade name in a trademark sense to endorse or promote
+products or services of Licensee, or any third party.
+
+8. By copying, installing or otherwise using Python, Licensee
+agrees to be bound by the terms and conditions of this License
+Agreement.