Skip to content

Commit

Permalink
Update Colab setup steps and documentation
Browse files Browse the repository at this point in the history
* Add colab/README.md with "Open in Colab" links for each notebook
* Add notes on installing tf-nightly to getting started docs
* Document using Bazelisk to manage Bazel versions supported by TensorFlow

Relates to #15

Closes #162

COPYBARA_INTEGRATE_REVIEW=#162 from google:scotttodd-colab 95d9917
PiperOrigin-RevId: 282673738
  • Loading branch information
ScottTodd authored and iree-copybara-bot committed Nov 27, 2019
1 parent f80505e commit 619c420
Show file tree
Hide file tree
Showing 6 changed files with 95 additions and 3 deletions.
1 change: 1 addition & 0 deletions .bazelversion
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
1.1.0
7 changes: 6 additions & 1 deletion bindings/python/pyiree/tf_interop/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,11 @@ cc_library(
],
)

# Runtime deps needed to compile the tensorflow compiler.
SAVED_MODEL_TF_RUNTIME_DEPS = [
"@org_tensorflow//tensorflow/core:ops",
]

TF_XLA_PASS_DEPS = [
"@org_tensorflow//tensorflow/compiler/mlir/xla:xla_legalize_tf",
]
Expand All @@ -77,7 +82,7 @@ cc_library(
"@org_tensorflow//tensorflow/compiler/mlir/tensorflow",
"@org_tensorflow//tensorflow/compiler/mlir/tensorflow:convert_graphdef",
"//bindings/python/pyiree:base",
] + TF_XLA_PASS_DEPS,
] + SAVED_MODEL_TF_RUNTIME_DEPS + TF_XLA_PASS_DEPS,
)

cc_library(
Expand Down
17 changes: 15 additions & 2 deletions build_tools/scripts/start_colab_kernel.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,30 +30,43 @@
# Setup colab (https://research.google.com/colaboratory/local-runtimes.html)
# python3 -m pip install jupyter_http_over_ws
# jupyter serverextension enable --py jupyter_http_over_ws
# If you plan on using TensorFlow, enable the TensorFlow parts of IREE's
# compiler by adding a define to your user.bazelrc file:
# build --define=iree_tensorflow=true

import os
import subprocess
import shutil
import sys

repo_root = None
bazel_env = dict(os.environ)
bazel_bin = None
bazel_exe = None


def setup_environment():
"""Sets up some environment globals."""
global bazel_bin
global repo_root
global bazel_exe

# Determine the repository root (two dir-levels up).
repo_root = os.path.dirname(
os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
print("Repository root: %s" % (repo_root,))

# Use 'bazelisk' instead of 'bazel' if it exists on the path.
# Bazelisk is an optional utility that pick versions of Bazel to use and
# passes through all command-line arguments to the real Bazel binary:
# https://github.com/bazelbuild/bazelisk
bazel_exe = "bazelisk" if shutil.which("bazelisk") else "bazel"
print("Using bazel executable: %s" % (bazel_exe))

# Detect python and query bazel for its output.
print("Setting Bazel PYTHON_BIN=%s" % (sys.executable,))
bazel_env["PYTHON_BIN"] = sys.executable
bazel_bin = subprocess.check_output(["bazel", "info", "bazel-bin"],
bazel_bin = subprocess.check_output([bazel_exe, "info", "bazel-bin"],
cwd=repo_root,
env=bazel_env).decode("utf-8")
bazel_bin = bazel_bin.splitlines()[0]
Expand All @@ -69,7 +82,7 @@ def build():
"""Builds the python bundle."""
print("Building python bindings...")
subprocess.check_call(
["bazel", "build", "//bindings/python/pyiree:everything_for_colab"],
[bazel_exe, "build", "//bindings/python/pyiree:everything_for_colab"],
cwd=repo_root,
env=bazel_env)

Expand Down
49 changes: 49 additions & 0 deletions colab/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
<!--
Copyright 2019 Google LLC
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
https://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->

# Google Colaboratory (Colab) Notebooks

To run these notebooks with a local runtime, refer to the
[Using Colab docs](../docs/using_colab.md).

## Notebooks

### [low_level_invoke_function\.ipynb](low_level_invoke_function.ipynb)

Shows off some concepts of the low level IREE python bindings

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/google/iree/blob/master/colab/low_level_invoke_function.ipynb)

### [simple_tensorflow_module_import\.ipynb](simple_tensorflow_module_import.ipynb)

Defines a simple TF module, saves it and loads it in IREE

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/google/iree/blob/master/colab/simple_tensorflow_module_import.ipynb)

## Working with GitHub

Refer to
[Colab's GitHub demo](https://colab.research.google.com/github/googlecolab/colabtools/blob/master/notebooks/colab-github-demo.ipynb)
for general information about using Colab with GitHub.

To make changes to a notebook in this repository, one possible workflow is:

* Open or create the notebook in Colab
* Connect to your local runtime
* Make your changes, run the notebook, etc.
* Download the modified notebook using `File > Download .ipynb`
* Move the downloaded notebook file into a clone of this repository and submit
a pull request
12 changes: 12 additions & 0 deletions docs/getting_started_on_linux.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,12 @@ Follow the
[install instructions](https://docs.bazel.build/versions/master/install-ubuntu.html)
and verify with: `bazel --version`.

If you are planning on using TensorFlow/Colab, make sure that the Bazel version
you install is
[supported by TensorFlow](https://www.tensorflow.org/install/source#install_bazel).
A .bazelversion file is also provided if you want to use
[Bazelisk](https://github.com/bazelbuild/bazelisk) to manage Bazel versions.

### Install clang

```
Expand Down Expand Up @@ -62,6 +68,12 @@ Install packages:
sudo pip3 install numpy
```

If using Colab, you may also want to install TensorFlow:

```shell
sudo pip3 install tf-nightly
```

### Install the Vulkan SDK

Some parts of the project link against the Vulkan SDK and require it be
Expand Down
12 changes: 12 additions & 0 deletions docs/getting_started_on_windows.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,12 @@ scoop install vscode

TODO: Upgrade to head python.

If you are planning on using TensorFlow/Colab, make sure that the Bazel version
you install is
[supported by TensorFlow](https://www.tensorflow.org/install/source#install_bazel).
A .bazelversion file is also provided if you want to use
[Bazelisk](https://github.com/bazelbuild/bazelisk) to manage Bazel versions.

## Setup MSYS2

Then run msys2 for the first time for subsequent setup by doing one of:
Expand Down Expand Up @@ -280,3 +286,9 @@ python ~/Downloads/get-pip.py
```shell
pip install numpy
```

If using Colab, you may also want to install TensorFlow:

```shell
pip install tf-nightly
```

0 comments on commit 619c420

Please sign in to comment.