Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Debug converter python code with debugpy #7283

Merged
merged 4 commits into from
Jan 19, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions .bazelrc
Original file line number Diff line number Diff line change
Expand Up @@ -69,15 +69,21 @@ build --action_env=BROWSERSTACK_USERNAME --action_env=BROWSERSTACK_KEY
run --action_env=BROWSERSTACK_USERNAME --action_env=BROWSERSTACK_KEY
test --action_env=BROWSERSTACK_USERNAME --action_env=BROWSERSTACK_KEY

# Make python debugging refer to the real files instead of symlinks
run --action_env=PYDEVD_RESOLVE_SYMLINKS=true

# Platform specific DISPLAY environment variable for webgl and headless setting
# for browser tests.
test:linux --test_env=DISPLAY
test:macos --define DISPLAY=true
test:windows --define DISPLAY=true --//:headless=false

# Enable debugging tests with --config=debug
run:debug --test_arg=--node_options=--inspect-brk --test_output=streamed --test_strategy=exclusive --test_timeout=9999 --nocache_test_results
test:debug --test_arg=--node_options=--inspect-brk --test_output=streamed --test_strategy=exclusive --test_timeout=9999 --nocache_test_results
run:debug --test_arg=--node_options=--inspect-brk --test_output=streamed --test_strategy=exclusive --test_timeout=999999 --nocache_test_results
test:debug --test_arg=--node_options=--inspect-brk --test_output=streamed --test_strategy=exclusive --test_timeout=999999 --nocache_test_results

test:debugpy --test_output=streamed --test_strategy=exclusive --test_timeout=999999 --nocache_test_results
run:debugpy --test_output=streamed --test_strategy=exclusive --test_timeout=999999 --nocache_test_results

# Load any settings specific to the current user.
# .bazelrc.user should appear in .gitignore so that settings are not shared with
Expand Down
13 changes: 13 additions & 0 deletions tfjs-converter/.vscode/launch.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{
"version": "0.2.0",
"configurations": [
{
"name": "Python: Attach",
"type": "python",
"request": "attach",
"port": 5724,
"host": "localhost"
}
],
"compounds": []
}
54 changes: 54 additions & 0 deletions tfjs-converter/python/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,57 @@ Use following command to install the library with support of interactive CLI:
```bash
pip install tensorflowjs[wizard]
```

Then, run the following to see a list of CLI options

```bash
tensorflowjs_converter --help
```

or, use the wizard

```bash
tensorflowjs_wizard
```

Alternatively, run the converter via its Bazel target. This must be run from withing the tfjs repo:

```bash
yarn bazel run //tfjs-converter/python/tensorflowjs/converters:converter -- --help
```

## Development

The python tests are run with Bazel.

```bash
yarn bazel test //tfjs-converter/python/...
```

Alternatively, run `yarn run-python-tests` to run the above command.

To debug a specific test case, use the `--test_filter` option. For example,

```bash
yarn bazel test //tfjs-converter/python/tensorflowjs/converters:tf_saved_model_conversion_v2_test --test_filter=ConvertTest.test_convert_saved_model_v1
```

Interactive debugging with breakpoints is supported by `debugpy` in VSCode.
To enable debugging, put this code at the top of the test file you want to
debug.

```python
import debugpy
debugpy.listen(('localhost', 5724))
print("Waiting for debugger to connect. See tfjs-converter python README")
debugpy.wait_for_client()
```

You may also need to add the following dependency to the test target in the
Bazel `BUILD` file if it's not already present.
```starlark
"//tfjs-converter/python/tensorflowjs:expect_debugpy_installed"
```

Then, run the test with `bazel run --config=debugpy` and connect
the VSCode debugger by selecting the `Python: Attach (Converter)` option.
1 change: 1 addition & 0 deletions tfjs-converter/python/requirements-dev.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,4 @@
PyInquirer==1.0.3
pylint==2.5.0; python_version > '3.0'
setuptools==65.6.3
debugpy==1.6.5
20 changes: 20 additions & 0 deletions tfjs-converter/python/requirements-dev_lock.txt
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,26 @@ cycler==0.11.0 \
--hash=sha256:3a27e95f763a428a739d2add979fa7494c912a32c17c4c38c4d5f082cad165a3 \
--hash=sha256:9c87405839a19696e837b3b818fed3f5f69f16f1eec1a1ad77e043dcea9c772f
# via matplotlib
debugpy==1.6.5 \
--hash=sha256:048368f121c08b00bbded161e8583817af5055982d2722450a69efe2051621c2 \
--hash=sha256:0f9afcc8cad6424695f3356dc9a7406d5b18e37ee2e73f34792881a44b02cc50 \
--hash=sha256:15bc5febe0edc79726517b1f8d57d7ac7c784567b5ba804aab8b1c9d07a57018 \
--hash=sha256:17039e392d6f38388a68bd02c5f823b32a92142a851e96ba3ec52aeb1ce9d900 \
--hash=sha256:286ae0c2def18ee0dc8a61fa76d51039ca8c11485b6ed3ef83e3efe8a23926ae \
--hash=sha256:377391341c4b86f403d93e467da8e2d05c22b683f08f9af3e16d980165b06b90 \
--hash=sha256:500dd4a9ff818f5c52dddb4a608c7de5371c2d7d905c505eb745556c579a9f11 \
--hash=sha256:5e55e6c79e215239dd0794ee0bf655412b934735a58e9d705e5c544f596f1603 \
--hash=sha256:62a06eb78378292ba6c427d861246574dc8b84471904973797b29dd33c7c2495 \
--hash=sha256:696165f021a6a17da08163eaae84f3faf5d8be68fb78cd78488dd347e625279c \
--hash=sha256:74e4eca42055759032e3f1909d1374ba1d729143e0c2729bb8cb5e8b5807c458 \
--hash=sha256:7e84d9e4420122384cb2cc762a00b4e17cbf998022890f89b195ce178f78ff47 \
--hash=sha256:8116e40a1cd0593bd2aba01d4d560ee08f018da8e8fbd4cbd24ff09b5f0e41ef \
--hash=sha256:8f3fab217fe7e2acb2d90732af1a871947def4e2b6654945ba1ebd94bd0bea26 \
--hash=sha256:947c686e8adb46726f3d5f19854f6aebf66c2edb91225643c7f44b40b064a235 \
--hash=sha256:9984fc00ab372c97f63786c400107f54224663ea293daab7b365a5b821d26309 \
--hash=sha256:9e809ef787802c808995e5b6ade714a25fa187f892b41a412d418a15a9c4a432 \
--hash=sha256:b5a74ecebe5253344501d9b23f74459c46428b30437fa9254cfb8cb129943242
# via -r tfjs-converter/python/requirements-dev.txt
dm-tree==0.1.7 \
--hash=sha256:0f01743cc2247170e64798c6b4b31853717054bf9ceec47a1b1b8c2a4baf5792 \
--hash=sha256:1379a02df36e2bbff9819ceafa55ccd436b15af398803f781f372f8ead7ed871 \
Expand Down
6 changes: 6 additions & 0 deletions tfjs-converter/python/tensorflowjs/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,12 @@ py_library(
deps = [requirement("packaging")],
)

py_library(
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note to reviewer: debugpy is already in google3, so this shouldn't break copybara.

name = "expect_debugpy_installed",
# debugpy is used for interactive debugging with vscode
deps = [requirement("debugpy")],
)

py_library(
name = "quantization",
srcs = ["quantization.py"],
Expand Down