Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split examples by language #1181

Merged
merged 29 commits into from
Feb 11, 2023
Merged
Changes from 1 commit
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
e9245db
Split python and rust examples into separate folders
emilk Feb 10, 2023
df4737a
objectron: ensure logging reaches the screen
emilk Feb 10, 2023
6e829e4
try fix
emilk Feb 10, 2023
2ae70b9
remove dead shellscript
emilk Feb 10, 2023
497ed70
fix objectron build-script
emilk Feb 10, 2023
a3eeb5a
Fixes
emilk Feb 10, 2023
9f90b82
Fix Rust objectron demo
emilk Feb 10, 2023
1c1b95d
Add readme for rust examples
emilk Feb 10, 2023
63e3e51
lint
emilk Feb 10, 2023
6fa1237
fix mypy
emilk Feb 10, 2023
06b8785
Merge branch 'main' into emilk/split-examples-by-language
teh-cmc Feb 10, 2023
47fcaee
Improve the docs of `rerun` (#1179)
emilk Feb 10, 2023
7a3041d
Add newlines before types, `impl`, `fn`, and docstrings
emilk Feb 10, 2023
2bea9d7
Update examples/requirements.txt
emilk Feb 10, 2023
a5dc541
Mark git directories as safe for our docker_ci container and fix carg…
jleibs Feb 10, 2023
b1557d1
Run cargo check after patching pre-release (#1190)
jleibs Feb 10, 2023
74be952
Allow logging the contents of an image file, not by path (#1193)
emilk Feb 11, 2023
a2a7780
Add --dataset and --resize flags to colmap (#1177)
jleibs Feb 11, 2023
e94670c
Enable `--web-viewer` in the Python Wheels (#1182)
emilk Feb 11, 2023
59cc8b7
Install web dependencies before running cargo check (#1196)
jleibs Feb 11, 2023
8bb3bf6
Update arrow2, arrow2_convert and polars (#1189)
emilk Feb 11, 2023
63244bc
Turn on and fix Python linting of examples (#1191)
emilk Feb 11, 2023
e525e3b
Merge branch 'main' into emilk/split-examples-by-language
emilk Feb 11, 2023
e748f92
Update the paths in .flake8
emilk Feb 11, 2023
e921307
Merge branch 'main' into emilk/split-examples-by-language
teh-cmc Feb 11, 2023
cbf1a51
examples/deep_sdf: sort scenes by relevance instead of lexically + be…
teh-cmc Feb 11, 2023
82b4a59
examples/raw_mesh: sort scenes by relevance instead of lexically + be…
teh-cmc Feb 11, 2023
c37d567
examples/raw_mesh: now offering a higher level --scene flag
teh-cmc Feb 11, 2023
79559b3
Merge remote-tracking branch 'origin/main' into emilk/split-examples-…
teh-cmc Feb 11, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Split python and rust examples into separate folders
emilk committed Feb 10, 2023
commit e9245dbdd41d18042d50dcd0d2cc049f1271c39d
42 changes: 21 additions & 21 deletions .github/workflows/python.yml
Original file line number Diff line number Diff line change
@@ -4,7 +4,7 @@ on:
pull_request:
paths:
- "rerun_py/**"
- "examples/**"
- "examples/python/**"
- ".github/workflows/python.yml"
push:
branches:
@@ -62,26 +62,26 @@ jobs:
matrix: ${{ steps.set-matrix.outputs.matrix }}

steps:
# Sets TAGGED_OR_MAIN if this workflow is running on a tag or the main branch.
- name: Set TAGGED_OR_MAIN
if: startsWith(github.ref, 'refs/tags/v') || github.ref == 'refs/heads/main'
run: echo "TAGGED_OR_MAIN=1" >> $GITHUB_ENV
# Sets TAGGED_OR_MAIN if this workflow is running on a tag or the main branch.
- name: Set TAGGED_OR_MAIN
if: startsWith(github.ref, 'refs/tags/v') || github.ref == 'refs/heads/main'
run: echo "TAGGED_OR_MAIN=1" >> $GITHUB_ENV

- id: set-matrix
shell: bash
run: |
matrix=()
- id: set-matrix
shell: bash
run: |
matrix=()

if [[ $TAGGED_OR_MAIN ]]; then
# MacOS build is really slow (>30 mins); uses up a lot of CI minutes
matrix+=('{"platform": "macos", "runs_on": "macos-latest"},')
fi
matrix+=('{"platform": "windows", "runs_on": "windows-latest-8-cores"},')
matrix+=('{"platform": "linux", "runs_on": "ubuntu-latest-16-cores"}')
if [[ $TAGGED_OR_MAIN ]]; then
# MacOS build is really slow (>30 mins); uses up a lot of CI minutes
matrix+=('{"platform": "macos", "runs_on": "macos-latest"},')
fi
matrix+=('{"platform": "windows", "runs_on": "windows-latest-8-cores"},')
matrix+=('{"platform": "linux", "runs_on": "ubuntu-latest-16-cores"}')

echo "Matrix values: ${matrix[@]}"
echo "Matrix values: ${matrix[@]}"

echo "matrix={\"include\":[${matrix[@]}]}" >> $GITHUB_OUTPUT
echo "matrix={\"include\":[${matrix[@]}]}" >> $GITHUB_OUTPUT

wheels:
name: Build Python Wheels
@@ -128,8 +128,8 @@ jobs:
- name: Generate Embedded RRD file
shell: bash
run: |
pip install -r examples/car/requirements.txt
python examples/car/main.py --headless --save car.rrd
pip install -r examples/python/car/requirements.txt
python examples/python/car/main.py --headless --save car.rrd
cp car.rrd rerun_py/rerun_sdk/rerun_demo/demo.rrd

- name: Build Wheel
@@ -153,8 +153,8 @@ jobs:

- name: Run Car example
run: |
pip install -r examples/car/requirements.txt
./examples/car/main.py --headless
pip install -r examples/python/car/requirements.txt
./examples/python/car/main.py --headless

- name: Upload wheels
uses: actions/upload-artifact@v3
4 changes: 2 additions & 2 deletions .mypy.ini
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[mypy]
files = rerun_py/rerun_sdk/rerun, rerun_py/tests, examples
exclude = examples/objectron/dataset/proto
files = rerun_py/rerun_sdk/rerun, rerun_py/tests, examples/python
exclude = examples/python/objectron/dataset/proto
namespace_packages = True
show_error_codes = True
strict = True
10 changes: 1 addition & 9 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,14 +1,6 @@
[workspace]
resolver = "2"
members = [
"crates/*",
"rerun_py",
"run_wasm",
"examples/api_demo",
"examples/minimal",
"examples/objectron",
"examples/raw_mesh",
]
members = ["crates/*", "examples/rust/*", "rerun_py", "run_wasm"]

[workspace.package]
version = "0.1.0"
36 changes: 1 addition & 35 deletions examples/README.md
Original file line number Diff line number Diff line change
@@ -1,35 +1 @@
# Rerun Examples
The simplest example is [`minimal`](minimal/main.py). You may want to start there!

## Setup
First install the Rerun Python SDK with `pip install rerun-sdk` (coming soon!)

<!-- TODO(emilk): remove the coming soon -->

## Dependencies
Each example comes with its own set of dependencies listed in a `requirements.txt` file. For example, to install dependencies and run the toy `car` example (which doesn't need to download any data) run:

```sh
pip install -r examples/car/requirements.txt
examples/car/main.py
```

You can also install all dependencies needed to run all examples with:

```sh
pip install -r examples/requirements.txt
```

## Running the examples
By default, the examples spawn a Rerun Viewer and stream log data to it.

You can instead save the log data to an `.rrd` file using `examples/car/main.py --save data.rrd`. You can then open that `.rrd` file with `rerun data.rrd`.

(`rerun` is an alias for `python -m rerun`).

NOTE: `.rrd` files do not yet guarantee any backwards or forwards compatibility. One version of Rerun will likely not be able to open an `.rrd` file generated by another Rerun version.

## Contributions welcome
Feel free to open a PR to add a new example!

See [`CONTRIBUTING.md`](../CONTRIBUTING.md) for details on how to contribute.
# Official Rerun examples
1 change: 0 additions & 1 deletion examples/objectron/dataset/.gitignore

This file was deleted.

Empty file.
35 changes: 35 additions & 0 deletions examples/python/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Rerun Examples
The simplest example is [`minimal`](minimal/main.py). You may want to start there!

## Setup
First install the Rerun Python SDK with `pip install rerun-sdk` (coming soon!)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
First install the Rerun Python SDK with `pip install rerun-sdk` (coming soon!)
First install the Rerun Python SDK with `pip install rerun-sdk`.

Might as well pre-emptively get this right.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: if rerun-sdk is already published, why can't I find it on https://pypi.org/project/rerun-sdk/ ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's only been published on testpypi, so pip install -i https://test.pypi.org/simple/ rerun-sdk

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So if pip install rerun-sdk doesn't work yet, then let's not write it yet.


<!-- TODO(emilk): remove the coming soon -->
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
<!-- TODO(emilk): remove the coming soon -->


## Dependencies
Each example comes with its own set of dependencies listed in a `requirements.txt` file. For example, to install dependencies and run the toy `car` example (which doesn't need to download any data) run:

```sh
pip install -r examples/python/car/requirements.txt
examples/python/car/main.py
```

You can also install all dependencies needed to run all examples with:

```sh
pip install -r examples/python/requirements.txt
```

## Running the examples
By default, the examples spawn a Rerun Viewer and stream log data to it.

You can instead save the log data to an `.rrd` file using `examples/python/car/main.py --save data.rrd`. You can then open that `.rrd` file with `rerun data.rrd`.

(`rerun` is an alias for `python -m rerun`).

NOTE: `.rrd` files do not yet guarantee any backwards or forwards compatibility. One version of Rerun will likely not be able to open an `.rrd` file generated by another Rerun version.

## Contributions welcome
Feel free to open a PR to add a new example!

See [`CONTRIBUTING.md`](../CONTRIBUTING.md) for details on how to contribute.
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -2,8 +2,8 @@
"""Minimal examples of Rerun SDK usage.

Example usage:
* Run all demos: `examples/api_demo/main.py`
* Run specific demo: `examples/api_demo/main.py --demo rects`
* Run all demos: `examples/python/api_demo/main.py`
* Run specific demo: `examples/python/api_demo/main.py --demo rects`
"""

import argparse
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -22,7 +22,7 @@
Run:
```sh
# assuming your virtual env is up
examples/deep_sdf/main.py
examples/python/deep_sdf/main.py
```
"""

File renamed without changes.
File renamed without changes.
2 changes: 1 addition & 1 deletion examples/dicom/main.py → examples/python/dicom/main.py
Original file line number Diff line number Diff line change
@@ -3,7 +3,7 @@

Run:
``` sh
python3 examples/dicom/main.py
python3 examples/python/dicom/main.py
```
"""

File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -7,7 +7,7 @@
import requests

DATASET_BASE_URL = "https://storage.googleapis.com/objectron"
LOCAL_DATASET_DIR: Final = Path(os.path.dirname(__file__)) / "downloaded"
LOCAL_DATASET_DIR: Final = Path(os.path.dirname(__file__)) / "dataset"
IMAGE_RESOLUTION: Final = (1440, 1920)
GEOMETRY_FILENAME: Final = "geometry.pbdata"
ANNOTATIONS_FILENAME: Final = "annotation.pbdata"
@@ -29,7 +29,7 @@
def ensure_downloaded(src_url: str, dst_path: Path) -> None:
os.makedirs(dst_path.parent, exist_ok=True)
if not dst_path.exists():
logging.info("Downloading from %s to %s", src_url, dst_path)
logging.info("Downloading %s to %s")
with requests.get(src_url, stream=True) as req:
req.raise_for_status()
with open(dst_path, "wb") as f:
Original file line number Diff line number Diff line change
@@ -3,7 +3,7 @@
"""
Example of using the Rerun SDK to log the Objectron dataset.

Example: `examples/objectron/main.py --recording chair`
Example: `examples/python/objectron/main.py --recording chair`
"""


@@ -18,15 +18,15 @@

import numpy as np
import numpy.typing as npt
from dataset.dataset import (
from download_dataset import (
ANNOTATIONS_FILENAME,
AVAILABLE_RECORDINGS,
GEOMETRY_FILENAME,
IMAGE_RESOLUTION,
LOCAL_DATASET_DIR,
ensure_recording_available,
)
from dataset.proto.objectron.proto import (
from proto.objectron.proto import (
ARCamera,
ARFrame,
ARPointCloud,
File renamed without changes.
File renamed without changes.
File renamed without changes.
2 changes: 1 addition & 1 deletion examples/plots/main.py → examples/python/plots/main.py
Original file line number Diff line number Diff line change
@@ -5,7 +5,7 @@

Run:
```sh
./examples/plot/main.py
./examples/python/plot/main.py
```
"""

File renamed without changes.
Original file line number Diff line number Diff line change
@@ -7,7 +7,7 @@
Run:
```sh
# assuming your virtual env is up
examples/raw_mesh/main.py
examples/python/raw_mesh/main.py
```
"""

File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -5,7 +5,7 @@

Run:
```sh
./examples/text_logging/main.py
./examples/python/text_logging/main.py
```
"""

File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -7,7 +7,7 @@ license.workspace = true
publish = false

[dependencies]
rerun = { path = "../../crates/rerun" }
rerun = { path = "../../../crates/rerun" }

anyhow.workspace = true
clap = { version = "4.0", features = ["derive"] }
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -7,4 +7,4 @@ license.workspace = true
publish = false

[dependencies]
rerun = { path = "../../crates/rerun", features = ["glam"] }
rerun = { path = "../../../crates/rerun", features = ["glam"] }
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -8,7 +8,7 @@ publish = false


[dependencies]
rerun = { path = "../../crates/rerun" }
rerun = { path = "../../../crates/rerun" }

anyhow.workspace = true
clap = { version = "4.0", features = ["derive"] }
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -413,7 +413,7 @@ fn main() -> anyhow::Result<()> {
anyhow::bail!(
"Could not read the recording, have you downloaded the dataset? \
Try running the python version first to download it automatically \
(`examples/objectron/main.py --recording {}`).",
(`examples/python/objectron/main.py --recording {}`).",
args.recording.to_possible_value().unwrap().get_name(),
);
};
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -7,7 +7,7 @@ license.workspace = true
publish = false

[dependencies]
rerun = { path = "../../crates/rerun" }
rerun = { path = "../../../crates/rerun" }

anyhow.workspace = true
bytes = "1.3"
File renamed without changes.
2 changes: 1 addition & 1 deletion justfile
Original file line number Diff line number Diff line change
@@ -54,7 +54,7 @@ py-format:

# Check that all the requirements.txt files for all the examples are correct
py-requirements:
find examples/ -name main.py | xargs -I _ sh -c 'cd $(dirname _) && echo $(pwd) && pip-missing-reqs . || exit 255'
find examples/python/ -name main.py | xargs -I _ sh -c 'cd $(dirname _) && echo $(pwd) && pip-missing-reqs . || exit 255'

# Run linting
py-lint: py-requirements
6 changes: 3 additions & 3 deletions rerun_py/README.md
Original file line number Diff line number Diff line change
@@ -22,7 +22,7 @@ See [`USAGE.md`](USAGE.md).

## Running the example code
```sh
python examples/car/main.py
python examples/python/car/main.py
```

By default, the example runs Rerun in buffered mode, in the same process as the example code. This means all logged data is buffered until `rerun.show()` is called in the end, which shows the viewer and blocks until the viewer is closed.
@@ -58,7 +58,7 @@ just py-test
just py-lint

# Run an example
python examples/car/main.py
python examples/python/car/main.py
```

### Logging and viewing in different processes
@@ -72,7 +72,7 @@ cargo run -p rerun --release

In a second terminal, run the example with the `--connect` option:
```sh
examples/car/main.py --connect
examples/python/car/main.py --connect
```

## Building an installable Python Wheel
2 changes: 1 addition & 1 deletion rerun_py/rerun_sdk/rerun/log/scalar.py
Original file line number Diff line number Diff line change
@@ -31,7 +31,7 @@ def log_scalar(
The current simulation time will be used for the time/X-axis, hence scalars
cannot be timeless!

See also examples/plots.
See also examples/python/plots.

Understanding the plot and attributes hierarchy

7 changes: 3 additions & 4 deletions scripts/check.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
#!/usr/bin/env bash
# This scripts runs various CI-like checks in a convenient way.
# This is likely outdated, but can still be useful.

set -eu
script_path=$( cd "$(dirname "${BASH_SOURCE[0]}")" ; pwd -P )
@@ -26,16 +27,14 @@ cargo doc --document-private-items --no-deps --all-features
(cd crates/re_web_server && cargo check --no-default-features)
(cd crates/re_ws_comms && cargo check --no-default-features)
(cd crates/rerun && cargo check --no-default-features)
(cd examples/nyud && cargo check --no-default-features)
(cd examples/objectron && cargo check --no-default-features)
(cd examples/rust/objectron && cargo check --no-default-features)

(cd crates/re_log_types && cargo check --all-features)
(cd crates/re_viewer && cargo check --all-features)
(cd crates/re_web_server && cargo check --all-features)
(cd crates/re_ws_comms && cargo check --all-features)
(cd crates/rerun && cargo check --all-features)
(cd examples/nyud && cargo check --all-features)
(cd examples/objectron && cargo check --all-features)
(cd examples/rust/objectron && cargo check --all-features)

./scripts/wasm_bindgen_check.sh