Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[python] Split Python wheel in two separate wheels #20054

Merged
merged 13 commits into from
Jul 13, 2022
Merged

[python] Split Python wheel in two separate wheels #20054

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
d948299
Move package information into manifest file
agners Jun 28, 2022
4e1bda1
Specify scripts in manifest as well
agners Jun 28, 2022
a495c82
Make build-chip-wheel.py independent of build variant
agners Jun 28, 2022
495fbc9
Move package requirements to manifest file as well
agners Jun 28, 2022
0719bd0
Split Python into library and repl package
agners Jun 28, 2022
662c5bb
Introduce common GN template chip_python_wheel_action
agners Jun 28, 2022
8da7f15
Add runtime dependency between chip-repl and chip-library
agners Jun 29, 2022
be45260
Split chip-library into chip-core and chip-clusters
agners Jun 29, 2022
6f69fc1
Apply restyled
agners Jun 29, 2022
e410d7e
Install all wheels for unit tests
agners Jul 5, 2022
a302ad4
Use new chip-repl build target
agners Jul 5, 2022
1f02266
Fix Cirque failures
agners Jul 13, 2022
97231f0
Fix wheel names for chip_clusters/chip_repl
agners Jul 13, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .github/workflows/build.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -316,7 +316,9 @@ jobs:
- name: Run Python library specific unit tests
timeout-minutes: 5
run: |
scripts/run_in_build_env.sh 'pip3 install ./out/controller/python/chip-0.0-cp37-abi3-linux_x86_64.whl'
scripts/run_in_build_env.sh 'pip3 install ./out/controller/python/chip_core-0.0-cp37-abi3-linux_x86_64.whl'
scripts/run_in_build_env.sh 'pip3 install ./out/controller/python/chip_clusters-0.0-py3-none-any.whl'
scripts/run_in_build_env.sh 'pip3 install ./out/controller/python/chip_repl-0.0-py3-none-any.whl'
scripts/run_in_build_env.sh '(cd src/controller/python/test/unit_tests/ && python3 -m unittest -v)'

build_darwin:
Expand Down
4 changes: 2 additions & 2 deletions BUILD.gn
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,7 @@ if (current_toolchain != "${dir_pw_toolchain}/default:default") {
if (enable_pylib) {
deps += [ "${chip_root}/src/pybindings/pycontroller" ]
}
deps += [ "${chip_root}/src/controller/python" ]
deps += [ "${chip_root}/src/controller/python:chip-repl" ]
agners marked this conversation as resolved.
Show resolved Hide resolved
}
}

Expand All @@ -166,7 +166,7 @@ if (current_toolchain != "${dir_pw_toolchain}/default:default") {
if (enable_pylib) {
data_deps += [ "${chip_root}/src/pybindings/pycontroller" ]
}
data_deps += [ "${chip_root}/src/controller/python" ]
data_deps += [ "${chip_root}/src/controller/python:chip-repl" ]
}

write_runtime_deps = "${root_out_dir}/certification.runtime_deps"
Expand Down
2 changes: 1 addition & 1 deletion scripts/build/builders/host.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -273,7 +273,7 @@ def __init__(self, root, runner, app: HostApp, board=HostBoard.NATIVE,
elif app == HostApp.PYTHON_BINDINGS:
self.extra_gn_options.append('enable_rtti=false')
self.extra_gn_options.append('chip_project_config_include_dirs=["//config/python"]')
self.build_command = 'python'
self.build_command = 'chip-repl'

def GnBuildArgs(self):
if self.board == HostBoard.NATIVE:
Expand Down
4 changes: 2 additions & 2 deletions scripts/build/testdata/build_linux_on_x64.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -359,7 +359,7 @@ ninja -C {out}/linux-arm64-ota-requestor
ninja -C {out}/linux-arm64-ota-requestor-ipv6only

# Building linux-arm64-python-bindings
ninja -C {out}/linux-arm64-python-bindings python
ninja -C {out}/linux-arm64-python-bindings chip-repl

# Building linux-arm64-shell
ninja -C {out}/linux-arm64-shell
Expand Down Expand Up @@ -470,7 +470,7 @@ ninja -C {out}/linux-x64-ota-requestor
ninja -C {out}/linux-x64-ota-requestor-ipv6only

# Building linux-x64-python-bindings
ninja -C {out}/linux-x64-python-bindings python
ninja -C {out}/linux-x64-python-bindings chip-repl

# Building linux-x64-rpc-console
ninja -C {out}/linux-x64-rpc-console
Expand Down
10 changes: 5 additions & 5 deletions scripts/build_python.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -120,13 +120,13 @@ gn --root="$CHIP_ROOT" gen "$OUTPUT_ROOT" --args="chip_detail_logging=$chip_deta
if [ "$enable_pybindings" == true ]; then
ninja -C "$OUTPUT_ROOT" pycontroller
else
ninja -C "$OUTPUT_ROOT" python
ninja -C "$OUTPUT_ROOT" chip-repl
fi

if [ "$enable_pybindings" == true ]; then
WHEEL=$(ls "$OUTPUT_ROOT"/pybindings/pycontroller/pychip-*.whl | head -n 1)
WHEEL=("$OUTPUT_ROOT"/pybindings/pycontroller/pychip-*.whl)
else
WHEEL=$(ls "$OUTPUT_ROOT"/controller/python/chip-*.whl | head -n 1)
WHEEL=("$OUTPUT_ROOT"/controller/python/chip*.whl)
fi

if [ "$install_wheel" = "no" ]; then
Expand All @@ -137,7 +137,7 @@ elif [ "$install_wheel" = "separate" ]; then

source "$ENVIRONMENT_ROOT"/bin/activate
"$ENVIRONMENT_ROOT"/bin/python -m pip install --upgrade pip
"$ENVIRONMENT_ROOT"/bin/pip install --upgrade --force-reinstall --no-cache-dir "$WHEEL"
"$ENVIRONMENT_ROOT"/bin/pip install --upgrade --force-reinstall --no-cache-dir "${WHEEL[@]}"

echo ""
echo_green "Compilation completed and WHL package installed in: "
Expand All @@ -146,7 +146,7 @@ elif [ "$install_wheel" = "separate" ]; then
echo_green "To use please run:"
echo_bold_white " source $ENVIRONMENT_ROOT/bin/activate"
elif [ "$install_wheel" = "build-env" ]; then
pip install --force-reinstall "$WHEEL"
pip install --force-reinstall "${WHEEL[@]}"

echo ""
echo_green "Compilation completed and WHL package installed in virtualenv for building sdk"
Expand Down
8 changes: 4 additions & 4 deletions scripts/build_python_device.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ gn --root="$CHIP_ROOT" gen "$OUTPUT_ROOT" --args="chip_detail_logging=$chip_deta
if [ "$enable_pybindings" == true ]; then
ninja -v -C "$OUTPUT_ROOT" pycontroller
else
ninja -v -C "$OUTPUT_ROOT" python
ninja -v -C "$OUTPUT_ROOT" chip-library
fi

# Create a virtual environment that has access to the built python tools
Expand All @@ -113,14 +113,14 @@ virtualenv --clear "$ENVIRONMENT_ROOT"
# Activate the new environment to register the python WHL

if [ "$enable_pybindings" == true ]; then
WHEEL=$(ls "$OUTPUT_ROOT"/pybindings/pycontroller/pychip-*.whl | head -n 1)
WHEEL=("$OUTPUT_ROOT"/pybindings/pycontroller/pychip-*.whl)
else
WHEEL=$(ls "$OUTPUT_ROOT"/controller/python/chip-*.whl | head -n 1)
WHEEL=("$OUTPUT_ROOT"/controller/python/chip_library*.whl)
fi

source "$ENVIRONMENT_ROOT"/bin/activate
"$ENVIRONMENT_ROOT"/bin/python -m pip install --upgrade pip
"$ENVIRONMENT_ROOT"/bin/pip install --upgrade --force-reinstall --no-cache-dir "$WHEEL"
"$ENVIRONMENT_ROOT"/bin/pip install --upgrade --force-reinstall --no-cache-dir "${WHEEL[@]}"

echo ""
echo_green "Compilation completed and WHL package installed in: "
Expand Down
Loading