Skip to content

Commit

Permalink
Separate out python dependencies for running scripts (project-chip#28582
Browse files Browse the repository at this point in the history
)

* Separate oput requirements for various python scripts

* Update certbins to install requirements for test script running instead of hard-coding dependencies

* Update text for help in build_python

* Fix the requirements install - need to copy the dependencies locally

* Update the sha to something that supports the latest options. This is the TE2 for 1.2 SHA

* A few more dependencies for py_matter_yamltests ... dependencies there seem off

* Revert submodule updates

* Update dependencies for chip repl to allow py_matter_idl

* Fix dependencies in yaml: need dependencies at runtime not at build time

---------

Co-authored-by: Andrei Litvin <[email protected]>
  • Loading branch information
2 people authored and Alexi Poth committed Aug 15, 2023
1 parent 9ebc9f7 commit 0c90669
Show file tree
Hide file tree
Showing 8 changed files with 28 additions and 22 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -427,7 +427,7 @@ jobs:
- name: Build Python REPL and example apps
run: |
scripts/run_in_build_env.sh './scripts/build_python.sh --install_virtual_env out/venv --extra_packages "mobly"'
scripts/run_in_build_env.sh './scripts/build_python.sh --install_virtual_env out/venv'
./scripts/run_in_build_env.sh \
"./scripts/build/build_examples.py \
--target linux-x64-all-clusters-ipv6only-no-ble-no-wifi-tsan-clang-test \
Expand Down
10 changes: 8 additions & 2 deletions integrations/docker/images/chip-cert-bins/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ FROM ubuntu:22.04 as chip-build-cert
LABEL org.opencontainers.image.source https://github.com/project-chip/connectedhomeip
ARG TARGETPLATFORM
# COMMITHASH defines the target commit to build from. May be passed in using --build-arg.
ARG COMMITHASH=7b99e6399c6069037c613782d78132c69b9dcabb
ARG COMMITHASH=c1ec2d777456924dcaa59b53351b00d73caf378f

# Ensure TARGETPLATFORM is set
RUN case ${TARGETPLATFORM} in \
Expand Down Expand Up @@ -268,5 +268,11 @@ COPY --from=chip-build-cert-bins /root/connectedhomeip/out/chip-app1 chip-app1
# Stage 3.1 Setup the Matter Python environment
COPY --from=chip-build-cert-bins /root/connectedhomeip/out/python_lib python_lib
COPY --from=chip-build-cert-bins /root/connectedhomeip/src/python_testing python_testing
RUN pip install click websockets lark diskcache

COPY --from=chip-build-cert-bins /root/connectedhomeip/scripts/tests/requirements.txt /tmp/requirements.txt
RUN pip install -r /tmp/requirements.txt && rm /tmp/requirements.txt

COPY --from=chip-build-cert-bins /root/connectedhomeip/src/python_testing/requirements.txt /tmp/requirements.txt
RUN pip install -r /tmp/requirements.txt && rm /tmp/requirements.txt

RUN pip install --no-cache-dir python_lib/controller/python/chip*.whl
6 changes: 4 additions & 2 deletions scripts/build_python.sh
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,8 @@ Input Options:
<path> represents where the virtual environment is to be created.
-c, --clean_virtual_env <yes|no> When installing a virtual environment, create/clean it first.
Defaults to yes.
--include_pytest_deps <yes|no> Install requirements.python_tests.txt.
--include_pytest_deps <yes|no> Install requirements.txt for running scripts/tests and
src/python_testing scripts.
Defaults to yes.
--extra_packages PACKAGES Install extra Python packages from PyPI
--include_yamltests Whether to install the matter_yamltests wheel.
Expand Down Expand Up @@ -214,7 +215,8 @@ if [ -n "$install_virtual_env" ]; then

if [ "$install_pytest_requirements" = "yes" ]; then
echo_blue "Installing python test dependencies ..."
"$ENVIRONMENT_ROOT"/bin/pip install -r "$CHIP_ROOT/scripts/setup/requirements.python_tests.txt"
"$ENVIRONMENT_ROOT"/bin/pip install -r "$CHIP_ROOT/scripts/tests/requirements.txt"
"$ENVIRONMENT_ROOT"/bin/pip install -r "$CHIP_ROOT/src/python_testing/requirements.txt"
fi

echo ""
Expand Down
4 changes: 4 additions & 0 deletions scripts/py_matter_idl/setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,10 @@ description = Parse matter idl files
[options]
packages = find:
zip_safe = False
install_requires=
lark
jinja2
stringcase

[options.package_data]
matter_idl =
Expand Down
4 changes: 3 additions & 1 deletion scripts/py_matter_yamltests/setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -20,5 +20,7 @@ description = Parse matter yaml tests files

[options]
packages = find:

zip_safe = False
install_requires=
websockets
lark
16 changes: 0 additions & 16 deletions scripts/setup/requirements.python_tests.txt

This file was deleted.

5 changes: 5 additions & 0 deletions scripts/tests/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Python requirements for scripts in this location
click
colorama
diskcache
websockets
3 changes: 3 additions & 0 deletions src/python_testing/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
mobly
pyasn1
pyasn1_modules

0 comments on commit 0c90669

Please sign in to comment.