Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove zap from dockerfile and update version update scripts #24732

Merged
merged 5 commits into from
Jan 31, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 15 additions & 6 deletions docs/code_generation.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,8 @@ The selection of enabled clusters and files is done using
[ZAP](https://github.com/project-chip/zap). You can download a recent release of
zap from its [releases page](https://github.com/project-chip/zap/releases). It
is recommended to download a release that is in sync with the currently in use
version by the SDK (see `integrations/docker/images/chip-build/Dockerfile` and
check the `ZAP_VERSION` setting).
version by the SDK (see `scripts/zap.json` and
`scripts/tools/zap/zap_execution.py` for the minimum supported version).

Beyond basic zap file selection, there are also `.json` zap settings that define
additional cluster info: source XML files, sdk-access methods and data types.
Expand All @@ -36,9 +36,17 @@ There are only two such files currently in use:

### Installing zap and environment variables

Matter scripts may need to invoke `zap-cli` (for code generation) or `zap` (to
start the UI tool). For this, scrips need to know where to find the commands. In
the following order, the scripts process these environment variables:
ZAP is generally installed as a third-party tool via CIPD during the build
environment bootstrap (see `scripts/zap.json`), which makes `zap-cli` available
in `$PATH` when running in a build environment.

**NOTE**: zap packages are currently NOT available for `arm64` (like when
compiling on Raspberry PI.). In these cases one should check out zap from source
and set `$ZAP_DEVELOPMENT_PATH` as described below.

When matter scripts need to invoke `zap-cli` (for code generation) or `zap` (to
start the UI tool), they make use of the following environment variables to
figure out where the zap tool is located (in order of precedence):

- if `$ZAP_DEVELOPMENT_PATH` is set, code assumes you are running zap from
source. Use this if you develop zap. Zap has to be bootstrapped (generally
Expand All @@ -48,7 +56,8 @@ the following order, the scripts process these environment variables:
- if `$ZAP_INSTALL_PATH` is set, code assumes that `zap` or `zap-cli` is
available in the given path. This is generally an unpacked release.

- otherwise, scripts will assume `zap`/`zap-cli` is in `$PATH`
- otherwise, scripts will assume `zap`/`zap-cli` is in `$PATH` (this is the
case when running in a bootstrapped environment)

### Using a UI to edit `.zap` files

Expand Down
18 changes: 0 additions & 18 deletions integrations/docker/images/chip-build/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -176,24 +176,6 @@ RUN set -x \
&& rm -rf glib \
&& : # last line

# Install a known ZAP release
# Only keep the cli version, since `zap` is 143MB and not usable (UI)
#
# NOTE: zap version occurs in several files (CI and docker)
# for a uniform update, a separate update script exists:
#
# scripts/tools/zap/version_update.py
ENV ZAP_VERSION=v2023.01.19-nightly
RUN set -x \
&& mkdir -p /opt/zap-${ZAP_VERSION} \
&& cd /opt/zap-${ZAP_VERSION} \
&& wget https://github.com/project-chip/zap/releases/download/${ZAP_VERSION}/zap-linux.zip \
&& unzip zap-linux.zip \
&& rm zap-linux.zip \
&& rm zap \
&& ln -s /opt/zap-${ZAP_VERSION}/zap-cli /usr/bin/ \
&& : # last line

# NodeJS: install a newer version than what apt-get would read
# This installs the latest LTS version of nodejs
#
Expand Down
2 changes: 1 addition & 1 deletion integrations/docker/images/chip-build/version
Original file line number Diff line number Diff line change
@@ -1 +1 @@
0.6.36 Version bump reason: K32W0-SDK 2.6.9 update
0.6.37 Version bump reason: removed ZAP from docker as CIPD now has it.
28 changes: 2 additions & 26 deletions scripts/tools/zap/version_update.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,38 +53,17 @@
'scripts/zap.json',
]

# Note that chip-cert-bits is assumed USAGE on purpose (it compiles code)
#
# The chip-build image change will affect all other images as they extend
# chip-build
DOCKER_FILES_DEPENDING_ON_ZAP_VERSION = [
'integrations/docker/images/chip-build/Dockerfile',
]


class UpdateChoice(Flag):
# Usage updates the CI, chip-cert and execution logic. Generally everything
# that would make use of the updated zap version
USAGE = auto()

# Docker updates just the chip-build (and as a side-effect underlying)
# image(s). This is a pre-requisite to be able to start using the new
# version.
DOCKER = auto()


__UPDATE_CHOICES__ = {
'docker': UpdateChoice.DOCKER,
'usage': UpdateChoice.USAGE,
'all': UpdateChoice.DOCKER | UpdateChoice.USAGE,
}

# NOTE: you likely need to also update
# integrations/docker/images/chip-build/version
#
# in PRs that update chip-build Dockerfiles. This update is not automated in
# this script.

# Apart from the above files which contain an exact ZAP version, the zap
# execution script contains the mimimal zap execution version, which generally
# we also enforce to be the current version.
Expand All @@ -106,9 +85,9 @@ class UpdateChoice(Flag):
help='Determines the verbosity of script output.')
@click.option(
'--update',
default='docker',
default='usage',
type=click.Choice(__UPDATE_CHOICES__.keys(), case_sensitive=False),
help='What to update: docker, usage, all. Default is "docker".')
help='What to update: usage (only choice currently).')
@click.option(
'--new-version',
default=None,
Expand Down Expand Up @@ -136,9 +115,6 @@ def version_update(log_level, update, new_version):
if UpdateChoice.USAGE in update:
files_to_update += USAGE_FILES_DEPENDING_ON_ZAP_VERSION

if UpdateChoice.DOCKER in update:
files_to_update += DOCKER_FILES_DEPENDING_ON_ZAP_VERSION

for name in files_to_update:
with open(os.path.join(CHIP_ROOT_DIR, name), 'rt') as f:
file_data = f.read()
Expand Down