Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update_cluster_revisions.py: Add possibility to bump cluster revisions to the latest in spec #36456

Open
wants to merge 14 commits into
base: master
Choose a base branch
from

Conversation

soares-sergio
Copy link
Contributor

@soares-sergio soares-sergio commented Nov 11, 2024

This PR adds a new feature to the update_cluster_revisions.py tool:

  • Bump the cluster revisions to the latest according to the spec if the --new-revision argument is omitted.

If --new-revision is provided, the tool uses it and behaves as previously implemented.
I've also added documentation in README.md on how to use the tool and expected output examples.

Copy link

semanticdiff-com bot commented Nov 11, 2024

Review changes with  SemanticDiff

Changed Files
File Status
  scripts/tools/zap/update_cluster_revisions.py  9% smaller
  scripts/tools/zap/README.md Unsupported file format

Copy link

PR #36456: Size comparison from c3b35eb to 53d4c18

Full report (3 builds for cc32xx, stm32)
platform target config section c3b35eb 53d4c18 change % change
cc32xx air-purifier CC3235SF_LAUNCHXL FLASH 625902 625902 0 0.0
RAM 205780 205780 0 0.0
lock CC3235SF_LAUNCHXL FLASH 667214 667214 0 0.0
RAM 205932 205932 0 0.0
stm32 light STM32WB5MM-DK FLASH 482920 482920 0 0.0
RAM 144844 144844 0 0.0

Copy link

github-actions bot commented Nov 11, 2024

PR #36456: Size comparison from c3b35eb to ab056da

Full report (68 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
platform target config section c3b35eb ab056da change % change
bl602 lighting-app bl602+mfd+littlefs+rpc FLASH 1349404 1349402 -2 -0.0
RAM 104080 104080 0 0.0
bl702 lighting-app bl702+eth FLASH 649330 649330 0 0.0
RAM 25321 25321 0 0.0
bl702+wifi FLASH 826910 826910 0 0.0
RAM 14053 14053 0 0.0
bl706+mfd+rpc+littlefs FLASH 1055108 1055106 -2 -0.0
RAM 23893 23893 0 0.0
bl702l lighting-app bl702l+mfd+littlefs FLASH 976474 976472 -2 -0.0
RAM 16556 16556 0 0.0
cc13x4_26x4 lighting-app LP_EM_CC1354P10_6 FLASH 837828 837828 0 0.0
RAM 123628 123628 0 0.0
lock-ftd LP_EM_CC1354P10_6 FLASH 823736 823736 0 0.0
RAM 125516 125516 0 0.0
pump-app LP_EM_CC1354P10_6 FLASH 770468 770468 0 0.0
RAM 113984 113984 0 0.0
pump-controller-app LP_EM_CC1354P10_6 FLASH 754680 754680 0 0.0
RAM 114192 114192 0 0.0
cc32xx air-purifier CC3235SF_LAUNCHXL FLASH 625902 625902 0 0.0
RAM 205780 205780 0 0.0
lock CC3235SF_LAUNCHXL FLASH 667214 667214 0 0.0
RAM 205932 205932 0 0.0
cyw30739 light CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 679473 679473 0 0.0
RAM 78684 78684 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 699317 699317 0 0.0
RAM 81324 81324 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 699317 699317 0 0.0
RAM 81324 81324 0 0.0
CYW930739M2EVB-02 unknown 2040 2040 0 0.0
FLASH 656253 656253 0 0.0
RAM 73752 73752 0 0.0
light-switch CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 616321 616321 0 0.0
RAM 71644 71644 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 635957 635957 0 0.0
RAM 74188 74188 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 635957 635957 0 0.0
RAM 74188 74188 0 0.0
lock CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 635777 635777 0 0.0
RAM 74692 74692 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 655485 655485 0 0.0
RAM 77236 77236 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 655485 655485 0 0.0
RAM 77236 77236 0 0.0
thermostat CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 611173 611173 0 0.0
RAM 68772 68772 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 631025 631025 0 0.0
RAM 71412 71412 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 631025 631025 0 0.0
RAM 71412 71412 0 0.0
efr32 lock-app BRD4187C FLASH 927564 927564 0 0.0
RAM 160160 160160 0 0.0
BRD4338a FLASH 743680 743680 0 0.0
RAM 233288 233288 0 0.0
window-app BRD4187C FLASH 1018784 1018776 -8 -0.0
RAM 128260 128260 0 0.0
esp32 all-clusters-app c3devkit DRAM 95328 95328 0 0.0
FLASH 1540416 1540416 0 0.0
IRAM 82542 82542 0 0.0
m5stack DRAM 116256 116256 0 0.0
FLASH 1549266 1549266 0 0.0
IRAM 117039 117039 0 0.0
linux air-purifier-app debug unknown 4712 4712 0 0.0
FLASH 2698929 2698929 0 0.0
RAM 129648 129648 0 0.0
all-clusters-app debug unknown 5552 5552 0 0.0
FLASH 5992626 5992626 0 0.0
RAM 523968 523968 0 0.0
all-clusters-minimal-app debug unknown 5448 5448 0 0.0
FLASH 5328234 5328234 0 0.0
RAM 242480 242480 0 0.0
bridge-app debug unknown 5432 5432 0 0.0
FLASH 4671004 4671004 0 0.0
RAM 218416 218416 0 0.0
chip-tool debug unknown 5984 5984 0 0.0
FLASH 12872224 12872224 0 0.0
RAM 584466 584466 0 0.0
chip-tool-ipv6only arm64 unknown 21400 21400 0 0.0
FLASH 11002736 11002736 0 0.0
RAM 635368 635368 0 0.0
fabric-admin debug unknown 5808 5808 0 0.0
FLASH 11277331 11277331 0 0.0
RAM 584818 584818 0 0.0
fabric-bridge-app debug unknown 4688 4688 0 0.0
FLASH 4495274 4495274 0 0.0
RAM 205392 205392 0 0.0
fabric-sync debug unknown 4896 4896 0 0.0
FLASH 5309749 5309749 0 0.0
RAM 466872 466872 0 0.0
lighting-app debug+rpc+ui unknown 6096 6096 0 0.0
FLASH 5606929 5606929 0 0.0
RAM 228552 228552 0 0.0
lock-app debug unknown 5368 5368 0 0.0
FLASH 4720478 4720478 0 0.0
RAM 204552 204552 0 0.0
ota-provider-app debug unknown 4744 4744 0 0.0
FLASH 4345910 4345910 0 0.0
RAM 198272 198272 0 0.0
ota-requestor-app debug unknown 4680 4680 0 0.0
FLASH 4484312 4484312 0 0.0
RAM 202840 202840 0 0.0
shell debug unknown 4240 4240 0 0.0
FLASH 3011325 3011325 0 0.0
RAM 160368 160368 0 0.0
thermostat-no-ble arm64 unknown 9440 9440 0 0.0
FLASH 4090000 4090000 0 0.0
RAM 242936 242936 0 0.0
tv-app debug unknown 5664 5664 0 0.0
FLASH 5943701 5943701 0 0.0
RAM 596336 596336 0 0.0
tv-casting-app debug unknown 5232 5232 0 0.0
FLASH 11079421 11079421 0 0.0
RAM 694536 694536 0 0.0
nrfconnect all-clusters-app nrf52840dk_nrf52840 FLASH 915308 915308 0 0.0
RAM 143281 143281 0 0.0
nrf7002dk_nrf5340_cpuapp FLASH 887568 887568 0 0.0
RAM 141468 141468 0 0.0
all-clusters-minimal-app nrf52840dk_nrf52840 FLASH 847564 847564 0 0.0
RAM 142189 142189 0 0.0
nxp contact k32w0+release FLASH 584096 584096 0 0.0
RAM 71044 71044 0 0.0
mcxw71+release FLASH 598640 598640 0 0.0
RAM 63144 63144 0 0.0
light k32w0+release FLASH 611156 611156 0 0.0
RAM 70436 70436 0 0.0
k32w1+release FLASH 684952 684952 0 0.0
RAM 48776 48776 0 0.0
lock mcxw71+release FLASH 748488 748488 0 0.0
RAM 67300 67300 0 0.0
psoc6 all-clusters cy8ckit_062s2_43012 FLASH 1642948 1642948 0 0.0
RAM 212056 212056 0 0.0
all-clusters-minimal cy8ckit_062s2_43012 FLASH 1550164 1550164 0 0.0
RAM 208856 208856 0 0.0
light cy8ckit_062s2_43012 FLASH 1466588 1466588 0 0.0
RAM 200840 200840 0 0.0
lock cy8ckit_062s2_43012 FLASH 1464764 1464764 0 0.0
RAM 225200 225200 0 0.0
qpg lighting-app qpg6105+debug FLASH 662120 662120 0 0.0
RAM 105380 105380 0 0.0
lock-app qpg6105+debug FLASH 620220 620220 0 0.0
RAM 99832 99832 0 0.0
stm32 light STM32WB5MM-DK FLASH 482920 482920 0 0.0
RAM 144844 144844 0 0.0
telink bridge-app tlsr9258a FLASH 684642 684642 0 0.0
RAM 91528 91528 0 0.0
contact-sensor-app tlsr9528a_retention FLASH 622654 622654 0 0.0
RAM 50464 50464 0 0.0
light-switch-app-ota-compress-lzma-shell-factory-data tlsr9528a FLASH 710580 710580 0 0.0
RAM 73804 73804 0 0.0
lighting-app-ota-factory-data tlsr9118bdk40d FLASH 628948 628948 0 0.0
RAM 145384 145384 0 0.0
lighting-app-ota-rpc-factory-data-4mb tlsr9518adk80d FLASH 814836 814836 0 0.0
RAM 100004 100004 0 0.0
tizen all-clusters-app arm unknown 4948 4948 0 0.0
FLASH 1722468 1722468 0 0.0
RAM 90656 90656 0 0.0
chip-tool-ubsan arm unknown 10832 10832 0 0.0
FLASH 18006294 18006294 0 0.0
RAM 7857036 7857036 0 0.0

@soares-sergio soares-sergio changed the title zap_cluster_rev: Add tool to bump cluster revisions on Zap files update_cluster_revisions.py: Add possibility to bump cluster revisions to the latest in spec Nov 25, 2024
Copy link
Contributor

@bzbarsky-apple bzbarsky-apple left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

scripts/tools/zap/update_cluster_revisions.py Outdated Show resolved Hide resolved
@@ -126,6 +170,41 @@ def updateOne(item):
subprocess.check_call(['./scripts/tools/zap/convert.py', target])


def updateOneToLatest(item):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please follow pep8 satule (e.g. update_one_to_latest)

json.dump(data, file)

# Now run convert.py on the file to have ZAP reformat it however it likes.
subprocess.check_call(['./scripts/tools/zap/convert.py', target])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please include python3 in the command

@@ -51,6 +70,34 @@ def getTargets(cluster_id: int):
return targets


def get_outdated_clusters(data: object, xml_clusters: dict, args) -> list[ClusterInfo]:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The contents of args could have been state of a VersionUpdateProcessor class.

Overall this script relies too much on passing data around that could be state of a class

def get_outdated_clusters(data: object, xml_clusters: dict, args) -> list[ClusterInfo]:
result = []
for endpoint in data.get("endpointTypes", []):
endpoint_id = endpoint.get("id") - 1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is NOT the endpoint_id. Endpoint IDs are separate from the id field. This may work oddly on ZAP files that have non-contiguous endpoint IDs.

except (KeyError, ValueError):
continue
# Filter in outdated clusters only
if (cluster_revision == spec_revision):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

avoid C-style () around whole condition

if (cluster_revision == spec_revision):
break
# If old_revision is present, filter in matching only
if (args.old_revision is not None and cluster_revision != args.old_revision):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Parenthesize sub-expressions:

Suggested change
if (args.old_revision is not None and cluster_revision != args.old_revision):
if (args.old_revision is not None) and (cluster_revision != args.old_revision):

cluster_code: int
cluster_spec_revision: int
cluster_name: str
json_attribute: object
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit, for hinting isn't this a dict[str,object]?

@@ -51,6 +70,34 @@ def getTargets(cluster_id: int):
return targets


def get_outdated_clusters(data: object, xml_clusters: dict, args) -> list[ClusterInfo]:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit, there is a mix of functions in this PR mixed with with snake_case and lowerCamelCase. Overall I think autopep8 does say we should to functions are snake_case. My suggestions is keep it the same for the file and if we want to migrate the entire file we do it all at once in the file as a follow up refactor

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or follow Tennessee's recommendation of starting the switch to pep8 and doing that for all your newly introduced functions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants