Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable Thermostat Clusters attribute tests from Darwin #11446

Merged

Conversation

chrisbeach59
Copy link
Contributor

Problem

Thermostat Cluster Server implements constraints per the spec.
The automated test generation for the cluster generates write values that are outside constraints
blocks #11058

Change overview

What's in this PR
Added excluding of Thermostat Cluster Attribute tests to Darwin zapt template file

Testing

Validated that after changes were made and regen:
All ZAP generated files generated successfully
ChipClusterTests.m did not contain any Thermostat Cluster tests

@boring-cyborg boring-cyborg bot added the darwin label Nov 4, 2021
@chrisbeach59 chrisbeach59 changed the title Disable Thermostat Clusters from Darwin Disable Thermostat Clusters attribute tests from Darwin Nov 4, 2021
@andy31415
Copy link
Contributor

@chrisbeach59 could we instead modify the tests to write values that are within spec range?

@chrisbeach59
Copy link
Contributor Author

@andy31415 We will need to generate manual test cases. They can not be auto generated. I have a series of tests that were used to validate the pre-attribute-changed-callback. Different tests need to be run based on the FeatureMap bits. The test results. Right now the scripting engine does not support reading an attribute and using it's value as a condition for other tests. Also the featuremap remains broken for little endian cpu's. I have some #defines in the thermostat-server.cpp code to work around this. (There is a PR in the ZAP project to address this that is still pending) I am hesitant to add the manual test cases to CI until this is addressed for fear of breaking CI, again.

@andy31415
Copy link
Contributor

@vivien-apple FIY.

From my side I am approving based on comments from Chris

@github-actions
Copy link

github-actions bot commented Nov 4, 2021

PR #11446: Size comparison from 29747a2 to 40fbf02

Full report (30 builds for efr32, esp32, k32w, mbed, nrfconnect, p6, qpg, telink)
platform target config section 29747a2 40fbf02 change % change
efr32 lighting-app BRD4161A (read only) 744860 744860 0 0.0
(read/write) 115696 115696 0 0.0
.bss 113924 113924 0 0.0
.data 1772 1772 0 0.0
.text 744852 744852 0 0.0
BRD4161A+rpc (read only) 732392 732392 0 0.0
(read/write) 132316 132316 0 0.0
.bss 130428 130428 0 0.0
.data 1888 1888 0 0.0
.text 732384 732384 0 0.0
lock-app BRD4161A (read only) 724128 724128 0 0.0
(read/write) 113516 113516 0 0.0
.bss 111780 111780 0 0.0
.data 1732 1732 0 0.0
.text 724120 724120 0 0.0
window-app BRD4161A (read only) 725024 725024 0 0.0
(read/write) 113836 113836 0 0.0
.bss 112100 112100 0 0.0
.data 1736 1736 0 0.0
.text 725016 725016 0 0.0
esp32 all-clusters-app c3devkit (read only) 880814 880814 0 0.0
(read/write) 1305592 1305592 0 0.0
.dram0.bss 57528 57528 0 0.0
.dram0.data 16464 16464 0 0.0
.flash.rodata 198360 198360 0 0.0
.flash.text 880814 880814 0 0.0
.iram0.text 57554 57554 0 0.0
m5stack (read only) 912095 912095 0 0.0
(read/write) 425380 425380 0 0.0
.dram0.bss 60032 60032 0 0.0
.dram0.data 32100 32100 0 0.0
.flash.rodata 207084 207084 0 0.0
.flash.text 912095 912095 0 0.0
.iram0.text 125115 125115 0 0.0
k32w lighting-app k32w061+se05x+release (read/write) 699240 699240 0 0.0
.bss 77784 77784 0 0.0
.data 1904 1904 0 0.0
.text 613752 613752 0 0.0
lock-app k32w061+debug (read/write) 592044 592044 0 0.0
.bss 68316 68316 0 0.0
.data 1872 1872 0 0.0
.text 516056 516056 0 0.0
shell k32w061+debug (read/write) 657536 657536 0 0.0
.bss 78952 78952 0 0.0
.data 1840 1840 0 0.0
.text 570944 570944 0 0.0
mbed all-clusters-app CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 2292664 2292664 0 0.0
.bss 178876 178876 0 0.0
.data 5216 5216 0 0.0
.heap 852352 852352 0 0.0
.text 1255264 1255264 0 0.0
lighting-app CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 2272992 2272992 0 0.0
.bss 171948 171948 0 0.0
.data 5568 5568 0 0.0
.heap 858928 858928 0 0.0
.text 1235592 1235592 0 0.0
lock-app CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 2250712 2250712 0 0.0
.bss 170844 170844 0 0.0
.data 5560 5560 0 0.0
.heap 860040 860040 0 0.0
.text 1213312 1213312 0 0.0
pigweed-app CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 1139744 1139744 0 0.0
.bss 11752 11752 0 0.0
.data 4368 4368 0 0.0
.heap 1020328 1020328 0 0.0
.text 103128 103128 0 0.0
shell CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 2048400 2048400 0 0.0
.bss 156224 156224 0 0.0
.data 4960 4960 0 0.0
.heap 875264 875264 0 0.0
.text 1011000 1011000 0 0.0
nrfconnect lighting-app nrf52840dk_nrf52840 (read/write) 861303 861303 0 0.0
bss 111248 111248 0 0.0
rodata 96808 96808 0 0.0
text 577624 577624 0 0.0
nrf52840dk_nrf52840+rpc (read/write) 823651 823651 0 0.0
bss 107600 107600 0 0.0
rodata 87988 87988 0 0.0
text 551772 551772 0 0.0
nrf5340dk_nrf5340_cpuapp (read/write) 786326 786326 0 0.0
bss 112624 112624 0 0.0
rodata 92064 92064 0 0.0
text 507080 507080 0 0.0
lock-app nrf52840dk_nrf52840 (read/write) 838011 838011 0 0.0
bss 110284 110284 0 0.0
rodata 93180 93180 0 0.0
text 559100 559100 0 0.0
nrf5340dk_nrf5340_cpuapp (read/write) 763306 763306 0 0.0
bss 111692 111692 0 0.0
rodata 88484 88484 0 0.0
text 488648 488648 0 0.0
pigweed-app nrf52840dk_nrf52840 (read/write) 497323 497323 0 0.0
bss 51824 51824 0 0.0
rodata 45776 45776 0 0.0
text 339436 339436 0 0.0
pump-app nrf52840dk_nrf52840 (read/write) 844119 844119 0 0.0
bss 110420 110420 0 0.0
rodata 94888 94888 0 0.0
text 563280 563280 0 0.0
pump-controller-app nrf52840dk_nrf52840 (read/write) 837863 837863 0 0.0
bss 110320 110320 0 0.0
rodata 93176 93176 0 0.0
text 558836 558836 0 0.0
shell nrf52840dk_nrf52840 (read/write) 775755 775755 0 0.0
bss 109072 109072 0 0.0
rodata 72432 72432 0 0.0
text 519664 519664 0 0.0
nrf5340dk_nrf5340_cpuapp (read/write) 690794 690794 0 0.0
bss 110052 110052 0 0.0
rodata 67076 67076 0 0.0
text 440256 440256 0 0.0
p6 all-clusters-app default (read/write) 2299176 2299176 0 0.0
.bss 106824 106824 0 0.0
.data 2520 2520 0 0.0
.heap 924000 924000 0 0.0
.text 1257440 1257440 0 0.0
lock-app default (read/write) 2212896 2212896 0 0.0
.bss 95648 95648 0 0.0
.data 2392 2392 0 0.0
.heap 935304 935304 0 0.0
.text 1171160 1171160 0 0.0
qpg lighting-app qpg6100+debug (read only) 490628 490628 0 0.0
(read/write) 114140 114140 0 0.0
.bss 50944 50944 0 0.0
.data 1004 1004 0 0.0
.text 485308 485308 0 0.0
lock-app qpg6100+debug (read only) 466952 466952 0 0.0
(read/write) 114144 114144 0 0.0
.bss 49888 49888 0 0.0
.data 960 960 0 0.0
.text 461632 461632 0 0.0
persistent-storage-app qpg6100+debug (read only) 153212 153212 0 0.0
(read/write) 114140 114140 0 0.0
.bss 18912 18912 0 0.0
.data 356 356 0 0.0
.text 147892 147892 0 0.0
telink lighting-app tlsr9518adk80d (read/write) 662982 662982 0 0.0
bss 69064 69064 0 0.0
noinit 33216 33216 0 0.0
text 458134 458134 0 0.0

@woody-apple
Copy link
Contributor

/rebase

When the pre-attribute-change-callback is added to the chip-all-clusters-app it is validating against spec constraints.  The auto generated tests violate these constraints and cause CI to fail.
@bzbarsky-apple bzbarsky-apple merged commit 0aa012a into project-chip:master Nov 5, 2021
@chrisbeach59 chrisbeach59 deleted the disable_thermostat_auto branch November 5, 2021 19:41
woody-apple pushed a commit that referenced this pull request Nov 9, 2021
* Remove Thermostat Cluster Tests from Darwin

When the pre-attribute-change-callback is added to the chip-all-clusters-app it is validating against spec constraints.  The auto generated tests violate these constraints and cause CI to fail.

* regen ChipClustersTests.m
PSONALl pushed a commit to PSONALl/connectedhomeip that referenced this pull request Dec 3, 2021
…#11446)

* Remove Thermostat Cluster Tests from Darwin

When the pre-attribute-change-callback is added to the chip-all-clusters-app it is validating against spec constraints.  The auto generated tests violate these constraints and cause CI to fail.

* regen ChipClustersTests.m
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants