Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add checking for correctness of error codes to YAML tests. #12306

Merged

Conversation

bzbarsky-apple
Copy link
Contributor

For now, uses errorWrongValue in tests where the wrong error code is
returned by the SDK.

Problem

YAML tests don't check whether the error status is the one the spec requires, just that it's some error status.

Change overview

Add checking for the correct status value.

Testing

Tests pass, after annotating a few places where the SDK is buggy with errorWrongValue.

For now, uses errorWrongValue in tests where the wrong error code is
returned by the SDK.
@github-actions
Copy link

github-actions bot commented Nov 27, 2021

PR #12306: Size comparison from 8a2fd0d to a7f459a

Increases (9 builds for efr32, esp32, linux, p6)
platform target config section 8a2fd0d a7f459a change % change
efr32 lighting-app BRD4161A (read only) 762072 762088 16 0.0
.text 762064 762080 16 0.0
esp32 all-clusters-app c3devkit (read only) 836422 836430 8 0.0
.flash.text 836422 836430 8 0.0
m5stack (read only) 907835 907867 32 0.0
.flash.text 902451 902483 32 0.0
linux all-clusters-app debug (read only) 1774185 1774201 16 0.0
.text 1496946 1496962 16 0.0
chip-tool debug (read only) 6085653 6090789 5136 0.1
.rodata 291816 291976 160 0.1
.text 5404709 5409685 4976 0.1
lighting-app debug+rpc (read only) 1628169 1628185 16 0.0
.text 1358658 1358674 16 0.0
ota-requestor-app debug (read only) 1405977 1405993 16 0.0
.text 1176530 1176546 16 0.0
p6 all-clusters-app default (read/write) 2311176 2311192 16 0.0
.text 1269440 1269456 16 0.0
lock-app default (read/write) 2222992 2223008 16 0.0
.text 1181256 1181272 16 0.0
Full report (38 builds for efr32, esp32, k32w, linux, mbed, nrfconnect, p6, qpg, telink)
platform target config section 8a2fd0d a7f459a change % change
efr32 lighting-app BRD4161A (read only) 762072 762088 16 0.0
(read/write) 119836 119836 0 0.0
.bss 118012 118012 0 0.0
.data 1820 1820 0 0.0
.text 762064 762080 16 0.0
BRD4161A+rpc (read only) 790520 790520 0 0.0
(read/write) 138132 138132 0 0.0
.bss 136212 136212 0 0.0
.data 1920 1920 0 0.0
.text 790512 790512 0 0.0
lock-app BRD4161A (read only) 736032 736032 0 0.0
(read/write) 117540 117540 0 0.0
.bss 115764 115764 0 0.0
.data 1776 1776 0 0.0
.text 736024 736024 0 0.0
window-app BRD4161A (read only) 739096 739096 0 0.0
(read/write) 117972 117972 0 0.0
.bss 116188 116188 0 0.0
.data 1784 1784 0 0.0
.text 739088 739088 0 0.0
esp32 all-clusters-app c3devkit (read only) 836422 836430 8 0.0
(read/write) 1224474 1224474 0 0.0
.dram0.bss 59144 59144 0 0.0
.dram0.data 14100 14100 0 0.0
.flash.rodata 165968 165968 0 0.0
.flash.text 836422 836430 8 0.0
.iram0.text 61394 61394 0 0.0
m5stack (read only) 907835 907867 32 0.0
(read/write) 423692 423692 0 0.0
.dram0.bss 64536 64536 0 0.0
.dram0.data 34072 34072 0 0.0
.flash.rodata 193804 193804 0 0.0
.flash.text 902451 902483 32 0.0
.iram0.text 122943 122943 0 0.0
k32w lighting-app k32w061+se05x+release (read/write) 723208 723208 0 0.0
.bss 78292 78292 0 0.0
.data 1956 1956 0 0.0
.text 637160 637160 0 0.0
lock-app k32w061+debug (read/write) 612280 612280 0 0.0
.bss 68740 68740 0 0.0
.data 1920 1920 0 0.0
.text 535820 535820 0 0.0
shell k32w061+debug (read/write) 677600 677600 0 0.0
.bss 79892 79892 0 0.0
.data 1892 1892 0 0.0
.text 590016 590016 0 0.0
linux all-clusters-app debug (read only) 1774185 1774201 16 0.0
(read/write) 131544 131544 0 0.0
.bss 60144 60144 0 0.0
.data 1170 1170 0 0.0
.data.rel.ro 64928 64928 0 0.0
.dynamic 592 592 0 0.0
.got 4112 4112 0 0.0
.init 27 27 0 0.0
.init_array 576 576 0 0.0
.rodata 138869 138869 0 0.0
.text 1496946 1496962 16 0.0
bridge-app debug+rpc (read only) 1348509 1348509 0 0.0
(read/write) 77856 77856 0 0.0
.bss 41744 41744 0 0.0
.data 1680 1680 0 0.0
.data.rel.ro 29384 29384 0 0.0
.dynamic 592 592 0 0.0
.got 3984 3984 0 0.0
.init 27 27 0 0.0
.init_array 424 424 0 0.0
.rodata 113500 113500 0 0.0
.text 1134917 1134917 0 0.0
chip-tool debug (read only) 6085653 6090789 5136 0.1
(read/write) 199920 199920 0 0.0
.bss 40064 40064 0 0.0
.data 2384 2384 0 0.0
.data.rel.ro 151888 151888 0 0.0
.dynamic 592 592 0 0.0
.got 4456 4456 0 0.0
.init 27 27 0 0.0
.init_array 488 488 0 0.0
.rodata 291816 291976 160 0.1
.text 5404709 5409685 4976 0.1
lighting-app debug+rpc (read only) 1628169 1628185 16 0.0
(read/write) 111104 111104 0 0.0
.bss 47440 47440 0 0.0
.data 1362 1362 0 0.0
.data.rel.ro 56976 56976 0 0.0
.dynamic 608 608 0 0.0
.got 4136 4136 0 0.0
.init 27 27 0 0.0
.init_array 552 552 0 0.0
.rodata 131793 131793 0 0.0
.text 1358658 1358674 16 0.0
ota-provider-app debug (read only) 1309537 1309537 0 0.0
(read/write) 76312 76312 0 0.0
.bss 44320 44320 0 0.0
.data 912 912 0 0.0
.data.rel.ro 25944 25944 0 0.0
.dynamic 592 592 0 0.0
.got 4048 4048 0 0.0
.init 27 27 0 0.0
.init_array 464 464 0 0.0
.rodata 114832 114832 0 0.0
.text 1094354 1094354 0 0.0
ota-requestor-app debug (read only) 1405977 1405993 16 0.0
(read/write) 80144 80144 0 0.0
.bss 46752 46752 0 0.0
.data 976 976 0 0.0
.data.rel.ro 27272 27272 0 0.0
.dynamic 592 592 0 0.0
.got 4032 4032 0 0.0
.init 27 27 0 0.0
.init_array 488 488 0 0.0
.rodata 126304 126304 0 0.0
.text 1176530 1176546 16 0.0
shell debug (read only) 820321 820321 0 0.0
(read/write) 66936 66936 0 0.0
.bss 23496 23496 0 0.0
.data 338 338 0 0.0
.data.rel.ro 38560 38560 0 0.0
.dynamic 592 592 0 0.0
.got 3560 3560 0 0.0
.init 27 27 0 0.0
.init_array 360 360 0 0.0
.rodata 79023 79023 0 0.0
.text 634610 634610 0 0.0
tv-app debug (read only) 1918385 1918385 0 0.0
(read/write) 320152 320152 0 0.0
.bss 250232 250232 0 0.0
.data 2880 2880 0 0.0
.data.rel.ro 61368 61368 0 0.0
.dynamic 592 592 0 0.0
.got 4432 4432 0 0.0
.init 27 27 0 0.0
.init_array 632 632 0 0.0
.rodata 159848 159848 0 0.0
.text 1610338 1610338 0 0.0
mbed all-clusters-app CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 2294736 2294736 0 0.0
.bss 181884 181884 0 0.0
.data 5240 5240 0 0.0
.heap 849320 849320 0 0.0
.text 1257336 1257336 0 0.0
lighting-app CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 2279952 2279952 0 0.0
.bss 172492 172492 0 0.0
.data 5600 5600 0 0.0
.heap 858352 858352 0 0.0
.text 1242552 1242552 0 0.0
lock-app CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 2253096 2253096 0 0.0
.bss 171308 171308 0 0.0
.data 5584 5584 0 0.0
.heap 859552 859552 0 0.0
.text 1215696 1215696 0 0.0
pigweed-app CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 1139744 1139744 0 0.0
.bss 11752 11752 0 0.0
.data 4368 4368 0 0.0
.heap 1020328 1020328 0 0.0
.text 103128 103128 0 0.0
shell CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 2050936 2050936 0 0.0
.bss 156496 156496 0 0.0
.data 4984 4984 0 0.0
.heap 874968 874968 0 0.0
.text 1013536 1013536 0 0.0
nrfconnect lighting-app nrf52840dk_nrf52840 (read/write) 875499 875499 0 0.0
bss 112660 112660 0 0.0
rodata 97188 97188 0 0.0
text 590008 590008 0 0.0
nrf52840dk_nrf52840+rpc (read/write) 838075 838075 0 0.0
bss 109012 109012 0 0.0
rodata 88436 88436 0 0.0
text 564324 564324 0 0.0
nrf5340dk_nrf5340_cpuapp (read/write) 800538 800538 0 0.0
bss 114036 114036 0 0.0
rodata 92444 92444 0 0.0
text 519472 519472 0 0.0
lock-app nrf52840dk_nrf52840 (read/write) 846427 846427 0 0.0
bss 109700 109700 0 0.0
rodata 92972 92972 0 0.0
text 568280 568280 0 0.0
nrf5340dk_nrf5340_cpuapp (read/write) 771698 771698 0 0.0
bss 111108 111108 0 0.0
rodata 88260 88260 0 0.0
text 497832 497832 0 0.0
pigweed-app nrf52840dk_nrf52840 (read/write) 497327 497327 0 0.0
bss 51824 51824 0 0.0
rodata 45780 45780 0 0.0
text 339436 339436 0 0.0
pump-app nrf52840dk_nrf52840 (read/write) 852399 852399 0 0.0
bss 109836 109836 0 0.0
rodata 94680 94680 0 0.0
text 572324 572324 0 0.0
pump-controller-app nrf52840dk_nrf52840 (read/write) 845523 845523 0 0.0
bss 109712 109712 0 0.0
rodata 92844 92844 0 0.0
text 567408 567408 0 0.0
shell nrf52840dk_nrf52840 (read/write) 778447 778447 0 0.0
bss 109180 109180 0 0.0
rodata 73084 73084 0 0.0
text 521576 521576 0 0.0
nrf5340dk_nrf5340_cpuapp (read/write) 693482 693482 0 0.0
bss 110164 110164 0 0.0
rodata 67724 67724 0 0.0
text 442184 442184 0 0.0
p6 all-clusters-app default (read/write) 2311176 2311192 16 0.0
.bss 114688 114688 0 0.0
.data 2544 2544 0 0.0
.heap 916112 916112 0 0.0
.text 1269440 1269456 16 0.0
lock-app default (read/write) 2222992 2223008 16 0.0
.bss 100976 100976 0 0.0
.data 2416 2416 0 0.0
.heap 929952 929952 0 0.0
.text 1181256 1181272 16 0.0
qpg lighting-app qpg6100+debug (read only) 502992 502992 0 0.0
(read/write) 114144 114144 0 0.0
.bss 50400 50400 0 0.0
.data 1024 1024 0 0.0
.text 497672 497672 0 0.0
lock-app qpg6100+debug (read only) 475708 475708 0 0.0
(read/write) 114140 114140 0 0.0
.bss 49272 49272 0 0.0
.data 980 980 0 0.0
.text 470388 470388 0 0.0
persistent-storage-app qpg6100+debug (read only) 105424 105424 0 0.0
(read/write) 114138 114138 0 0.0
.bss 12002 12002 0 0.0
.data 276 276 0 0.0
.text 100104 100104 0 0.0
telink lighting-app tlsr9518adk80d (read/write) 777286 777286 0 0.0
bss 79236 79236 0 0.0
noinit 37160 37160 0 0.0
text 540616 540616 0 0.0

Copy link
Contributor

@vivien-apple vivien-apple left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing that.

src/app/tests/suites/TestCluster.yaml Show resolved Hide resolved
@andy31415 andy31415 merged commit 800bd23 into project-chip:master Nov 29, 2021
@bzbarsky-apple bzbarsky-apple deleted the check-status-codes-yaml branch November 29, 2021 20:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants