Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make Tests job fail quickly instead of timing out on discovery failure. #16439

Merged
merged 1 commit into from
Mar 18, 2022
Merged

Make Tests job fail quickly instead of timing out on discovery failure. #16439

merged 1 commit into from
Mar 18, 2022

Conversation

bzbarsky-apple
Copy link
Contributor

If waitForAnyAdvertisement failed, we would not actually clean things up and sit there until the job timed out. Two changes to fix that:

  1. Add the app to the app register before calling start() on it, so
    that the killAll() on the app register has a chance of killing the
    app.

  2. Set self.process immediately, instead of doing it after things that
    might fail (like waitForAnyAdvertisement), so killing the app actually
    works.

Problem

Failures during test startup end up waiting for the CI job to time out instead of fast-failing.

Change overview

See above.

Testing

Introduced a raise in waitForAnyAdvertisement and checked that without these changes run_test_suite.py never terminates and with the changes it does.

@bzbarsky-apple
Make Tests job fail quickly instead of timing out on discovery failure.
ee728c1 
If waitForAnyAdvertisement failed, we would not actually clean things up and sit there until the job timed out.  Two changes to fix that:

1) Add the app to the app register before calling start() on it, so
   that the killAll() on the app register has a chance of killing the
   app.

2) Set self.process immediately, instead of doing it after things that
   might fail (like waitForAnyAdvertisement), so killing the app actually
   works.
@github-actions
Copy link

github-actions bot commented Mar 18, 2022 •
edited
Loading

PR #16439: Size comparison from fea74e6 to ee728c1

Full report (27 builds for cyw30739, efr32, esp32, k32w, linux, mbed, nrfconnect, p6, telink)
platform target config section fea74e6 ee728c1 change % change
cyw30739 light cyw930739m2evb_01 (read/write) 603674 603674 0 0.0
.app_xip_area 510772 510772 0 0.0
.bss 75656 75656 0 0.0
.data 596 596 0 0.0
.rodata 0 0 0 0.0
.text 0 0 0 0.0
lock cyw930739m2evb_01 (read/write) 561462 561462 0 0.0
.app_xip_area 470088 470088 0 0.0
.bss 74160 74160 0 0.0
.data 560 560 0 0.0
.rodata 0 0 0 0.0
.text 0 0 0 0.0
ota-requestor-no-progress-logging cyw930739m2evb_01 (read/write) 571306 571306 0 0.0
.app_xip_area 470284 470284 0 0.0
.bss 83488 83488 0 0.0
.data 500 500 0 0.0
.rodata 0 0 0 0.0
.text 112 112 0 0.0
efr32 lighting-app BRD4161A (read only) 921752 921752 0 0.0
(read/write) 128760 128760 0 0.0
.bss 126768 126768 0 0.0
.data 1988 1988 0 0.0
.text 921744 921744 0 0.0
BRD4161A+rpc (read only) 950572 950572 0 0.0
(read/write) 144712 144712 0 0.0
.bss 142544 142544 0 0.0
.data 2168 2168 0 0.0
.text 950564 950564 0 0.0
window-app BRD4161A (read only) 852088 852088 0 0.0
(read/write) 126704 126704 0 0.0
.bss 124840 124840 0 0.0
.data 1864 1864 0 0.0
.text 852080 852080 0 0.0
esp32 all-clusters-app c3devkit (read only) 961620 961620 0 0.0
(read/write) 1395034 1395034 0 0.0
.dram0.bss 64048 64048 0 0.0
.dram0.data 14188 14188 0 0.0
.flash.rodata 197888 197888 0 0.0
.flash.text 961620 961620 0 0.0
.iram0.text 62016 62016 0 0.0
m5stack (read only) 1017775 1017775 0 0.0
(read/write) 462892 462892 0 0.0
.dram0.bss 69576 69576 0 0.0
.dram0.data 34016 34016 0 0.0
.flash.rodata 227464 227464 0 0.0
.flash.text 1012391 1012391 0 0.0
.iram0.text 123107 123107 0 0.0
k32w light k32w061+release (read/write) 700912 700912 0 0.0
.bss 77656 77656 0 0.0
.data 1868 1868 0 0.0
.text 615588 615588 0 0.0
lock k32w061+release (read/write) 700764 700764 0 0.0
.bss 77624 77624 0 0.0
.data 1908 1908 0 0.0
.text 615432 615432 0 0.0
linux all-clusters-app debug (read only) 2473969 2473969 0 0.0
(read/write) 146576 146576 0 0.0
.bss 60736 60736 0 0.0
.data 1264 1264 0 0.0
.data.rel.ro 78840 78840 0 0.0
.dynamic 592 592 0 0.0
.got 4168 4168 0 0.0
.init 27 27 0 0.0
.init_array 936 936 0 0.0
.rodata 211205 211205 0 0.0
.text 2099314 2099314 0 0.0
bridge-app debug+rpc (read only) 1749741 1749741 0 0.0
(read/write) 92752 92752 0 0.0
.bss 47784 47784 0 0.0
.data 1952 1952 0 0.0
.data.rel.ro 37936 37936 0 0.0
.dynamic 592 592 0 0.0
.got 3920 3920 0 0.0
.init 27 27 0 0.0
.init_array 544 544 0 0.0
.rodata 143756 143756 0 0.0
.text 1490773 1490773 0 0.0
chip-tool debug (read only) 10143941 10143941 0 0.0
(read/write) 358272 358272 0 0.0
.bss 25728 25728 0 0.0
.data 1072 1072 0 0.0
.data.rel.ro 325408 325408 0 0.0
.dynamic 608 608 0 0.0
.got 4784 4784 0 0.0
.init 27 27 0 0.0
.init_array 632 632 0 0.0
.rodata 517781 517781 0 0.0
.text 8843557 8843557 0 0.0
chip-tool-ipv6only arm64 (read only) 9768076 9768076 0 0.0
(read/write) 476113 476113 0 0.0
.bss 44033 44033 0 0.0
.data 1128 1128 0 0.0
.data.rel.ro 371824 371824 0 0.0
.dynamic 560 560 0 0.0
.got 55328 55328 0 0.0
.init 24 24 0 0.0
.init_array 192 192 0 0.0
.rodata 493620 493620 0 0.0
.text 8223092 8223092 0 0.0
door-lock-app debug (read only) 2001033 2001033 0 0.0
(read/write) 120224 120224 0 0.0
.bss 51008 51008 0 0.0
.data 992 992 0 0.0
.data.rel.ro 62840 62840 0 0.0
.dynamic 592 592 0 0.0
.got 4120 4120 0 0.0
.init 27 27 0 0.0
.init_array 664 664 0 0.0
.rodata 179964 179964 0 0.0
.text 1671410 1671410 0 0.0
lighting-app debug+rpc (read only) 2132761 2132761 0 0.0
(read/write) 125544 125544 0 0.0
.bss 51776 51776 0 0.0
.data 1472 1472 0 0.0
.data.rel.ro 66792 66792 0 0.0
.dynamic 608 608 0 0.0
.got 4152 4152 0 0.0
.init 27 27 0 0.0
.init_array 736 736 0 0.0
.rodata 172668 172668 0 0.0
.text 1803410 1803410 0 0.0
ota-provider-app debug (read only) 1936049 1936049 0 0.0
(read/write) 116048 116048 0 0.0
.bss 50912 50912 0 0.0
.data 1288 1288 0 0.0
.data.rel.ro 58200 58200 0 0.0
.dynamic 608 608 0 0.0
.got 4376 4376 0 0.0
.init 27 27 0 0.0
.init_array 616 616 0 0.0
.rodata 165963 165963 0 0.0
.text 1618322 1618322 0 0.0
ota-requestor-app debug (read only) 1966041 1966041 0 0.0
(read/write) 119016 119016 0 0.0
.bss 51904 51904 0 0.0
.data 1416 1416 0 0.0
.data.rel.ro 60264 60264 0 0.0
.dynamic 592 592 0 0.0
.got 4168 4168 0 0.0
.init 27 27 0 0.0
.init_array 640 640 0 0.0
.rodata 162212 162212 0 0.0
.text 1651842 1651842 0 0.0
shell debug (read only) 2424129 2424129 0 0.0
(read/write) 150632 150632 0 0.0
.bss 70664 70664 0 0.0
.data 784 784 0 0.0
.data.rel.ro 73472 73472 0 0.0
.dynamic 592 592 0 0.0
.got 4152 4152 0 0.0
.init 27 27 0 0.0
.init_array 920 920 0 0.0
.rodata 209138 209138 0 0.0
.text 2058290 2058290 0 0.0
thermostat-no-ble arm64 (read only) 2222004 2222004 0 0.0
(read/write) 149809 149809 0 0.0
.bss 65713 65713 0 0.0
.data 1024 1024 0 0.0
.data.rel.ro 75728 75728 0 0.0
.dynamic 560 560 0 0.0
.got 4352 4352 0 0.0
.init 24 24 0 0.0
.init_array 360 360 0 0.0
.rodata 137956 137956 0 0.0
.text 1863984 1863984 0 0.0
tv-app debug (read only) 2674345 2674345 0 0.0
(read/write) 251136 251136 0 0.0
.bss 167808 167808 0 0.0
.data 3104 3104 0 0.0
.data.rel.ro 74168 74168 0 0.0
.dynamic 592 592 0 0.0
.got 4552 4552 0 0.0
.init 27 27 0 0.0
.init_array 888 888 0 0.0
.rodata 206485 206485 0 0.0
.text 2293010 2293010 0 0.0
mbed lock-app CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 2352940 2352940 0 0.0
.bss 186644 186644 0 0.0
.data 5752 5752 0 0.0
.text 1315540 1315540 0 0.0
nrfconnect all-clusters-app nrf52840dk_nrf52840 (read/write) 1155567 1155567 0 0.0
bss 146724 146724 0 0.0
rodata 145856 145856 0 0.0
text 787836 787836 0 0.0
p6 all-clusters-app default (read/write) 2492264 2492264 0 0.0
.bss 120072 120072 0 0.0
.data 2632 2632 0 0.0
.text 1450528 1450528 0 0.0
light-app default (read/write) 2395832 2395832 0 0.0
.bss 113536 113536 0 0.0
.data 2488 2488 0 0.0
.text 1354096 1354096 0 0.0
lock-app default (read/write) 2359376 2359376 0 0.0
.bss 113280 113280 0 0.0
.data 2448 2448 0 0.0
.text 1317640 1317640 0 0.0
telink lighting-app tlsr9518adk80d (read/write) 895434 895434 0 0.0
bss 87432 87432 0 0.0
noinit 37160 37160 0 0.0
text 633090 633090 0 0.0

@andy31415 andy31415 merged commit bbc944c into project-chip:master Mar 18, 2022
@bzbarsky-apple bzbarsky-apple deleted the fix-test-script-cleanup branch March 18, 2022 18:27
andrei-menzopol pushed a commit to andrei-menzopol/connectedhomeip that referenced this pull request Apr 14, 2022
@bzbarsky-apple @andrei-menzopol
Make Tests job fail quickly instead of timing out on discovery failur…
2834998 
…e. (project-chip#16439)

If waitForAnyAdvertisement failed, we would not actually clean things up and sit there until the job timed out.  Two changes to fix that:

1) Add the app to the app register before calling start() on it, so
   that the killAll() on the app register has a chance of killing the
   app.

2) Set self.process immediately, instead of doing it after things that
   might fail (like waitForAnyAdvertisement), so killing the app actually
   works.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andy31415 andy31415 andy31415 approved these changes

@anush-apple anush-apple Awaiting requested review from anush-apple

@austinh0 austinh0 Awaiting requested review from austinh0

@Byungjoo-Lee Byungjoo-Lee Awaiting requested review from Byungjoo-Lee

@carol-apple carol-apple Awaiting requested review from carol-apple

@chrisdecenzo chrisdecenzo Awaiting requested review from chrisdecenzo

@chshu chshu Awaiting requested review from chshu

@chulspro chulspro Awaiting requested review from chulspro

@Damian-Nordic Damian-Nordic Awaiting requested review from Damian-Nordic

@dhrishi dhrishi Awaiting requested review from dhrishi

@electrocucaracha electrocucaracha Awaiting requested review from electrocucaracha

@emargolis emargolis Awaiting requested review from emargolis

@franck-apple franck-apple Awaiting requested review from franck-apple

@gjc13 gjc13 Awaiting requested review from gjc13

@hawk248 hawk248 Awaiting requested review from hawk248

@holbrookt holbrookt Awaiting requested review from holbrookt

@harsha-rajendran harsha-rajendran Awaiting requested review from harsha-rajendran

@isiu-apple isiu-apple Awaiting requested review from isiu-apple

@jelderton jelderton Awaiting requested review from jelderton

@jepenven-silabs jepenven-silabs Awaiting requested review from jepenven-silabs

@jmartinez-silabs jmartinez-silabs Awaiting requested review from jmartinez-silabs

@lazarkov lazarkov Awaiting requested review from lazarkov

@LuDuda LuDuda Awaiting requested review from LuDuda

@mlepage-google mlepage-google Awaiting requested review from mlepage-google

@msandstedt msandstedt Awaiting requested review from msandstedt

@robszewczyk robszewczyk Awaiting requested review from robszewczyk

@sagar-apple sagar-apple Awaiting requested review from sagar-apple

@saurabhst saurabhst Awaiting requested review from saurabhst

@selissia selissia Awaiting requested review from selissia

@tcarmelveilleux tcarmelveilleux Awaiting requested review from tcarmelveilleux

@tecimovic tecimovic Awaiting requested review from tecimovic

@turon turon Awaiting requested review from turon

@vijs vijs Awaiting requested review from vijs

@vivien-apple vivien-apple Awaiting requested review from vivien-apple

@wbschiller wbschiller Awaiting requested review from wbschiller

@woody-apple woody-apple Awaiting requested review from woody-apple

@xylophone21 xylophone21 Awaiting requested review from xylophone21

@yufengwangca yufengwangca Awaiting requested review from yufengwangca

@yunhanw-google yunhanw-google Awaiting requested review from yunhanw-google

Assignees
No one assigned
Labels
fast track review - pending scripts
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@bzbarsky-apple @andy31415