Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix and enable Matter Casting automation CI check for commissioner-generated-passcode flow. #34178

Conversation

shaoltan-amazon
Copy link
Contributor

Fixes #34151

Problem
The commit with the CI check for the commissioner-generated-passcode was reverted as it was consistently failing after being merged in.

Solution
Fixed the string that was causing the error.

Testing

  1. Pulled the latest changes of the connectedhomeip repository.
  2. Built the Linux tv-app locally by running: scripts/examples/gn_build_example.sh examples/tv-app/linux/ out/tv-app
  3. Built the Linux tv-casting-app locally by running: scripts/examples/gn_build_example.sh examples/tv-casting-app/linux/ out/tv-casting-app chip_casting_simplified=true
  4. Verified that the test script still works for the commissionee-generated-passcode flow by running: python3 ./scripts/tests/run_tv_casting_test.py
  5. Verified that the test script still works for the commissioner-generated-passcode flow by running: python3 ./scripts/tests/run_tv_casting_test.py --commissioner-generated-passcode=True

Copy link

github-actions bot commented Jul 3, 2024

PR #34178: Size comparison from f8a633e to 6c02db3

Full report (85 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, mbed, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
platform target config section f8a633e 6c02db3 change % change
bl602 lighting-app bl602 FLASH 1271088 1271088 0 0.0
RAM 95328 95328 0 0.0
bl602+mfd FLASH 1285602 1285602 0 0.0
RAM 95472 95472 0 0.0
bl602+rpc FLASH 1310304 1310304 0 0.0
RAM 103752 103752 0 0.0
bl702 lighting-app bl702 FLASH 1092238 1092238 0 0.0
RAM 15161 15161 0 0.0
bl702+mfd FLASH 1102932 1102932 0 0.0
RAM 15313 15313 0 0.0
bl702+rpc FLASH 1182048 1182048 0 0.0
RAM 24181 24181 0 0.0
bl706-eth FLASH 875328 875328 0 0.0
RAM 27272 27272 0 0.0
bl706-wifi FLASH 1127724 1127724 0 0.0
RAM 14605 14605 0 0.0
bl702l lighting-app bl702l FLASH 1079124 1079124 0 0.0
RAM 21732 21732 0 0.0
bl702l+mfd FLASH 1090386 1090386 0 0.0
RAM 21892 21892 0 0.0
cc13x4_26x4 lighting-app LP_EM_CC1354P10_6 FLASH 797344 797344 0 0.0
RAM 106636 106636 0 0.0
lock-ftd LP_EM_CC1354P10_6 FLASH 816260 816260 0 0.0
RAM 116948 116948 0 0.0
lock-mtd LP_EM_CC1354P10_6 FLASH 807712 807712 0 0.0
RAM 111236 111236 0 0.0
pump-app LP_EM_CC1354P10_6 FLASH 760552 760552 0 0.0
RAM 105328 105328 0 0.0
pump-controller-app LP_EM_CC1354P10_6 FLASH 746272 746272 0 0.0
RAM 105576 105576 0 0.0
cc32xx air-purifier CC3235SF_LAUNCHXL FLASH 606186 606186 0 0.0
RAM 204508 204508 0 0.0
lock CC3235SF_LAUNCHXL FLASH 651214 651214 0 0.0
RAM 204780 204780 0 0.0
cyw30739 light CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 666345 666345 0 0.0
RAM 75100 75100 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 686197 686197 0 0.0
RAM 77732 77732 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 686197 686197 0 0.0
RAM 77732 77732 0 0.0
CYW930739M2EVB-02 unknown 2040 2040 0 0.0
FLASH 643133 643133 0 0.0
RAM 70168 70168 0 0.0
light-switch CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 608873 608873 0 0.0
RAM 70804 70804 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 628517 628517 0 0.0
RAM 73356 73356 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 628517 628517 0 0.0
RAM 73356 73356 0 0.0
lock CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 624401 624401 0 0.0
RAM 73820 73820 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 644117 644117 0 0.0
RAM 76372 76372 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 644117 644117 0 0.0
RAM 76372 76372 0 0.0
thermostat CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 592973 592973 0 0.0
RAM 67788 67788 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 612825 612825 0 0.0
RAM 70420 70420 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 612825 612825 0 0.0
RAM 70420 70420 0 0.0
efr32 lighting-app BRD4187C FLASH 923444 923436 -8 -0.0
RAM 134996 134996 0 0.0
lock-app BRD4338a FLASH 740132 740132 0 0.0
RAM 203996 203996 0 0.0
window-app BRD4187C FLASH 1012376 1012368 -8 -0.0
RAM 129632 129632 0 0.0
esp32 all-clusters-app c3devkit DRAM 88396 88396 0 0.0
FLASH 1468016 1468016 0 0.0
IRAM 75570 75570 0 0.0
m5stack DRAM 114868 114868 0 0.0
FLASH 1537007 1537007 0 0.0
IRAM 125403 125403 0 0.0
linux air-purifier-app debug unknown 4592 4592 0 0.0
FLASH 2530224 2530224 0 0.0
RAM 125112 125112 0 0.0
all-clusters-app debug unknown 5368 5368 0 0.0
FLASH 5584014 5584014 0 0.0
RAM 488152 488152 0 0.0
all-clusters-minimal-app debug unknown 5288 5288 0 0.0
FLASH 5053944 5053944 0 0.0
RAM 232968 232968 0 0.0
bridge-app debug unknown 5256 5256 0 0.0
FLASH 4478440 4478440 0 0.0
RAM 212832 212832 0 0.0
chip-tool debug unknown 5728 5728 0 0.0
FLASH 11714375 11714375 0 0.0
RAM 543394 543394 0 0.0
chip-tool-ipv6only arm64 unknown 20032 20032 0 0.0
FLASH 10830716 10830716 0 0.0
RAM 592368 592368 0 0.0
fabric-admin debug unknown 5616 5616 0 0.0
FLASH 10797751 10797751 0 0.0
RAM 540266 540266 0 0.0
fabric-bridge-app debug unknown 4528 4528 0 0.0
FLASH 4243992 4243992 0 0.0
RAM 198832 198832 0 0.0
lighting-app debug+rpc+ui unknown 5936 5936 0 0.0
FLASH 5373682 5373682 0 0.0
RAM 221640 221640 0 0.0
lock-app debug unknown 5192 5192 0 0.0
FLASH 4542008 4542008 0 0.0
RAM 200248 200248 0 0.0
ota-provider-app debug unknown 4576 4576 0 0.0
FLASH 4197880 4197880 0 0.0
RAM 194544 194544 0 0.0
ota-requestor-app debug unknown 4512 4512 0 0.0
FLASH 4323240 4323240 0 0.0
RAM 199168 199168 0 0.0
shell debug unknown 4112 4112 0 0.0
FLASH 2796957 2796957 0 0.0
RAM 150512 150512 0 0.0
thermostat-no-ble arm64 unknown 9184 9184 0 0.0
FLASH 4168332 4168332 0 0.0
RAM 235864 235864 0 0.0
tv-app debug unknown 5472 5472 0 0.0
FLASH 5605704 5605704 0 0.0
RAM 341800 341800 0 0.0
tv-casting-app debug unknown 5096 5096 0 0.0
FLASH 9883950 9883950 0 0.0
RAM 400824 400824 0 0.0
mbed lock-app-release cy8cproto_062_4343w FLASH 1502428 1502428 0 0.0
RAM 226656 226656 0 0.0
nrfconnect all-clusters-app nrf52840dk_nrf52840 FLASH 881476 881476 0 0.0
RAM 139693 139693 0 0.0
nrf7002dk_nrf5340_cpuapp FLASH 951992 951992 0 0.0
RAM 138121 138121 0 0.0
all-clusters-minimal-app nrf52840dk_nrf52840 FLASH 826988 826988 0 0.0
RAM 138591 138591 0 0.0
nxp contact k32w0+release FLASH 575996 575996 0 0.0
RAM 70024 70024 0 0.0
k32w1+release FLASH 591232 591232 0 0.0
RAM 74056 74056 0 0.0
light k32w0+release FLASH 610120 610120 0 0.0
RAM 69500 69500 0 0.0
k32w1+release FLASH 674824 674824 0 0.0
RAM 82816 82816 0 0.0
psoc6 all-clusters cy8ckit_062s2_43012 FLASH 1614140 1614140 0 0.0
RAM 207148 207148 0 0.0
all-clusters-minimal cy8ckit_062s2_43012 FLASH 1534908 1534908 0 0.0
RAM 204052 204052 0 0.0
light cy8ckit_062s2_43012 FLASH 1461404 1461404 0 0.0
RAM 197332 197332 0 0.0
lock cy8ckit_062s2_43012 FLASH 1463268 1463268 0 0.0
RAM 224396 224396 0 0.0
qpg lighting-app qpg6105+debug FLASH 651156 651156 0 0.0
RAM 104564 104564 0 0.0
lock-app qpg6105+debug FLASH 611280 611280 0 0.0
RAM 99240 99240 0 0.0
stm32 light STM32WB5MM-DK FLASH 472544 472544 0 0.0
RAM 141652 141652 0 0.0
telink air-quality-sensor-app tlsr9528a_retention FLASH 632646 632646 0 0.0
RAM 50528 50528 0 0.0
all-clusters-app tlsr9118bdk40d FLASH 657414 657414 0 0.0
RAM 145872 145872 0 0.0
all-clusters-minimal-app tlsr9528a FLASH 777718 777718 0 0.0
RAM 110684 110684 0 0.0
bridge-app tlsr9258a FLASH 675642 675642 0 0.0
RAM 95304 95304 0 0.0
contact-sensor-app tlsr9528a_retention FLASH 634230 634230 0 0.0
RAM 50572 50572 0 0.0
light-switch-app-ota-shell-factory-data tlsr9528a FLASH 720030 720030 0 0.0
RAM 77148 77148 0 0.0
lighting-app-ota-factory-data tlsr9118bdk40d FLASH 612458 612458 0 0.0
RAM 142104 142104 0 0.0
lighting-app-ota-rpc-factory-data-4mb tlsr9518adk80d FLASH 800244 800244 0 0.0
RAM 100508 100508 0 0.0
lock-app-dfu tlsr9528a FLASH 665804 665804 0 0.0
RAM 69860 69860 0 0.0
ota-requestor-app tlsr9258a FLASH 694972 694972 0 0.0
RAM 95028 95028 0 0.0
pump-app tlsr9518adk80d FLASH 616504 616504 0 0.0
RAM 56952 56952 0 0.0
pump-controller-app tlsr9518adk80d FLASH 606888 606888 0 0.0
RAM 56752 56752 0 0.0
shell tlsr9518adk80d FLASH 466214 466214 0 0.0
RAM 72484 72484 0 0.0
smoke_co_alarm-app tlsr9528a_retention FLASH 640844 640844 0 0.0
RAM 52200 52200 0 0.0
temperature-measurement-app-mars-ota tlsr9518adk80d FLASH 650714 650714 0 0.0
RAM 60388 60388 0 0.0
thermostat tlsr9518adk80d FLASH 625778 625778 0 0.0
RAM 57084 57084 0 0.0
window-covering tlsr9118bdk40d FLASH 519004 519004 0 0.0
RAM 97800 97800 0 0.0
tizen all-clusters-app arm unknown 1584 1584 0 0.0
FLASH 1634688 1634688 0 0.0
RAM 46012 46012 0 0.0
chip-tool-ubsan arm unknown 2384 2384 0 0.0
FLASH 16156442 16156442 0 0.0
RAM 7097980 7097980 0 0.0

@sharadb-amazon sharadb-amazon enabled auto-merge (squash) July 3, 2024 19:21
@ksperling-apple
Copy link
Contributor

The test failure on the original version was usually an mDNS timeout, see e.g. https://github.com/project-chip/connectedhomeip/actions/runs/9749187740/job/26905811716

Is that what you fixed?

I think it would be good to just re-run the CI job a few times to make sure it passes consistently.

@shaoltan-amazon
Copy link
Contributor Author

Hi @ksperling-apple, the timeout was because the test couldn't find the string CommandHandler() setcommissionerpasscode user entered passcode: 12345678 in the Linux tv-casting-app output as noted in the error log: ERROR - commissioner_generated_passcode_test - Did not find the expected output string(s) in the tv-casting-app subprocess within the timeout: ['CommandHandler() setcommissionerpasscode user entered passcode: 12345678']. See line 95 under the job step Test casting from Linux tv-casting-app to Linux tv-app - Commissioner Generated Passcode for reference. This was due to a string format mismatch that occurred when the source code got updated and merged in (PR-34093) right before the CI check PR-34079. Per the latest source code, the string should have a dash (-) between the words user and entered -> CommandHandler() setcommissionerpasscode user-entered passcode: 12345678. Here is the string in the source code for reference.

So the fix was to update the string in the test script to match the latest string format in the source code.

Copy link
Contributor

@ksperling-apple ksperling-apple left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI Test passed 3 times, looks good to me.

@sharadb-amazon sharadb-amazon merged commit 35eba86 into project-chip:master Jul 4, 2024
68 checks passed
austina-csa pushed a commit to austina-csa/connectedhomeip that referenced this pull request Jul 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants