Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add some options to dump the message expected by the Test Harness #26054

Merged

Conversation

vivien-apple
Copy link
Contributor

Problem

In order to use the python runner, the test harness expects some special strings.
This is an attempt to get it correct.

@vivien-apple vivien-apple self-assigned this Apr 11, 2023
@github-actions
Copy link

PR #26054: Size comparison from 1636cd1 to 7afa882

Increases (1 build for cc32xx)
platform target config section 1636cd1 7afa882 change % change
cc32xx lock CC3235SF_LAUNCHXL .debug_info 20303346 20303347 1 0.0
Full report (1 build for cc32xx)
platform target config section 1636cd1 7afa882 change % change
cc32xx lock CC3235SF_LAUNCHXL 0 0 0 0.0
(read only) 642953 642953 0 0.0
(read/write) 203848 203848 0 0.0
.ARM.attributes 44 44 0 0.0
.ARM.exidx 8 8 0 0.0
.bss 197248 197248 0 0.0
.comment 194 194 0 0.0
.data 1480 1480 0 0.0
.debug_abbrev 933129 933129 0 0.0
.debug_aranges 87616 87616 0 0.0
.debug_frame 301352 301352 0 0.0
.debug_info 20303346 20303347 1 0.0
.debug_line 2679969 2679969 0 0.0
.debug_loc 2824782 2824782 0 0.0
.debug_ranges 286208 286208 0 0.0
.debug_str 3039557 3039557 0 0.0
.ramVecs 780 780 0 0.0
.resetVecs 64 64 0 0.0
.rodata 104353 104353 0 0.0
.shstrtab 232 232 0 0.0
.stab 204 204 0 0.0
.stabstr 441 441 0 0.0
.stack 2048 2048 0 0.0
.strtab 377533 377533 0 0.0
.symtab 256800 256800 0 0.0
.text 536480 536480 0 0.0

@vivien-apple vivien-apple force-pushed the TestHarness_PythonLogs branch from 7afa882 to 143dcc8 Compare April 14, 2023 10:16
@vivien-apple
Copy link
Contributor Author

I have updated the PR to contains logs for the logs command as well as some logs when the device connection succeed or fails.

@github-actions
Copy link

PR #26054: Size comparison from 62974a0 to 72883a1

Increases (1 build for cc32xx)
platform target config section 62974a0 72883a1 change % change
cc32xx lock CC3235SF_LAUNCHXL .debug_info 20326060 20326061 1 0.0
Full report (1 build for cc32xx)
platform target config section 62974a0 72883a1 change % change
cc32xx lock CC3235SF_LAUNCHXL 0 0 0 0.0
(read only) 643081 643081 0 0.0
(read/write) 203848 203848 0 0.0
.ARM.attributes 44 44 0 0.0
.ARM.exidx 8 8 0 0.0
.bss 197248 197248 0 0.0
.comment 194 194 0 0.0
.data 1480 1480 0 0.0
.debug_abbrev 933224 933224 0 0.0
.debug_aranges 87760 87760 0 0.0
.debug_frame 302028 302028 0 0.0
.debug_info 20326060 20326061 1 0.0
.debug_line 2687403 2687403 0 0.0
.debug_loc 2838361 2838361 0 0.0
.debug_ranges 288040 288040 0 0.0
.debug_str 3042379 3042379 0 0.0
.ramVecs 780 780 0 0.0
.resetVecs 64 64 0 0.0
.rodata 104393 104393 0 0.0
.shstrtab 232 232 0 0.0
.stab 204 204 0 0.0
.stabstr 441 441 0 0.0
.stack 2048 2048 0 0.0
.strtab 377626 377626 0 0.0
.symtab 256832 256832 0 0.0
.text 536568 536568 0 0.0

scripts/py_matter_yamltests/matter_yamltests/hooks.py Outdated Show resolved Hide resolved
scripts/tests/yaml/runner.py Outdated Show resolved Hide resolved
@vivien-apple vivien-apple force-pushed the TestHarness_PythonLogs branch from 72883a1 to 6b472c5 Compare April 18, 2023 10:42
@github-actions
Copy link

PR #26054: Size comparison from 4ccbbb7 to 6b472c5

Increases (1 build for cc32xx)
platform target config section 4ccbbb7b 6b472c5 change % change
cc32xx lock CC3235SF_LAUNCHXL .debug_info 20330829 20330830 1 0.0
Full report (1 build for cc32xx)
platform target config section 4ccbbb7b 6b472c5 change % change
cc32xx lock CC3235SF_LAUNCHXL 0 0 0 0.0
(read only) 643249 643249 0 0.0
(read/write) 203848 203848 0 0.0
.ARM.attributes 44 44 0 0.0
.ARM.exidx 8 8 0 0.0
.bss 197248 197248 0 0.0
.comment 194 194 0 0.0
.data 1480 1480 0 0.0
.debug_abbrev 933224 933224 0 0.0
.debug_aranges 87792 87792 0 0.0
.debug_frame 302140 302140 0 0.0
.debug_info 20330829 20330830 1 0.0
.debug_line 2687904 2687904 0 0.0
.debug_loc 2838960 2838960 0 0.0
.debug_ranges 288072 288072 0 0.0
.debug_str 3042335 3042335 0 0.0
.ramVecs 780 780 0 0.0
.resetVecs 64 64 0 0.0
.rodata 104401 104401 0 0.0
.shstrtab 232 232 0 0.0
.stab 204 204 0 0.0
.stabstr 441 441 0 0.0
.stack 2048 2048 0 0.0
.strtab 377963 377963 0 0.0
.symtab 256976 256976 0 0.0
.text 536728 536728 0 0.0

@mikaelhm
Copy link
Contributor

I verified with a PoC integration with Matter TH that this does work.

Screenshot 2023-04-28 at 8 56 52 AM

@mikaelhm
Copy link
Contributor

mikaelhm commented May 1, 2023

@vivien-apple these are the strings TH is currently scanning for in chip-tool and chip-app log output:


#chip-tool test flow
"***** Test Step"
"***** Test Failure:"
"***** Step Failure:"
"**** Skipping:"

#chip-tool user-prompt matches:
"USER_PROMPT"

#chip-app - prompt tester to perform action
"Wait for"

#chip-app failure:
"AttributePath does not match"

#chip-app manual pairing code:
"Manual pairing code:"

@vivien-apple vivien-apple merged commit 34b2a80 into project-chip:master May 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants