Testing fixes for TC_SWTCH from TE2 #34984
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- all-clusters-app was not generating button position changes in some cases. This was not detected in some situations since the test cases don't always test for this. - Prompts are missing endpoint ID which makes it hard when running per-endpoint tests to know where it applies. - Some partials could fail on decode errors, causing test errors instead of fails. This PR: - Adds correct generation of positions on press/release. - Adds a way to claim endpoint tested in user prompts - Fixes failing on decode errors in partials Testing done: - TC_SWTCH still passes - Manually validated button position in multi-press test/simulation (update to TC_SWTCH needs test plan changes). Issue is in all-clusters-app for CI only. See CHIP-Specifications/chip-test-plans#4493
tcarmelveilleux
added
the
matter-1.4-te2-script-change
|
Review changes with SemanticDiff. Analyzed 2 of 4 files. Overall, the semantic diff is 19% smaller than the GitHub diff.
|
pullapprove
bot
requested review from
andy31415,
andyg-apple,
anush-apple,
arkq,
axelnxp,
bauerschwan,
bzbarsky-apple,
carol-apple,
cecille,
chapongatien,
chrisdecenzo,
chshu,
chulspro,
cliffamzn,
Damian-Nordic,
dhrishi,
doru91,
fessehaeve,
harimau-qirex,
harsha-rajendran,
hawk248,
hicklin and
jepenven-silabs
pullapprove
bot
requested review from
selissia,
sharadb-amazon,
tecimovic,
tima-q,
tobiasgraf,
turon,
vivien-apple,
wiba-nordic,
woody-apple,
younghak-hwang,
yufengwangca and
yunhanw-google
|
PR #34984: Size comparison from 350b13f to 6c833dc Full report (84 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
cecille
reviewed
Aug 14, 2024
woody-apple
approved these changes
Aug 14, 2024
|
PR #34984: Size comparison from 6d65c0a to 2d62102 Full report (10 builds for nrfconnect, nxp, qpg, stm32)
|
tehampson
approved these changes
Aug 27, 2024
|
PR #34984: Size comparison from d705323 to d56d9e1 Full report (77 builds for bl602, bl702, bl702l, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
hasty
pushed a commit
to hasty/connectedhomeip
that referenced
this pull request
Aug 29, 2024
* Testing fixes for TC_SWTCH from TE2 - all-clusters-app was not generating button position changes in some cases. This was not detected in some situations since the test cases don't always test for this. - Prompts are missing endpoint ID which makes it hard when running per-endpoint tests to know where it applies. - Some partials could fail on decode errors, causing test errors instead of fails. This PR: - Adds correct generation of positions on press/release. - Adds a way to claim endpoint tested in user prompts - Fixes failing on decode errors in partials Testing done: - TC_SWTCH still passes - Manually validated button position in multi-press test/simulation (update to TC_SWTCH needs test plan changes). Issue is in all-clusters-app for CI only. See CHIP-Specifications/chip-test-plans#4493 * Restyled by autopep8 * Update prompt support --------- Co-authored-by: Restyled.io <[email protected]>
shgutte
pushed a commit
to shgutte/connectedhomeip
that referenced
this pull request
Sep 10, 2024
* Testing fixes for TC_SWTCH from TE2 - all-clusters-app was not generating button position changes in some cases. This was not detected in some situations since the test cases don't always test for this. - Prompts are missing endpoint ID which makes it hard when running per-endpoint tests to know where it applies. - Some partials could fail on decode errors, causing test errors instead of fails. This PR: - Adds correct generation of positions on press/release. - Adds a way to claim endpoint tested in user prompts - Fixes failing on decode errors in partials Testing done: - TC_SWTCH still passes - Manually validated button position in multi-press test/simulation (update to TC_SWTCH needs test plan changes). Issue is in all-clusters-app for CI only. See CHIP-Specifications/chip-test-plans#4493 * Restyled by autopep8 * Update prompt support --------- Co-authored-by: Restyled.io <[email protected]>
mergify bot
pushed a commit
that referenced
this pull request
Oct 23, 2024
* [HVAC] Check if number of preset scenarios exceeds maximum number of scenarios * [NXP][Zephyr] Provide AP band in connection request parameters (#35181) Signed-off-by: Axel Le Bourhis <[email protected]> * Plumbing for CADMIN attribute updates from fabric-admin to fabric-bridge (#35222) * Fix TC_BRBINFO_4_1 for execution on TH (#35257) * [Fabric-Admin] Move DeviceSynchronization from pairing command to device_manager (#35260) * Move DeviceSynchronization from pairing command to device_manager * Restyled by gn --------- Co-authored-by: Restyled.io <[email protected]> * Add command-line argument to allow userprompt at start of ECOINFO_2_1 (#35234) --------- Co-authored-by: Restyled.io <[email protected]> Co-authored-by: saurabhst <[email protected]> * Testing fixes for TC_SWTCH from TE2 (#34984) * Testing fixes for TC_SWTCH from TE2 - all-clusters-app was not generating button position changes in some cases. This was not detected in some situations since the test cases don't always test for this. - Prompts are missing endpoint ID which makes it hard when running per-endpoint tests to know where it applies. - Some partials could fail on decode errors, causing test errors instead of fails. This PR: - Adds correct generation of positions on press/release. - Adds a way to claim endpoint tested in user prompts - Fixes failing on decode errors in partials Testing done: - TC_SWTCH still passes - Manually validated button position in multi-press test/simulation (update to TC_SWTCH needs test plan changes). Issue is in all-clusters-app for CI only. See CHIP-Specifications/chip-test-plans#4493 * Restyled by autopep8 * Update prompt support --------- Co-authored-by: Restyled.io <[email protected]> * Add test cases for testing additional Presets write and commit constr… (#35141) * Add test cases for testing additional Presets write and commit constraints - Add a test for adding a preset with a preset scenario not present in PresetTypes - Add a test for testing addition of presets such that the total number of presets added is greater than the total number of presets supported * Add rollback after test step 18 * Modify the number of presets supported test case to read the number of presets supported and build a preset list whose size exceeds that to test * Modify the number of presets supported test case to read the number of presets supported and build a preset list whose size exceeds that to test * Update thermostat-delegate-impl.h * Address review comments * Add support to check for numberOfPresets supported for each preset type and build the presets list with multiple presets of each type * Restyled by autopep8 * Fix log line formatting * Update src/python_testing/TC_TSTAT_4_2.py Co-authored-by: Boris Zbarsky <[email protected]> * Fix test step 17 to find a preset scenario in PresetScenarioEnum that is not present in PresetTypes to run the test - Fix test step 18 to build a presets list that exceeds the number of presets supported correctly * Restyled by autopep8 * Fix lint errors * Add a while loop to add more presets until max is reached --------- Co-authored-by: Restyled.io <[email protected]> Co-authored-by: Boris Zbarsky <[email protected]> * Allow TestAccessControl to run with ARL (#35262) * Allow TestAccessControl to run with ARL Since RequestType is required, set each test data entry to some value that will pass AccessRestrictionProvider checks (since the focus is on AccessControl for these tests). * Copy the test data's request path and optionally add RequestType * Make zap_downloadl.py create a usable zap.app on Mac (#35242) Use the unzip utility on Mac for unzipping instead of zipfile. In addition to not supporting file modes (which the script already works around) the zipfile module also doesn't support symlinks. The embedded frameworks inside zap.app rely on symlinks for the application to work. * TBRM Tests scripts consistency with te2 fixes (#35153) * Add files via upload Add yaml test script for TBRM * Update TEST_TC_TBRM_2.2.yaml * Update TEST_TC_TBRM_2.3.yaml * Update TEST_TC_TBRM_2.4.yaml * Test script consitancy wit test plan after TE2 * Test script consitancy wit test plan after TE2 * Update src/app/tests/suites/certification/Test_TC_TBRM_2_3.yaml Co-authored-by: Karsten Sperling <[email protected]> * Update src/app/tests/suites/certification/Test_TC_TBRM_2_2.yaml Co-authored-by: Karsten Sperling <[email protected]> * Restyled by whitespace * Restyled by prettier-yaml * Test_TC_TBRM_2_4. synchronisation with TC-THNETDIR-2.3 according test Plan * Restyled by whitespace * Test tweaks to get CI to pass - Use pairing payload matching the other parameters - Check response of ArmFailSafe commands - Fix bad merge in commissioner_commands.py * Restyled by prettier-yaml --------- Co-authored-by: StephaneGUELEC <[email protected]> Co-authored-by: Karsten Sperling <[email protected]> Co-authored-by: Restyled.io <[email protected]> Co-authored-by: Karsten Sperling <[email protected]> * [HVAC] Alter Thermostat Preset tests to not rely on knowledge of the server's initial state * Pick midpoint setpoints for new presets * Lint fixes * Apply suggestions from code review Co-authored-by: Boris Zbarsky <[email protected]> * Fixes from code review * Apply suggestions from code review Co-authored-by: Boris Zbarsky <[email protected]> * Fix remaining places with hard-coded setpoints * Don't abort test if there are no built-in presets * Remove unneeded length check * Fix max number of preset types * Add test for individual preset scenario limits * Fix lint issue * Return invalid in state if we're unable to iterate over the preset types for some reason * Apply suggestions from code review Co-authored-by: Boris Zbarsky <[email protected]> * Remove unneeded active preset setting * Restyled patch * Suggestions from code review --------- Signed-off-by: Axel Le Bourhis <[email protected]> Co-authored-by: Axel Le Bourhis <[email protected]> Co-authored-by: Terence Hampson <[email protected]> Co-authored-by: Yufeng Wang <[email protected]> Co-authored-by: Restyled.io <[email protected]> Co-authored-by: saurabhst <[email protected]> Co-authored-by: Tennessee Carmel-Veilleux <[email protected]> Co-authored-by: Nivi Sarkar <[email protected]> Co-authored-by: Boris Zbarsky <[email protected]> Co-authored-by: Thomas Lea <[email protected]> Co-authored-by: Karsten Sperling <[email protected]> Co-authored-by: marchemi <[email protected]> Co-authored-by: StephaneGUELEC <[email protected]> Co-authored-by: Karsten Sperling <[email protected]>
yyzhong-g
pushed a commit
to yyzhong-g/connectedhomeip
that referenced
this pull request
Dec 12, 2024
* [HVAC] Check if number of preset scenarios exceeds maximum number of scenarios * [NXP][Zephyr] Provide AP band in connection request parameters (project-chip#35181) Signed-off-by: Axel Le Bourhis <[email protected]> * Plumbing for CADMIN attribute updates from fabric-admin to fabric-bridge (project-chip#35222) * Fix TC_BRBINFO_4_1 for execution on TH (project-chip#35257) * [Fabric-Admin] Move DeviceSynchronization from pairing command to device_manager (project-chip#35260) * Move DeviceSynchronization from pairing command to device_manager * Restyled by gn --------- Co-authored-by: Restyled.io <[email protected]> * Add command-line argument to allow userprompt at start of ECOINFO_2_1 (project-chip#35234) --------- Co-authored-by: Restyled.io <[email protected]> Co-authored-by: saurabhst <[email protected]> * Testing fixes for TC_SWTCH from TE2 (project-chip#34984) * Testing fixes for TC_SWTCH from TE2 - all-clusters-app was not generating button position changes in some cases. This was not detected in some situations since the test cases don't always test for this. - Prompts are missing endpoint ID which makes it hard when running per-endpoint tests to know where it applies. - Some partials could fail on decode errors, causing test errors instead of fails. This PR: - Adds correct generation of positions on press/release. - Adds a way to claim endpoint tested in user prompts - Fixes failing on decode errors in partials Testing done: - TC_SWTCH still passes - Manually validated button position in multi-press test/simulation (update to TC_SWTCH needs test plan changes). Issue is in all-clusters-app for CI only. See https://github.com/CHIP-Specifications/chip-test-plans/issues/4493 * Restyled by autopep8 * Update prompt support --------- Co-authored-by: Restyled.io <[email protected]> * Add test cases for testing additional Presets write and commit constr… (project-chip#35141) * Add test cases for testing additional Presets write and commit constraints - Add a test for adding a preset with a preset scenario not present in PresetTypes - Add a test for testing addition of presets such that the total number of presets added is greater than the total number of presets supported * Add rollback after test step 18 * Modify the number of presets supported test case to read the number of presets supported and build a preset list whose size exceeds that to test * Modify the number of presets supported test case to read the number of presets supported and build a preset list whose size exceeds that to test * Update thermostat-delegate-impl.h * Address review comments * Add support to check for numberOfPresets supported for each preset type and build the presets list with multiple presets of each type * Restyled by autopep8 * Fix log line formatting * Update src/python_testing/TC_TSTAT_4_2.py Co-authored-by: Boris Zbarsky <[email protected]> * Fix test step 17 to find a preset scenario in PresetScenarioEnum that is not present in PresetTypes to run the test - Fix test step 18 to build a presets list that exceeds the number of presets supported correctly * Restyled by autopep8 * Fix lint errors * Add a while loop to add more presets until max is reached --------- Co-authored-by: Restyled.io <[email protected]> Co-authored-by: Boris Zbarsky <[email protected]> * Allow TestAccessControl to run with ARL (project-chip#35262) * Allow TestAccessControl to run with ARL Since RequestType is required, set each test data entry to some value that will pass AccessRestrictionProvider checks (since the focus is on AccessControl for these tests). * Copy the test data's request path and optionally add RequestType * Make zap_downloadl.py create a usable zap.app on Mac (project-chip#35242) Use the unzip utility on Mac for unzipping instead of zipfile. In addition to not supporting file modes (which the script already works around) the zipfile module also doesn't support symlinks. The embedded frameworks inside zap.app rely on symlinks for the application to work. * TBRM Tests scripts consistency with te2 fixes (project-chip#35153) * Add files via upload Add yaml test script for TBRM * Update TEST_TC_TBRM_2.2.yaml * Update TEST_TC_TBRM_2.3.yaml * Update TEST_TC_TBRM_2.4.yaml * Test script consitancy wit test plan after TE2 * Test script consitancy wit test plan after TE2 * Update src/app/tests/suites/certification/Test_TC_TBRM_2_3.yaml Co-authored-by: Karsten Sperling <[email protected]> * Update src/app/tests/suites/certification/Test_TC_TBRM_2_2.yaml Co-authored-by: Karsten Sperling <[email protected]> * Restyled by whitespace * Restyled by prettier-yaml * Test_TC_TBRM_2_4. synchronisation with TC-THNETDIR-2.3 according test Plan * Restyled by whitespace * Test tweaks to get CI to pass - Use pairing payload matching the other parameters - Check response of ArmFailSafe commands - Fix bad merge in commissioner_commands.py * Restyled by prettier-yaml --------- Co-authored-by: StephaneGUELEC <[email protected]> Co-authored-by: Karsten Sperling <[email protected]> Co-authored-by: Restyled.io <[email protected]> Co-authored-by: Karsten Sperling <[email protected]> * [HVAC] Alter Thermostat Preset tests to not rely on knowledge of the server's initial state * Pick midpoint setpoints for new presets * Lint fixes * Apply suggestions from code review Co-authored-by: Boris Zbarsky <[email protected]> * Fixes from code review * Apply suggestions from code review Co-authored-by: Boris Zbarsky <[email protected]> * Fix remaining places with hard-coded setpoints * Don't abort test if there are no built-in presets * Remove unneeded length check * Fix max number of preset types * Add test for individual preset scenario limits * Fix lint issue * Return invalid in state if we're unable to iterate over the preset types for some reason * Apply suggestions from code review Co-authored-by: Boris Zbarsky <[email protected]> * Remove unneeded active preset setting * Restyled patch * Suggestions from code review --------- Signed-off-by: Axel Le Bourhis <[email protected]> Co-authored-by: Axel Le Bourhis <[email protected]> Co-authored-by: Terence Hampson <[email protected]> Co-authored-by: Yufeng Wang <[email protected]> Co-authored-by: Restyled.io <[email protected]> Co-authored-by: saurabhst <[email protected]> Co-authored-by: Tennessee Carmel-Veilleux <[email protected]> Co-authored-by: Nivi Sarkar <[email protected]> Co-authored-by: Boris Zbarsky <[email protected]> Co-authored-by: Thomas Lea <[email protected]> Co-authored-by: Karsten Sperling <[email protected]> Co-authored-by: marchemi <[email protected]> Co-authored-by: StephaneGUELEC <[email protected]> Co-authored-by: Karsten Sperling <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR:
Testing done: