Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TC_EEVSE_2.2 through 2.5 python script has to be updated as per the test plan changes #35399

Conversation

jamesharrow
Copy link
Contributor

@jamesharrow jamesharrow commented Sep 4, 2024

Some late notice change to align the test script words with test plan.

  • Mostly 'TestStep' changes to align with test plan (see PR: https://github.com/CHIP-Specifications/chip-test-plans/pull/4071)

    • A new auto-gen script generated the text so this should align with the test plan.
    • Found a minor issue in the test plan when using that
  • TC_EEVSE_2.5 has new step 8 which tries to send a EnableCharging when in Diagnostics test mode, so this vaildates it gets a Failure in response.

Note that most of the test step changes are auto-gen'd from the test plan (with macro expansion) - so should align.

Copy link

semanticdiff-com bot commented Sep 4, 2024

Review changes with SemanticDiff.

Analyzed 4 of 4 files.

Overall, the semantic diff is 71% smaller than the GitHub diff.

Filename Status
✔️ src/python_testing/TC_EEVSE_2_2.py 75.56% smaller
✔️ src/python_testing/TC_EEVSE_2_3.py 56.91% smaller
✔️ src/python_testing/TC_EEVSE_2_4.py 82.17% smaller
✔️ src/python_testing/TC_EEVSE_2_5.py 63.94% smaller

@github-actions github-actions bot added tests matter-1.4-te2-script-change Script changes before end of Matter 1.4 TE2 labels Sep 4, 2024
@jamesharrow jamesharrow changed the base branch from 34100-tc-eevse-2224-25-python-script-has-to-be-updated-as-per-the-test-plan-changes to master September 4, 2024 09:47
@jamesharrow jamesharrow marked this pull request as ready for review September 4, 2024 09:48
@jamesharrow jamesharrow changed the title tc eevse 2224 25 python script has to be updated as per the test plan changes TC_EEVSE_2.2 through 2.5 python script has to be updated as per the test plan changes Sep 4, 2024
Copy link

github-actions bot commented Sep 4, 2024

PR #35399: Size comparison from 1e1fe1e to 36af6e4

Full report (77 builds for bl602, bl702, bl702l, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
platform target config section 1e1fe1e 36af6e4 change % change
bl602 lighting-app bl602 FLASH 1279604 1279604 0 0.0
RAM 95904 95904 0 0.0
bl602+mfd FLASH 1293458 1293458 0 0.0
RAM 96048 96048 0 0.0
bl602+rpc FLASH 1318568 1318568 0 0.0
RAM 104328 104328 0 0.0
bl702 lighting-app bl702 FLASH 945190 945190 0 0.0
RAM 15209 15209 0 0.0
bl702+mfd FLASH 947334 947334 0 0.0
RAM 15353 15353 0 0.0
bl702+rpc FLASH 1040246 1040246 0 0.0
RAM 24221 24221 0 0.0
bl706-eth FLASH 648006 648006 0 0.0
RAM 25305 25305 0 0.0
bl706-wifi FLASH 895420 895420 0 0.0
RAM 14525 14525 0 0.0
bl702l lighting-app bl702l FLASH 961594 961594 0 0.0
RAM 16868 16868 0 0.0
bl702l+mfd FLASH 964216 964216 0 0.0
RAM 17012 17012 0 0.0
cyw30739 light CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 677257 677257 0 0.0
RAM 78324 78324 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 697117 697117 0 0.0
RAM 80956 80956 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 697117 697117 0 0.0
RAM 80956 80956 0 0.0
CYW930739M2EVB-02 unknown 2040 2040 0 0.0
FLASH 654045 654045 0 0.0
RAM 73392 73392 0 0.0
light-switch CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 611193 611193 0 0.0
RAM 71292 71292 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 630821 630821 0 0.0
RAM 73844 73844 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 630821 630821 0 0.0
RAM 73844 73844 0 0.0
lock CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 629233 629233 0 0.0
RAM 74308 74308 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 648941 648941 0 0.0
RAM 76860 76860 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 648941 648941 0 0.0
RAM 76860 76860 0 0.0
thermostat CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 603501 603501 0 0.0
RAM 68348 68348 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 623361 623361 0 0.0
RAM 70980 70980 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 623361 623361 0 0.0
RAM 70980 70980 0 0.0
efr32 lighting-app BRD4187C FLASH 935316 935316 0 0.0
RAM 135120 135120 0 0.0
lock-app BRD4338a FLASH 740764 740764 0 0.0
RAM 197184 197184 0 0.0
window-app BRD4187C FLASH 1017456 1017448 -8 -0.0
RAM 127060 127060 0 0.0
esp32 all-clusters-app c3devkit DRAM 94224 94224 0 0.0
FLASH 1540250 1540250 0 0.0
IRAM 82538 82538 0 0.0
m5stack DRAM 115160 115160 0 0.0
FLASH 1550698 1550698 0 0.0
IRAM 117039 117039 0 0.0
linux air-purifier-app debug unknown 4624 4624 0 0.0
FLASH 2763665 2763665 0 0.0
RAM 128536 128536 0 0.0
all-clusters-app debug unknown 5408 5408 0 0.0
FLASH 6078532 6078532 0 0.0
RAM 510536 510536 0 0.0
all-clusters-minimal-app debug unknown 5304 5304 0 0.0
FLASH 5402956 5402956 0 0.0
RAM 240376 240376 0 0.0
bridge-app debug unknown 5288 5288 0 0.0
FLASH 4732472 4732472 0 0.0
RAM 217368 217368 0 0.0
chip-tool debug unknown 5832 5832 0 0.0
FLASH 12651036 12651036 0 0.0
RAM 558354 558354 0 0.0
chip-tool-ipv6only arm64 unknown 20352 20352 0 0.0
FLASH 11322892 11322892 0 0.0
RAM 608424 608424 0 0.0
fabric-admin debug unknown 5664 5664 0 0.0
FLASH 10957287 10957287 0 0.0
RAM 557354 557354 0 0.0
fabric-bridge-app debug unknown 4520 4520 0 0.0
FLASH 4555270 4555270 0 0.0
RAM 204016 204016 0 0.0
lighting-app debug+rpc+ui unknown 5936 5936 0 0.0
FLASH 5684369 5684369 0 0.0
RAM 228184 228184 0 0.0
lock-app debug unknown 5224 5224 0 0.0
FLASH 4787074 4787074 0 0.0
RAM 203768 203768 0 0.0
ota-provider-app debug unknown 4600 4600 0 0.0
FLASH 4426612 4426612 0 0.0
RAM 197752 197752 0 0.0
ota-requestor-app debug unknown 4536 4536 0 0.0
FLASH 4565198 4565198 0 0.0
RAM 202288 202288 0 0.0
shell debug unknown 4176 4176 0 0.0
FLASH 3105277 3105277 0 0.0
RAM 159168 159168 0 0.0
thermostat-no-ble arm64 unknown 9296 9296 0 0.0
FLASH 4325900 4325900 0 0.0
RAM 241896 241896 0 0.0
tv-app debug unknown 5504 5504 0 0.0
FLASH 6011813 6011813 0 0.0
RAM 583192 583192 0 0.0
tv-casting-app debug unknown 5168 5168 0 0.0
FLASH 10854957 10854957 0 0.0
RAM 645768 645768 0 0.0
nrfconnect all-clusters-app nrf52840dk_nrf52840 FLASH 915384 915384 0 0.0
RAM 142243 142243 0 0.0
nrf7002dk_nrf5340_cpuapp FLASH 886616 886616 0 0.0
RAM 140382 140382 0 0.0
all-clusters-minimal-app nrf52840dk_nrf52840 FLASH 846812 846812 0 0.0
RAM 141062 141062 0 0.0
nxp contact k32w0+release FLASH 577796 577796 0 0.0
RAM 70444 70444 0 0.0
k32w1+release FLASH 593248 593248 0 0.0
RAM 62712 62712 0 0.0
light k32w0+release FLASH 613416 613416 0 0.0
RAM 69908 69908 0 0.0
k32w1+release FLASH 676872 676872 0 0.0
RAM 47848 47848 0 0.0
psoc6 all-clusters cy8ckit_062s2_43012 FLASH 1647932 1647932 0 0.0
RAM 210944 210944 0 0.0
all-clusters-minimal cy8ckit_062s2_43012 FLASH 1551604 1551604 0 0.0
RAM 207656 207656 0 0.0
light cy8ckit_062s2_43012 FLASH 1470396 1470396 0 0.0
RAM 200752 200752 0 0.0
lock cy8ckit_062s2_43012 FLASH 1463636 1463636 0 0.0
RAM 225064 225064 0 0.0
qpg lighting-app qpg6105+debug FLASH 656812 656812 0 0.0
RAM 105152 105152 0 0.0
lock-app qpg6105+debug FLASH 614208 614208 0 0.0
RAM 99612 99612 0 0.0
stm32 light STM32WB5MM-DK FLASH 483216 483216 0 0.0
RAM 144740 144740 0 0.0
telink air-quality-sensor-app tlsr9528a_retention FLASH 620504 620504 0 0.0
RAM 50932 50932 0 0.0
all-clusters-app tlsr9118bdk40d FLASH 687434 687434 0 0.0
RAM 149576 149576 0 0.0
all-clusters-minimal-app tlsr9528a FLASH 779874 779874 0 0.0
RAM 110736 110736 0 0.0
bridge-app tlsr9258a FLASH 679496 679496 0 0.0
RAM 91588 91588 0 0.0
contact-sensor-app tlsr9528a_retention FLASH 622096 622096 0 0.0
RAM 50972 50972 0 0.0
light-switch-app-ota-shell-factory-data tlsr9528a FLASH 708106 708106 0 0.0
RAM 74312 74312 0 0.0
lighting-app-ota-factory-data tlsr9118bdk40d FLASH 625734 625734 0 0.0
RAM 145556 145556 0 0.0
lighting-app-ota-rpc-factory-data-4mb tlsr9518adk80d FLASH 811866 811866 0 0.0
RAM 99480 99480 0 0.0
lock-app-dfu tlsr9528a FLASH 655078 655078 0 0.0
RAM 67008 67008 0 0.0
ota-requestor-app tlsr9258a FLASH 697362 697362 0 0.0
RAM 91272 91272 0 0.0
pump-app-usb tlsr9518adk80d FLASH 631804 631804 0 0.0
RAM 55752 55752 0 0.0
pump-controller-app tlsr9518adk80d FLASH 609052 609052 0 0.0
RAM 52996 52996 0 0.0
shell tlsr9518adk80d FLASH 466890 466890 0 0.0
RAM 68660 68660 0 0.0
smoke_co_alarm-app tlsr9528a_retention FLASH 628882 628882 0 0.0
RAM 52692 52692 0 0.0
temperature-measurement-app-mars-ota tlsr9518adk80d FLASH 653034 653034 0 0.0
RAM 56632 56632 0 0.0
thermostat tlsr9518adk80d FLASH 635784 635784 0 0.0
RAM 53400 53400 0 0.0
window-covering tlsr9118bdk40d FLASH 523038 523038 0 0.0
RAM 98532 98532 0 0.0
tizen all-clusters-app arm unknown 4840 4840 0 0.0
FLASH 1724876 1724876 0 0.0
RAM 89500 89500 0 0.0
chip-tool-ubsan arm unknown 10272 10272 0 0.0
FLASH 17511238 17511238 0 0.0
RAM 7601356 7601356 0 0.0

@mergify mergify bot merged commit 7bc9639 into project-chip:master Sep 4, 2024
67 checks passed
@jamesharrow jamesharrow deleted the 34100-tc-eevse-2224-25-python-script-has-to-be-updated-as-per-the-test-plan-changes branch September 4, 2024 13:45
shgutte pushed a commit to shgutte/connectedhomeip that referenced this pull request Sep 10, 2024
…est plan changes (project-chip#35399)

* Updated 2.2 test steps - autogen'd from test plan. No functional changes.

* Update TC EEVSE 2.3 with autogen'd test steps. No functional code change.

* Updated TC_EEVSE_2.4 test steps - no functional code changes.

* TC_EEVSE_2.5 - updated test steps - no functional changes.

* TC_EEVSE_2.5 added new step 8 (try to send Enable Charge Command when in diagnostics mode which should return failure).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
matter-1.4-te2-script-change Script changes before end of Matter 1.4 TE2 review - approved tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[TC-EEVSE-2.2,2.4 & 2.5] Python script has to be updated as per the test plan changes
3 participants