Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[YAML][CI] Update the test runner to implement type checking #21404

Merged

Conversation

woody-apple
Copy link
Contributor

Problem

This PR add basic type checking to YAML tests.
In summary this PR just checks that the type that is used as a constraint match with the type that is declared in the XML.
The actual value checking itself is done when chip::app::DataModel::Decode(*data, value) is called by the test framework, which fails if the payload can not be decoded/converted to the types eclared in the XML.

As a followup to this PR it would be nice to actually use the type name from the spec instead of the current type name from chip-types.xml. For example the spec says bool where chip-types.xml says boolean, or the spec says map32where chip-types.xml says bitmap32, etc...

Change overview

  • Update src/app/tests/suites tests constraints to make sure the type is correct
  • Update src/app/zap-templates/zcl/data-model/chip/operational-credentials-cluster.xml to use uint8 for CurrentFabricIndex instead of fabric_idx since this is what is in the spec (I'm not sure the spec is right here...)
  • Update src/app/zap-templates/zcl/data-model/chip/application-basic-cluster.xml to use vendor_id type for the vendorID attribute per spec
  • Update the test runner to take the type constraint into accou

Testing

A lot of generated tests are now failing instead of issuing a warning if the type is incorrect

* [YAML][CI] Update the test runner to implement type checking

* Update src/app/tests/suites/ with boolean instead of bool types

* Update src/app/tests/suites/ with int*u instead of uint* types

* Update src/app/tests/suites/ with int*s instead of int* types

* Update src/app/tests/suites/ with bitmap* instead of map* types

* Update src/app/tests/suites/ with char_string instead of string types

* Update src/app/tests/suites/ with octet_string instead of octstr types

* Update src/app/zap-templates/zcl/data-model/chip/operational-credentials-cluster.xml to use uint8 for CurrentFabricIndex instead of fabric_idx

* NODE_ID to node_id

* ENUM8 -> enum8

* Update Test_TC_DGWIFI_2_1.yaml to check for enum8 for SecurityType instead of enum

* Use int8s in Test_TC_TSTAT_2_2 instead of temp-s8 which is a type defined in the Thermostat spec but that does not match with anything specific in the SDK

* Convert OutPutInfo to list since OutputInfo is the inner type

* Check for enum8 instead of RoutingRole in Test_TC_DGTHREAD_2_1

* Use ApplicationBasicApplication instead of struct in Test_TC_APBSC_9_1

* vendor-id to vendor_id

* Update src/app/zap-templates/zcl/data-model/chip/application-basic-cluster.xml to use vendor_id type for the vendorID attribute per spec

* Update Test_TC_BINFO_2_1 to check for type vendor_id instead of int16u for the vendorID attribute

* Update Test_TC_CHANNEL_5_3 to check for type ChannelInfo instead of list when reading the current channel attribute

* Update generated code
@github-actions
Copy link

github-actions bot commented Jul 29, 2022

PR #21404: Size comparison from 079a65d to b6483f5

Increases (2 builds for cc13x2_26x2, telink)
platform target config section 079a65d b6483f5 change % change
cc13x2_26x2 lock-ftd LP_CC2652R7 (read only) 671175 671183 8 0.0
.text 594432 594440 8 0.0
telink light-switch-app tlsr9518adk80d text 567156 567158 2 0.0
Decreases (3 builds for cc13x2_26x2, linux, telink)
platform target config section 079a65d b6483f5 change % change
cc13x2_26x2 lock-ftd LP_CC2652R7 (read/write) 170360 170352 -8 -0.0
linux chip-tool-ipv6only arm64 (read only) 9845204 9841892 -3312 -0.0
.rodata 467572 467492 -80 -0.0
.text 7795140 7791908 -3232 -0.0
telink lighting-app tlsr9518adk80d text 584022 584020 -2 -0.0
Full report (32 builds for bl602, cc13x2_26x2, cyw30739, efr32, esp32, k32w, linux, mbed, nrfconnect, p6, telink)
platform target config section 079a65d b6483f5 change % change
bl602 lighting-app bl602 (read/write) 1381290 1381290 0 0.0
.bss 117618 117618 0 0.0
.data 4480 4480 0 0.0
.text 1051564 1051564 0 0.0
bl602+rpc (read/write) 1426738 1426738 0 0.0
.bss 125058 125058 0 0.0
.data 4600 4600 0 0.0
.text 1083256 1083256 0 0.0
cc13x2_26x2 all-clusters-app LP_CC2652R7 (read only) 668307 668307 0 0.0
(read/write) 183036 183036 0 0.0
.bss 74236 74236 0 0.0
.data 3372 3372 0 0.0
.rodata 88195 88195 0 0.0
.text 579796 579796 0 0.0
all-clusters-minimal-app LP_CC2652R7 (read only) 633915 633915 0 0.0
(read/write) 157820 157820 0 0.0
.bss 73532 73532 0 0.0
.data 3372 3372 0 0.0
.rodata 77411 77411 0 0.0
.text 556180 556180 0 0.0
lock-ftd LP_CC2652R7 (read only) 671175 671183 8 0.0
(read/write) 170360 170352 -8 -0.0
.bss 71316 71316 0 0.0
.data 3296 3296 0 0.0
.rodata 76263 76263 0 0.0
.text 594432 594440 8 0.0
lock-mtd LP_CC2652R7 (read only) 653291 653291 0 0.0
(read/write) 183932 183932 0 0.0
.bss 67004 67004 0 0.0
.data 3296 3296 0 0.0
.rodata 100875 100875 0 0.0
.text 551936 551936 0 0.0
pump-app LP_CC2652R7 (read only) 680631 680631 0 0.0
(read/write) 161736 161736 0 0.0
.bss 71380 71380 0 0.0
.data 3296 3296 0 0.0
.rodata 88863 88863 0 0.0
.text 591284 591284 0 0.0
pump-controller-app LP_CC2652R7 (read only) 666199 666199 0 0.0
(read/write) 176304 176304 0 0.0
.bss 71516 71516 0 0.0
.data 3292 3292 0 0.0
.rodata 84679 84679 0 0.0
.text 581040 581040 0 0.0
shell LP_CC2652R7 (read only) 661006 661006 0 0.0
(read/write) 185856 185856 0 0.0
.bss 76556 76556 0 0.0
.data 3376 3376 0 0.0
.rodata 85166 85166 0 0.0
.text 575524 575524 0 0.0
cyw30739 light cyw930739m2evb_01 (read/write) 583086 583086 0 0.0
.app_xip_area 460048 460048 0 0.0
.bss 65624 65624 0 0.0
.data 744 744 0 0.0
.rodata 0 0 0 0.0
.text 112 112 0 0.0
lock cyw930739m2evb_01 (read/write) 588822 588822 0 0.0
.app_xip_area 461056 461056 0 0.0
.bss 70352 70352 0 0.0
.data 748 748 0 0.0
.rodata 0 0 0 0.0
.text 112 112 0 0.0
ota-requestor-no-progress-logging cyw930739m2evb_01 (read/write) 588642 588642 0 0.0
.app_xip_area 466420 466420 0 0.0
.bss 64864 64864 0 0.0
.data 688 688 0 0.0
.rodata 0 0 0 0.0
.text 112 112 0 0.0
efr32 lighting-app BRD4161A (read/write) 1087920 1087920 0 0.0
.bss 133260 133260 0 0.0
.data 2064 2064 0 0.0
.text 952576 952576 0 0.0
BRD4161A+rpc (read/write) 1142252 1142252 0 0.0
.bss 149940 149940 0 0.0
.data 2276 2276 0 0.0
.text 990016 990016 0 0.0
BRD4161A+rs911x (read/write) 973192 973192 0 0.0
.bss 161736 161736 0 0.0
.data 2048 2048 0 0.0
.text 809388 809388 0 0.0
lock-app BRD4161A+wf200 (read/write) 1128016 1128016 0 0.0
.bss 144368 144368 0 0.0
.data 2056 2056 0 0.0
.text 981572 981572 0 0.0
window-app BRD4161A (read/write) 1081204 1081204 0 0.0
.bss 134732 134732 0 0.0
.data 2092 2092 0 0.0
.text 944356 944356 0 0.0
esp32 all-clusters-app c3devkit (read only) 1022872 1022872 0 0.0
(read/write) 1486274 1486274 0 0.0
.dram0.bss 70296 70296 0 0.0
.dram0.data 14600 14600 0 0.0
.flash.rodata 215936 215936 0 0.0
.flash.text 1022872 1022872 0 0.0
.iram0.text 62902 62902 0 0.0
m5stack (read only) 1076327 1076327 0 0.0
(read/write) 488312 488312 0 0.0
.dram0.bss 75808 75808 0 0.0
.dram0.data 34144 34144 0 0.0
.flash.rodata 246364 246364 0 0.0
.flash.text 1070943 1070943 0 0.0
.iram0.text 123267 123267 0 0.0
k32w light k32w0+release (read/write) 642688 642688 0 0.0
.bss 69712 69712 0 0.0
.data 2044 2044 0 0.0
.text 568204 568204 0 0.0
lock k32w0+release (read/write) 699584 699584 0 0.0
.bss 70152 70152 0 0.0
.data 2052 2052 0 0.0
.text 624652 624652 0 0.0
linux chip-tool-ipv6only arm64 (read only) 9845204 9841892 -3312 -0.0
(read/write) 678753 678753 0 0.0
.bss 32897 32897 0 0.0
.data 3272 3272 0 0.0
.data.rel.ro 624080 624080 0 0.0
.dynamic 560 560 0 0.0
.got 13552 13552 0 0.0
.init 24 24 0 0.0
.init_array 192 192 0 0.0
.rodata 467572 467492 -80 -0.0
.text 7795140 7791908 -3232 -0.0
thermostat-no-ble arm64 (read only) 2342812 2342812 0 0.0
(read/write) 141633 141633 0 0.0
.bss 55313 55313 0 0.0
.data 1672 1672 0 0.0
.data.rel.ro 75880 75880 0 0.0
.dynamic 560 560 0 0.0
.got 4992 4992 0 0.0
.init 24 24 0 0.0
.init_array 408 408 0 0.0
.rodata 138964 138964 0 0.0
.text 1966608 1966608 0 0.0
mbed lock-app CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 2450536 2450536 0 0.0
.bss 214516 214516 0 0.0
.data 5872 5872 0 0.0
.text 1413180 1413180 0 0.0
nrfconnect all-clusters-app nrf52840dk_nrf52840 (read/write) 1177343 1177343 0 0.0
bss 143120 143120 0 0.0
rodata 142552 142552 0 0.0
text 812712 812712 0 0.0
all-clusters-minimal-app nrf52840dk_nrf52840 (read/write) 1157399 1157399 0 0.0
bss 142356 142356 0 0.0
rodata 134088 134088 0 0.0
text 802028 802028 0 0.0
p6 all-clusters-app default (read only) 881560 881560 0 0.0
(read/write) 1687036 1687036 0 0.0
.bss 149136 149136 0 0.0
.data 2648 2648 0 0.0
.text 1526864 1526864 0 0.0
all-clusters-minimal-app default (read only) 882280 882280 0 0.0
(read/write) 1631164 1631164 0 0.0
.bss 148416 148416 0 0.0
.data 2648 2648 0 0.0
.text 1471712 1471712 0 0.0
light-app default (read only) 890584 890584 0 0.0
(read/write) 1552348 1552348 0 0.0
.bss 140320 140320 0 0.0
.data 2440 2440 0 0.0
.text 1401200 1401200 0 0.0
lock-app default (read only) 886112 886112 0 0.0
(read/write) 1589764 1589764 0 0.0
.bss 144776 144776 0 0.0
.data 2456 2456 0 0.0
.text 1434144 1434144 0 0.0
telink light-switch-app tlsr9518adk80d (read/write) 799520 799520 0 0.0
bss 70796 70796 0 0.0
noinit 40416 40416 0 0.0
text 567156 567158 2 0.0
lighting-app tlsr9518adk80d (read/write) 819936 819936 0 0.0
bss 71640 71640 0 0.0
noinit 40416 40416 0 0.0
text 584022 584020 -2 -0.0

@woody-apple woody-apple merged commit a4580e2 into sve Jul 29, 2022
@woody-apple woody-apple deleted the cherry-pick-f93e6cb403ff655b142391e96fb6a1ab3d484413 branch July 29, 2022 18:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants