[YAML][CI] Update the test runner to implement type checking #21354
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pullapprove
bot
requested review from
anush-apple,
arkq,
Byungjoo-Lee,
bzbarsky-apple,
carol-apple,
chrisdecenzo,
chshu,
chulspro,
Damian-Nordic,
dhrishi,
electrocucaracha,
franck-apple,
gjc13,
harimau-qirex,
harsha-rajendran,
hawk248,
isiu-apple,
jelderton,
jepenven-silabs,
jmartinez-silabs,
jtung-apple,
kghost,
kpschoedel,
lazarkov,
LuDuda,
mlepage-google,
mrjerryjohns,
msandstedt and
rgoliver
…uster.xml to use vendor_id type for the vendorID attribute per spec
…u for the vendorID attribute
…ist when reading the current channel attribute
…edThreshold is int8u and not int16u
…ce is int16u and not int16s
… instead of list...
…ml to use utc instead of epoch_s for SetpointChangeSourceTimestamp per spec
vivien-apple
force-pushed
the
YAML_CheckTypes
branch
from
cdfd931 to
99234a1
Compare
|
PR #21354: Size comparison from 375e841 to 99234a1 Increases (3 builds for cc13x2_26x2, mbed)
Decreases (6 builds for bl602, cc13x2_26x2, cyw30739, linux)
Full report (32 builds for bl602, cc13x2_26x2, cyw30739, efr32, esp32, k32w, linux, mbed, nrfconnect, p6, telink)
|
github-actions bot
pushed a commit
that referenced
this pull request
Jul 29, 2022
* [YAML][CI] Update the test runner to implement type checking * Update src/app/tests/suites/ with boolean instead of bool types * Update src/app/tests/suites/ with int*u instead of uint* types * Update src/app/tests/suites/ with int*s instead of int* types * Update src/app/tests/suites/ with bitmap* instead of map* types * Update src/app/tests/suites/ with char_string instead of string types * Update src/app/tests/suites/ with octet_string instead of octstr types * Update src/app/zap-templates/zcl/data-model/chip/operational-credentials-cluster.xml to use uint8 for CurrentFabricIndex instead of fabric_idx * NODE_ID to node_id * ENUM8 -> enum8 * Update Test_TC_DGWIFI_2_1.yaml to check for enum8 for SecurityType instead of enum * Use int8s in Test_TC_TSTAT_2_2 instead of temp-s8 which is a type defined in the Thermostat spec but that does not match with anything specific in the SDK * Convert OutPutInfo to list since OutputInfo is the inner type * Check for enum8 instead of RoutingRole in Test_TC_DGTHREAD_2_1 * Use ApplicationBasicApplication instead of struct in Test_TC_APBSC_9_1 * vendor-id to vendor_id * Update src/app/zap-templates/zcl/data-model/chip/application-basic-cluster.xml to use vendor_id type for the vendorID attribute per spec * Update Test_TC_BINFO_2_1 to check for type vendor_id instead of int16u for the vendorID attribute * Update Test_TC_CHANNEL_5_3 to check for type ChannelInfo instead of list when reading the current channel attribute * Update generated code
woody-apple
added a commit
that referenced
this pull request
Jul 29, 2022
…#21404) * [YAML][CI] Update the test runner to implement type checking * Update src/app/tests/suites/ with boolean instead of bool types * Update src/app/tests/suites/ with int*u instead of uint* types * Update src/app/tests/suites/ with int*s instead of int* types * Update src/app/tests/suites/ with bitmap* instead of map* types * Update src/app/tests/suites/ with char_string instead of string types * Update src/app/tests/suites/ with octet_string instead of octstr types * Update src/app/zap-templates/zcl/data-model/chip/operational-credentials-cluster.xml to use uint8 for CurrentFabricIndex instead of fabric_idx * NODE_ID to node_id * ENUM8 -> enum8 * Update Test_TC_DGWIFI_2_1.yaml to check for enum8 for SecurityType instead of enum * Use int8s in Test_TC_TSTAT_2_2 instead of temp-s8 which is a type defined in the Thermostat spec but that does not match with anything specific in the SDK * Convert OutPutInfo to list since OutputInfo is the inner type * Check for enum8 instead of RoutingRole in Test_TC_DGTHREAD_2_1 * Use ApplicationBasicApplication instead of struct in Test_TC_APBSC_9_1 * vendor-id to vendor_id * Update src/app/zap-templates/zcl/data-model/chip/application-basic-cluster.xml to use vendor_id type for the vendorID attribute per spec * Update Test_TC_BINFO_2_1 to check for type vendor_id instead of int16u for the vendorID attribute * Update Test_TC_CHANNEL_5_3 to check for type ChannelInfo instead of list when reading the current channel attribute * Update generated code Co-authored-by: Vivien Nicolas <[email protected]>
isiu-apple
pushed a commit
to isiu-apple/connectedhomeip
that referenced
this pull request
Sep 16, 2022
…-chip#21354) * [YAML][CI] Update the test runner to implement type checking * Update src/app/tests/suites/ with boolean instead of bool types * Update src/app/tests/suites/ with int*u instead of uint* types * Update src/app/tests/suites/ with int*s instead of int* types * Update src/app/tests/suites/ with bitmap* instead of map* types * Update src/app/tests/suites/ with char_string instead of string types * Update src/app/tests/suites/ with octet_string instead of octstr types * Update src/app/zap-templates/zcl/data-model/chip/operational-credentials-cluster.xml to use uint8 for CurrentFabricIndex instead of fabric_idx * NODE_ID to node_id * ENUM8 -> enum8 * Update Test_TC_DGWIFI_2_1.yaml to check for enum8 for SecurityType instead of enum * Use int8s in Test_TC_TSTAT_2_2 instead of temp-s8 which is a type defined in the Thermostat spec but that does not match with anything specific in the SDK * Convert OutPutInfo to list since OutputInfo is the inner type * Check for enum8 instead of RoutingRole in Test_TC_DGTHREAD_2_1 * Use ApplicationBasicApplication instead of struct in Test_TC_APBSC_9_1 * vendor-id to vendor_id * Update src/app/zap-templates/zcl/data-model/chip/application-basic-cluster.xml to use vendor_id type for the vendorID attribute per spec * Update Test_TC_BINFO_2_1 to check for type vendor_id instead of int16u for the vendorID attribute * Update Test_TC_CHANNEL_5_3 to check for type ChannelInfo instead of list when reading the current channel attribute * Update generated code
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
This PR add basic type checking to YAML tests.
In summary this PR just checks that the type that is used as a constraint match with the type that is declared in the XML.
The actual value checking itself is done when
chip::app::DataModel::Decode(*data, value)is called by the test framework, which fails if the payload can not be decoded/converted to the types eclared in the XML.As a followup to this PR it would be nice to actually use the type name from the spec instead of the current type name from
chip-types.xml. For example the spec saysboolwherechip-types.xmlsaysboolean, or the spec saysmap32wherechip-types.xmlsaysbitmap32, etc...Change overview
src/app/tests/suitestests constraints to make sure thetypeis correctsrc/app/zap-templates/zcl/data-model/chip/operational-credentials-cluster.xmlto useuint8forCurrentFabricIndexinstead offabric_idxsince this is what is in the spec (I'm not sure the spec is right here...)src/app/zap-templates/zcl/data-model/chip/application-basic-cluster.xmlto usevendor_idtype for thevendorIDattribute per specTesting
A lot of generated tests are now failing instead of issuing a warning if the type is incorrect