Auto-select data model set based on specification version #37394
+245
−22
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Review changes with Changed Files
|
|
PR #37394: Size comparison from 6292e6e to aecb846 Full report (72 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds auto-selection of the data model files into the data-model-based tests. Because of the limited set of changes between 1.4.1 and 1.4, this means the 1.4.1 test harness should be able to be used to certify 1.4 devices. Further work is required to expand this to minor versions.
Notes are the work to expand this to minor version and also to handle testing of multiple specification revisions in the CI is being tracked here: project-chip/matter-test-scripts#448
Testing
Added unit-style tests for data model selection in TestSpecParsingSelection.py (added to CI).
Because the specification version is hard coded in the SDK, we can only realistically test against whatever the current spec version is in the CI. Instead, I manually generated versions of the door lock app with the different versions (1.4 or 1.4.1) and Terms and Conditions either set or not set, then ran TC_AccessChecker.py, TC_DeviceConformance.py, TC_DeviceBasicComposition.py against them. Results as follows:
1.4, no TC - all pass except IDM-10.5 (ota requestor cluster - known issue)
1.4 with TC - DeviceConformance fails with errors about provisional TC things
1.4.1 no TC - all pass except IDM-10.5 (ota requestor cluster - known issue)
1.4.1 with TC - all pass except IDM-10.5 (ota requestor cluster - known issue)
^ These pass and fail as expected