[TC-DL-1.1] FeatureMap and Attribute/Command Lists mismatch

[yinyihu-silabs]

Problem

FeatureMap from chip-tool does not match the outputs from AttributeList and AcceptedCommandList

Given a FeatureMap below, the follow features are enabled: PIN, RID, SCH, NOT

./chip-tool doorlock read  feature-map 5135 1
[1651516926.982768][9765:9770] CHIP:TOO: Endpoint: 1 Cluster: 0x0000_0101 Attribute 0x0000_FFFC DataVersion: 3777711483
[1651516926.982883][9765:9770] CHIP:TOO:   FeatureMap: 275

However, AttributeList and AcceptedCommandList return:

./chip-tool doorlock read attribute-list 5135 1
[1651517112.153220][9777:9782] CHIP:TOO: Endpoint: 1 Cluster: 0x0000_0101 Attribute 0x0000_FFFB DataVersion: 3777711483
[1651517112.153338][9777:9782] CHIP:TOO:   AttributeList: 29 entries
[1651517112.153368][9777:9782] CHIP:TOO:     [1]: 0
[1651517112.153392][9777:9782] CHIP:TOO:     [2]: 1
[1651517112.153414][9777:9782] CHIP:TOO:     [3]: 2
[1651517112.153436][9777:9782] CHIP:TOO:     [4]: 3 -> DPS
[1651517112.153458][9777:9782] CHIP:TOO:     [5]: 17 -> USR
[1651517112.153479][9777:9782] CHIP:TOO:     [6]: 18
[1651517112.153501][9777:9782] CHIP:TOO:     [7]: 19
[1651517112.153523][9777:9782] CHIP:TOO:     [8]: 20
[1651517112.153545][9777:9782] CHIP:TOO:     [9]: 21
							<- 22 SCH missing
[1651517112.153566][9777:9782] CHIP:TOO:     [10]: 23
[1651517112.153588][9777:9782] CHIP:TOO:     [11]: 24
[1651517112.153610][9777:9782] CHIP:TOO:     [12]: 25
[1651517112.153632][9777:9782] CHIP:TOO:     [13]: 26
[1651517112.153651][9777:9782] CHIP:TOO:     [14]: 27 -> USR
[1651517112.153673][9777:9782] CHIP:TOO:     [15]: 33
[1651517112.153717][9777:9782] CHIP:TOO:     [16]: 35
[1651517112.153743][9777:9782] CHIP:TOO:     [17]: 36
[1651517112.153765][9777:9782] CHIP:TOO:     [18]: 37
[1651517112.153787][9777:9782] CHIP:TOO:     [19]: 38
[1651517112.153808][9777:9782] CHIP:TOO:     [20]: 41
[1651517112.153830][9777:9782] CHIP:TOO:     [21]: 43
[1651517112.153895][9777:9782] CHIP:TOO:     [22]: 48
[1651517112.153920][9777:9782] CHIP:TOO:     [23]: 49
[1651517112.153946][9777:9782] CHIP:TOO:     [24]: 51
[1651517112.153968][9777:9782] CHIP:TOO:     [25]: 65528
[1651517112.153990][9777:9782] CHIP:TOO:     [26]: 65529
[1651517112.154012][9777:9782] CHIP:TOO:     [27]: 65531
[1651517112.154034][9777:9782] CHIP:TOO:     [28]: 65532
[1651517112.154055][9777:9782] CHIP:TOO:     [29]: 65533

./chip-tool doorlock read accepted-command-list 5135 1
[1651517138.466536][9784:9789] CHIP:TOO: Endpoint: 1 Cluster: 0x0000_0101 Attribute 0x0000_FFF9 DataVersion: 3777711483
[1651517138.466612][9784:9789] CHIP:TOO:   AcceptedCommandList: 15 entries
[1651517138.466642][9784:9789] CHIP:TOO:     [1]: 0
[1651517138.466664][9784:9789] CHIP:TOO:     [2]: 1
[1651517138.466685][9784:9789] CHIP:TOO:     [3]: 3
							<- 5-10 missing
[1651517138.466707][9784:9789] CHIP:TOO:     [4]: 11
[1651517138.466729][9784:9789] CHIP:TOO:     [5]: 12
[1651517138.466750][9784:9789] CHIP:TOO:     [6]: 13
[1651517138.466771][9784:9789] CHIP:TOO:     [7]: 14
[1651517138.466792][9784:9789] CHIP:TOO:     [8]: 15
[1651517138.466813][9784:9789] CHIP:TOO:     [9]: 16
							 <- 17-25 missing
[1651517138.466835][9784:9789] CHIP:TOO:     [10]: 26 -> USR
[1651517138.466856][9784:9789] CHIP:TOO:     [11]: 27 -> USR
[1651517138.466877][9784:9789] CHIP:TOO:     [12]: 29 
[1651517138.466898][9784:9789] CHIP:TOO:     [13]: 34 ->USR
[1651517138.466919][9784:9789] CHIP:TOO:     [14]: 36 ->USR
[1651517138.466940][9784:9789] CHIP:TOO:     [15]: 38 ->USR

As results, many DoorLock test are not testable when required attributes/commands are not supported.

[bzbarsky-apple]

@yinyihu-silabs Is this using all-clusters-app or lock-app as the server?

[yinyihu-silabs]

lock-app

[woody-apple]

Cert Blocker Review: @franck-apple to find an assignee (maybe @Morozov-5F )

[Morozov-5F]

I can just enable those commands and attributes in the lock-app but they won't be actually implemented. SDK part of the door lock currently supports only a limited subset of all the commands.

[bzbarsky-apple]

I think the important part is that anything claimed to be supported by the feature map needs to actually be supported.

[andy31415]

I believe we should enable all features and implement at least stubs for any features we expect V1 production devices to want certifiable.

Either path of "remove feature flags" or "implement stubs" works, but they affect what certification will certify.

[woody-apple]

Cert Blocker Review: Tagging @cjandhyala to set up an evaluation of this test plan as well.

[sumaky]

@cjandhyala @woody-apple Initially DL had all the requirements as in AppCluster test plan . But as part of PR reviews we had request from reviewers to remove non matter specific requirements. Now in test plan we deleted non matter specific requirements and got the PR approval
https://github.com/CHIP-Specifications/connectedhomeip-spec/blob/master/src/device_types/DoorLock.adoc
Test plan may have to have checks based on feature maps. we will review and provide an update