Skip to content

Commit

Permalink
Fix TC_CADMIN_1_9 and TC_CADMIN_1_10 tests to not be broken. (#21690)
Browse files Browse the repository at this point in the history 
There were several issues:

* The values of the "payload" variable were mixed up between the two tests.  as
  a result, "payload" actually had an incorrect setup code in both tests, and
  the tests testing that commissioning fails actually tested nothing at all,
  because it was guaranteed to fail.

* Some of the setup payloads involved claimed a zero discovery capability
  bitmask, instead of the expected "bit 2 is set to indicate on-network
  commissioning".  This led to a lot of BLE traffic when these tests run.

The specific fixes here are:

1) In TC_CADMIN_1_9, regenerate the "payload" value by running:

      chip-tool payload generate-qrcode --vendor-id 65521 --product-id 32769 --discriminator 3840 --rendezvous 4 --setup-pin-code 47514138

   and regenerate "payload2" by running:

      chip-tool payload generate-qrcode --vendor-id 65521 --product-id 32769 --discriminator 3840 --rendezvous 4 --setup-pin-code 20202023

   This ensures that the only difference between "payload" and "payload2" is the
   setup code, and sets the setup code for "payload" to match the verifier we
   are using.

2) In TC_CADMIN_1_10, regenerate the "payload" value by running:

       chip-tool payload generate-qrcode --vendor-id 65521 --product-id 32769 --discriminator 3840 --rendezvous 4 --setup-pin-code 20202021

   and regenerate "payload2" by running:

      chip-tool payload generate-qrcode --vendor-id 65521 --product-id 32769 --discriminator 3840 --rendezvous 4 --setup-pin-code 20202023

3) In both tests, add verification that the "payload" value can in fact
   commission the device properly when a commissioning window is opened the way
   the test opens it, so the rest of the test is meaningful.

4) Rename "payload" and "payload2" to more meaningful names.
  • Loading branch information
@bzbarsky-apple @pull
bzbarsky-apple authored and pull[bot] committed Sep 15, 2023
1 parent 0a34141 commit 1075640
Show file tree
Hide file tree
Showing 3 changed files with 503 additions and 173 deletions.
108 changes: 83 additions & 25 deletions src/app/tests/suites/certification/Test_TC_CADMIN_1_10.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -35,12 +35,12 @@ config:
discriminator:
type: int16u
defaultValue: 3840
payload:
correctPayload:
type: char_string
defaultValue: "MT:0000000000I31506010"
payload2:
defaultValue: "MT:-24J0AFN00KA0648G00"
incorrectSetupCodePayload:
type: char_string
defaultValue: "MT:0000000000I.0648G00"
defaultValue: "MT:-24J0AFN00I.0648G00"

tests:
- label: "TH_CR1 starts a commissioning process with DUT_CE"
Expand All @@ -52,6 +52,63 @@ tests:
- name: "nodeId"
value: nodeId

- label:
"Open commissioning window exactly like we plan to later to verify
that `correctPayload` can in fact commision the device."
cluster: "AdministratorCommissioning"
command: "OpenBasicCommissioningWindow"
PICS: CADMIN.S.C01.Rsp
timedInteractionTimeoutMs: 10000
arguments:
values:
- name: "CommissioningTimeout"
value: 900

- label:
"Temporarily commission device with `correctPayload` just like we will
try to later"
identity: "beta"
cluster: "CommissionerCommands"
command: "PairWithCode"
# Only run this if we ran the previous step.
PICS: CADMIN.S && CADMIN.S.C01.Rsp
arguments:
values:
- name: "nodeId"
value: nodeId2
- name: "payload"
value: correctPayload

- label: "Wait for a CASE session"
identity: "beta"
cluster: "DelayCommands"
command: "WaitForCommissionee"
PICS: CADMIN.S
arguments:
values:
- name: "nodeId"
value: nodeId2

- label: "Read FabricIndex of beta so we can delete the temp fabric"
identity: "beta"
cluster: "Operational Credentials"
command: "readAttribute"
attribute: "CurrentFabricIndex"
# Only run this if we commissioned the device.
PICS: CADMIN.S.C01.Rsp
response:
saveAs: tempFabricIndex

- label: "Remove the temp fabric"
cluster: "Operational Credentials"
command: "RemoveFabric"
# Only run this if we commissioned the device.
PICS: CADMIN.S.C01.Rsp
arguments:
values:
- name: "FabricIndex"
value: tempFabricIndex

- label: "TH_CR1 opens a commissioning window on DUT_CE"
cluster: "AdministratorCommissioning"
command: "OpenBasicCommissioningWindow"
Expand All @@ -74,7 +131,7 @@ tests:
- name: "nodeId"
value: nodeId2
- name: "payload"
value: payload2
value: incorrectSetupCodePayload
response:
error: FAILURE

Expand All @@ -90,7 +147,7 @@ tests:
- name: "nodeId"
value: nodeId2
- name: "payload"
value: payload2
value: incorrectSetupCodePayload
response:
error: FAILURE

Expand All @@ -106,7 +163,7 @@ tests:
- name: "nodeId"
value: nodeId2
- name: "payload"
value: payload2
value: incorrectSetupCodePayload
response:
error: FAILURE

Expand All @@ -122,7 +179,7 @@ tests:
- name: "nodeId"
value: nodeId2
- name: "payload"
value: payload2
value: incorrectSetupCodePayload
response:
error: FAILURE

Expand All @@ -138,7 +195,7 @@ tests:
- name: "nodeId"
value: nodeId2
- name: "payload"
value: payload2
value: incorrectSetupCodePayload
response:
error: FAILURE

Expand All @@ -154,7 +211,7 @@ tests:
- name: "nodeId"
value: nodeId2
- name: "payload"
value: payload2
value: incorrectSetupCodePayload
response:
error: FAILURE

Expand All @@ -170,7 +227,7 @@ tests:
- name: "nodeId"
value: nodeId2
- name: "payload"
value: payload2
value: incorrectSetupCodePayload
response:
error: FAILURE

Expand All @@ -186,7 +243,7 @@ tests:
- name: "nodeId"
value: nodeId2
- name: "payload"
value: payload2
value: incorrectSetupCodePayload
response:
error: FAILURE

Expand All @@ -202,7 +259,7 @@ tests:
- name: "nodeId"
value: nodeId2
- name: "payload"
value: payload2
value: incorrectSetupCodePayload
response:
error: FAILURE

Expand All @@ -217,7 +274,7 @@ tests:
- name: "nodeId"
value: nodeId2
- name: "payload"
value: payload2
value: incorrectSetupCodePayload
response:
error: FAILURE

Expand All @@ -233,7 +290,7 @@ tests:
- name: "nodeId"
value: nodeId2
- name: "payload"
value: payload2
value: incorrectSetupCodePayload
response:
error: FAILURE

Expand All @@ -249,7 +306,7 @@ tests:
- name: "nodeId"
value: nodeId2
- name: "payload"
value: payload2
value: incorrectSetupCodePayload
response:
error: FAILURE

Expand All @@ -265,7 +322,7 @@ tests:
- name: "nodeId"
value: nodeId2
- name: "payload"
value: payload2
value: incorrectSetupCodePayload
response:
error: FAILURE

Expand All @@ -281,7 +338,7 @@ tests:
- name: "nodeId"
value: nodeId2
- name: "payload"
value: payload2
value: incorrectSetupCodePayload
response:
error: FAILURE

Expand All @@ -297,7 +354,7 @@ tests:
- name: "nodeId"
value: nodeId2
- name: "payload"
value: payload2
value: incorrectSetupCodePayload
response:
error: FAILURE

Expand All @@ -313,7 +370,7 @@ tests:
- name: "nodeId"
value: nodeId2
- name: "payload"
value: payload2
value: incorrectSetupCodePayload
response:
error: FAILURE

Expand All @@ -329,7 +386,7 @@ tests:
- name: "nodeId"
value: nodeId2
- name: "payload"
value: payload2
value: incorrectSetupCodePayload
response:
error: FAILURE

Expand All @@ -345,7 +402,7 @@ tests:
- name: "nodeId"
value: nodeId2
- name: "payload"
value: payload2
value: incorrectSetupCodePayload
response:
error: FAILURE

Expand All @@ -361,7 +418,7 @@ tests:
- name: "nodeId"
value: nodeId2
- name: "payload"
value: payload2
value: incorrectSetupCodePayload
response:
error: FAILURE

Expand All @@ -377,10 +434,11 @@ tests:
- name: "nodeId"
value: nodeId2
- name: "payload"
value: payload2
value: incorrectSetupCodePayload
response:
error: FAILURE

# This step must match the verification step above where we checked `correctPayload`
- label:
"TH_CR2 attempts to do PASE to DUT_CE using the correct onboarding
payload"
Expand All @@ -393,6 +451,6 @@ tests:
- name: "nodeId"
value: nodeId2
- name: "payload"
value: payload
value: correctPayload
response:
error: FAILURE
Loading

0 comments on commit 1075640

Please sign in to comment.