Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[chiptool.py] chiptool.py tests command missing support for several command line arguments used by Matter TH #26303

Closed
mikaelhm opened this issue Apr 28, 2023 · 2 comments · Fixed by #26333
Labels
yaml Missing features or bugs in the YAML test harness

Comments

@mikaelhm
Copy link
Contributor

mikaelhm commented Apr 28, 2023

Reproduction steps / Feature

Steps to reproduce:

  1. Build chip-tool and chip-all-clusters-app in SDK Repo

  2. Run chip-all-clusters-app

  3. Perform "paring" using chiptool.py from same ethernet

    ./scripts/tests/yaml/chiptool.py pairing onnetwork-long 0x12344321 20202021 3840
  4. Attempt to run a test case Matter TH:

    ./scripts/tests/yaml/chiptool.py tests Test_TC_ACE_1_1 \
       --nodeId 0x12344321 \
       --delayInMs 250 \
       --timeout 900 \
       --continueOnFailure 1 \
       --trace_file "/tmp/trace_log_TEST_Test_TC_ACE_1_1.log" \
       --trace_decode 1
  5. Note that this fails with error:

    "/root/scripts/py_matter_yamltests/matter_yamltests/parser.py", line 977, in __apply_config_override
    if isinstance(config[key], dict) and 'defaultValue' in config[key]:
    KeyError: 'delayInMs'
    

I tried removing the parameters one at a time, and verified that the following arguments are not supported:

  • --delayInMs
  • --timeout
  • --continueOnFailure
  • --trace_file
  • --trace_decode

Note: --nodeId is supported, but never works as it raises a TypeError no matter the format of the value:

"/root/examples/chip-tool/py_matter_chip_tool_adapter/matter_chip_tool_adapter/encoder.py", line 186, in __maybe_add_destination
    destination_argument_value = hex(request.node_id)
TypeError: 'str' object cannot be interpreted as an integer

Platform

python

Platform Version(s)

No response

Type

Manually tested with SDK

(Optional) If manually tested please explain why this is only manually tested

No response

Anything else?

continueOnFailure and delayInMs commands are implemented here for chip-tool binary:
https://github.com/project-chip/connectedhomeip/blob/77d72a84039fea5b551378d9f7d5119f5d9a16ab/examples/chip-tool/commands/tests/TestCommand.h#L52C1-L54

timeout is defined here for chip-tool binary:
https://github.com/project-chip/connectedhomeip/blob/77d72a84039fea5b551378d9f7d5119f5d9a16ab/examples/chip-tool/commands/clusters/ModelCommand.h#LL52C1-L52C1

@bzbarsky-apple
Copy link
Contributor

@vivien-apple

@krypton36
Copy link
Contributor

When adding these, can you please make it where it is part of some config in the runner? chiptool.py can update the parameters of the config base on the command line args. The reason I ask for this is because I am working on implementing a new runner specifically for the test harness and we will need these arguments to be available directly as part of a config.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
yaml Missing features or bugs in the YAML test harness
Projects
None yet
3 participants