Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Test_TC_IDM_1_2] Unknown clusters lead to harness failures in matter_chip_tool_adapter #29115

Closed
bzbarsky-apple opened this issue Sep 7, 2023 · 5 comments · Fixed by #29469
Closed

[Test_TC_IDM_1_2] Unknown clusters lead to harness failures in matter_chip_tool_adapter #29115

bzbarsky-apple opened this issue Sep 7, 2023 · 5 comments · Fixed by #29469
Assignees
@vivien-apple
Labels
build issue needs triage request cert blocker tests

Comments

@bzbarsky-apple
Copy link
Contributor

Test issue(s)

When trying to run the Test_TC_IDM_1_2 YAML test against all-clusters-app, I get failures like so:

2023-09-07 15:13:16.794 ERROR   15:13:16.713 - TEST ERR  :   File "examples/chip-tool/py_matter_chip_tool_adapter/matter_chip_tool_adapter/decoder.py", line 113, in __translate_names
2023-09-07 15:13:16.794 ERROR   15:13:16.713 - TEST ERR  :     raise KeyError(
2023-09-07 15:13:16.794 ERROR   15:13:16.713 - TEST ERR  : KeyError: 'Error: The cluster (0x7) attribute (0xfff9) definition can not be found. Please update the cluster definition.'
2023-09-07 15:13:16.794 ERROR   15:13:16.736 - TEST OUT  :              7. Running Get the list of available cluster commands on the device (all endpoints)

It seems like his wildcard read is getting dumped into code that expects to know about all clusters, which we just do not. This specific cluster is draft, but this would presumably also fail for manufacturer-specific things.

Platform

core (please add to version below)

Anything else?

No response
@github-project-automation github-project-automation bot moved this to Open Cert Blockers in [Certification] Blockers Sep 7, 2023
@cecille
Copy link
Contributor

cecille commented Sep 8, 2023

Is this happening on ToT? Why would this not cause CI failures?

@cecille
Copy link
Contributor

cecille commented Sep 8, 2023

Is this blocking something in SVE right now?

@bzbarsky-apple
Copy link
Contributor Author

bzbarsky-apple commented Sep 8, 2023 •
edited
Loading

Is this happening on ToT?

Yes.

Why would this not cause CI failures?

Because for several weeks now we have not been running YAML tests in CI, effectively. I ran into this while trying to fix that in #29096 and I disabled this YAML test as part of that PR, since it fails. This is the issue tracking the fact that it's disabled.

Is this blocking something in SVE right now?

No. SVE is using src/python_testing/TC_IDM_1_2.py as I understand it. Raising the question: Why does src/app/tests/suites/certification/Test_TC_IDM_1_2.yaml even exist?

@cecille
Copy link
Contributor

cecille commented Sep 9, 2023 •
edited
Loading

Is this happening on ToT?

Yes.

Why would this not cause CI failures?

Because for several weeks now we have not been running YAML tests in CI, effectively. I ran into this while trying to fix that in #29096 and I disabled this YAML test as part of that PR, since it fails. This is the issue tracking the fact that it's disabled.

Oh, that's special. :(

Is this blocking something in SVE right now?

No. SVE is using src/python_testing/TC_IDM_1_2.py as I understand it. Raising the question: Why does src/app/tests/suites/certification/Test_TC_IDM_1_2.yaml even exist?

I'm not sure, but it seems like the easiest thing to do to unblock cert/fix CI is to delete that test, and just open an issue to fix wildcard reads in the yaml runner, if we still care about wildcards in yaml.

@bzbarsky-apple
Copy link
Contributor Author

Oh, that's special. :(

Hey, at least now we have CI that will fail if we we not running YAML tests, I hope... ;)

the easiest thing to do to unblock cert/fix CI is to delete that test,

That seems perfectly reasonable to me if we are not using this "certification" test for cert. @manjunath-grl @raju-apple

cecille added a commit to cecille/connectedhomeip that referenced this issue Sep 27, 2023
@cecille
Remove TC-IDM-1.2 yaml
b762907 
This test has a functional python implementation that is fully
automated. The yaml is no longer required, and doesn't work anyway.

Fixes: project-chip#29115
@cecille cecille mentioned this issue Sep 27, 2023
Merged
@mergify mergify bot closed this as completed in #29469 Sep 28, 2023
mergify bot pushed a commit that referenced this issue Sep 28, 2023
@cecille
Remove TC-IDM-1.2 yaml (#29469)
d1fa6ee 
This test has a functional python implementation that is fully
automated. The yaml is no longer required, and doesn't work anyway.

Fixes: #29115
@github-project-automation github-project-automation bot moved this from Open Cert Blockers to Complete in [Certification] Blockers Sep 28, 2023
@github-project-automation github-project-automation bot moved this from Todo to Done in [Build] Build Issues Sep 28, 2023
HunsupJung pushed a commit to HunsupJung/connectedhomeip that referenced this issue Oct 23, 2023
@cecille @HunsupJung
Remove TC-IDM-1.2 yaml (project-chip#29469)
589b459 
This test has a functional python implementation that is fully
automated. The yaml is no longer required, and doesn't work anyway.

Fixes: project-chip#29115
@bzbarsky-apple bzbarsky-apple mentioned this issue Dec 4, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@vivien-apple vivien-apple

Labels
build issue needs triage request cert blocker tests
Projects
[Build] Build Issues
Archived in project
[Certification] Blockers
Archived in project
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Remove TC-IDM-1.2 yaml cecille/connectedhomeip
3 participants
@cecille @bzbarsky-apple @vivien-apple