Create test script to verify that the Linux tv-casting-app is able to discover the Linux tv-app. #32919
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pullapprove
bot
requested review from
andyg-apple,
anush-apple,
arkq,
bzbarsky-apple,
carol-apple,
cecille,
chrisdecenzo,
chshu,
chulspro,
cliffamzn,
Damian-Nordic,
dhrishi,
harimau-qirex,
harsha-rajendran,
hawk248,
hicklin,
jepenven-silabs,
jmartinez-silabs,
jmeg-sfy,
joonhaengHeo,
jtung-apple,
kkasperczyk-no,
ksperling-apple,
lazarkov,
lpbeliveau-silabs,
LuDuda and
mhazley
andy31415
reviewed
Apr 23, 2024
andy31415
reviewed
Apr 23, 2024
andy31415
reviewed
Apr 23, 2024
andy31415
reviewed
Apr 23, 2024
andy31415
reviewed
Apr 23, 2024
andy31415
reviewed
Apr 23, 2024
shaoltan-amazon
force-pushed
the
test-Linux-discovery
branch
from
7c72514 to
dc5a683
Compare
|
PR #32919: Size comparison from 1bc65f7 to dc5a683 Decreases (1 build for efr32)
Full report (71 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, mbed, nrfconnect, nxp, psoc6, qpg, stm32, telink)
|
andy31415
reviewed
Apr 25, 2024
andy31415
approved these changes
Apr 25, 2024
discover the Linux tv-app. (project-chip#32918)
…updated validate_value function, and added succeeding fast logic.
…dule instead of print statements.
Added '.' at the end of code comments.
…file handling, and for managing subprocesses. Added clarifying code comments, utilized temporary directory to store log files, added path arguments to the test program, added command to not allow buffering for output streams, and changed from sleep() to tv-app start up time-out condition.
…_ID, and DEVICE_TYPE. Updated the run command in the workflow yaml file.
…pdated to follow doc-style comment format, removed redundant comments, removed LogFileManager class as temporary directory will handle file clean up, split test_discovery_fn to improve code readability.
…mporary_logs_to_console` function.
shaoltan-amazon
force-pushed
the
test-Linux-discovery
branch
from
dc5a683 to
e082247
Compare
|
PR #32919: Size comparison from 8b21ecf to e082247 Decreases (1 build for efr32)
Full report (71 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, mbed, nrfconnect, nxp, psoc6, qpg, stm32, telink)
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #32918.
Problem
There is no CI check in place to test that the discovery between the Linux tv-casting-app and the Linux tv-app continues to work whenever a PR is created.
Solution Overview
Create a test script to verify that the discovery between the Linux tv-casting-app and the Linux tv-app continue to work. The test script will be a part of the workflow yaml file that will run whenever a PR is created.
The main idea is to run the Linux tv-casting-app and the Linux tv-app as separate processes and write their respective output to separate log files. The output of the tv-app process will be parsed in realtime for the string
Started commissionerto indicate that the tv-app has started successfully. Next, the output of the tv-casting-app process will also be parsed in realtime for specific strings to indicate a successful or a failed discovery. If the strings of interest are found, then we print them to the console along withDiscovery success!. Otherwise, we print the reason for the discovery failure along withDiscovery failed!. Then, we will also dump the logs of both the tv-casting-app and the tv-app to the console and exit on error.In order for discovery to be considered as successful, the
Discovered Commissionerin the tv-casting-app needs to have aVendor IDvalue of0xFFF1(65521),Product IDvalue of0x8001(32769), and aDevice Typevalue of0x23(35) as these values correspond to the Test TV App. Discovery fails when any of the entries of interest do not match the expected values or if "No commissioner discovered" string is found in the output of the tv-casting-app.Note: The default relative paths to the tv-app and the tv-casting-app are
out/tv-app/chip-tv-appandout/tv-casting-app/chip-tv-casting-apprespectively. However, the user can pass their relative path via the command line by running:python3 run_tv_casting_test.py --tv-app-rel-path=path/to/tv-app --tv-casting-app-rel-path=path/to/tv-casting-appTesting
Tested the script locally by running
python3 ./scripts/tests/run_tv_casting_test.pyand observing the output. Also verified that the CI check works as expected by observing the workflow run on my local branch.