Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix and enable Matter Casting automation CI check for commissioner-generated-passcode flow. #34178

Merged
merged 1 commit into from
Jul 4, 2024
Merged

Fix and enable Matter Casting automation CI check for commissioner-generated-passcode flow. #34178

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion .github/workflows/examples-linux-tv-casting-app.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,12 +63,22 @@ jobs:
./scripts/run_in_build_env.sh \
"scripts/examples/gn_build_example.sh examples/tv-casting-app/linux/ out/tv-casting-app chip_casting_simplified=true"

- name: Test casting from Linux tv-casting-app to Linux tv-app
- name:
Test casting from Linux tv-casting-app to Linux tv-app -
Commissionee Generated Passcode
run: |
./scripts/run_in_build_env.sh \
"python3 ./scripts/tests/run_tv_casting_test.py"
timeout-minutes: 2 # Comment this out to debug if GitHub Action times out.

- name:
Test casting from Linux tv-casting-app to Linux tv-app -
Commissioner Generated Passcode
run: |
./scripts/run_in_build_env.sh \
"python3 ./scripts/tests/run_tv_casting_test.py --commissioner-generated-passcode=True"
timeout-minutes: 2 # Comment this out to debug if GitHub Action times out.

- name: Uploading Size Reports
uses: ./.github/actions/upload-size-reports
if: ${{ !env.ACT }}
Expand Down
118 changes: 113 additions & 5 deletions scripts/tests/linux/tv_casting_test_sequences.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,9 +67,13 @@

# Values that identify the Linux tv-app and are noted in the 'Device Configuration' in the Linux tv-app output
# as well as under the 'Discovered Commissioner' details in the Linux tv-casting-app output.
VENDOR_ID = 0xFFF1 # Spec 7.20.2.1 MEI code: test vendor IDs are 0xFFF1 to 0xFFF4
PRODUCT_ID = 0x8001 # Test product id
DEVICE_TYPE_CASTING_VIDEO_PLAYER = 0x23 # Device type library 10.3: Casting Video Player
VENDOR_ID = 0xFFF1 # 0xFFF1 = 65521; Spec 7.20.2.1 MEI code: test vendor IDs are 0xFFF1 to 0xFFF4
PRODUCT_ID = 0x8001 # 0x8001 = 32769 = Test product id
DEVICE_TYPE_CASTING_VIDEO_PLAYER = 0x23 # 0x23 = 35 = Device type library 10.3: Casting Video Player

# 0x457 = 1111 = Target Content Application Vendor ID for the commissioner generated passcode flow
COMMISSIONER_GENERATED_PASSCODE_VENDOR_ID = 0x457
COMMISSIONER_GENERATED_PASSCODE = '0x00BC_614E' # 0x00BC_614E = 12345678 = Default commissioner generated passcode

# Value to verify the subscription state against in the Linux tv-casting-app output.
ATTRIBUTE_CURRENT_PLAYBACK_STATE = 0x0000_0000 # Application Cluster Spec 6.10.6 Attribute ID: Current State of Playback
Expand All @@ -94,7 +98,7 @@
Step(app=App.TV_CASTING_APP, output_msg=['Discovered CastingPlayer #0', f'Product ID: {PRODUCT_ID}', f'Vendor ID: {VENDOR_ID}',
f'Device Type: {DEVICE_TYPE_CASTING_VIDEO_PLAYER}', 'Supports Commissioner Generated Passcode: true']),

# Send `cast request {valid_discovered_commissioner_number}\n` command to the tv-casting-app subprocess.
# Send `cast request {valid_discovered_castingplayer_number}\n` command to the tv-casting-app subprocess.
Step(app=App.TV_CASTING_APP, input_cmd='cast request 0\n'),

# Validate that the tv-casting-app begins the commissioning process.
Expand All @@ -112,7 +116,7 @@
# Validate that we received the instructions on the tv-app output for sending the `controller ux ok` command.
Step(app=App.TV_APP, output_msg=['Via Shell Enter: controller ux ok|cancel']),

# Send `controller ux ok` command to the tv-app subprocess.
# Send `controller ux ok\n` command to the tv-app subprocess.
Step(app=App.TV_APP, input_cmd='controller ux ok\n'),

# Validate that pairing succeeded between the tv-casting-app and the tv-app.
Expand Down Expand Up @@ -140,6 +144,110 @@
# Signal to stop the tv-casting-app as we finished validation.
Step(app=App.TV_CASTING_APP, input_cmd=STOP_APP),

# Signal to stop the tv-app as we finished validation.
Step(app=App.TV_APP, input_cmd=STOP_APP)
]
),
Sequence(
name='commissioner_generated_passcode_test',
steps=[
# Signal to start the tv-app.
Step(app=App.TV_APP, input_cmd=START_APP),

# Validate that the tv-app is up and running.
Step(app=App.TV_APP, timeout_sec=APP_MAX_START_WAIT_SEC, output_msg=['Started commissioner']),

# Signal to start the tv-casting-app.
Step(app=App.TV_CASTING_APP, input_cmd=START_APP),

# Validate that the server is properly initialized in the tv-casting-app output.
Step(app=App.TV_CASTING_APP, timeout_sec=APP_MAX_START_WAIT_SEC, output_msg=['Server initialization complete']),

# Validate that there is a valid discovered casting player with {PRODUCT_ID}, {VENDOR_ID}, {DEVICE_TYPE_CASTING_VIDEO_PLAYER}, and the
# `Commissioner Generated Passcode` flag is set to true in the tv-casting-app output.
Step(app=App.TV_CASTING_APP, output_msg=['Discovered CastingPlayer #0', f'Product ID: {PRODUCT_ID}', f'Vendor ID: {VENDOR_ID}',
f'Device Type: {DEVICE_TYPE_CASTING_VIDEO_PLAYER}', 'Supports Commissioner Generated Passcode: true']),

# Send `cast request {valid_discovered_castingplayer_number} commissioner-generated-passcode\n` command to the tv-casting-app subprocess.
Step(app=App.TV_CASTING_APP, input_cmd='cast request 0 commissioner-generated-passcode\n'),

# Validate that the tv-casting-app begins the commissioning process.
Step(app=App.TV_CASTING_APP, output_msg=[
'CastingPlayer::VerifyOrEstablishConnection() calling OpenBasicCommissioningWindow()']),

# Validate that the `IdentificationDeclaration` message sent from the tv-casting-app to the tv-app will contain the following entries:
# mCommissionerPasscode: true -> This flag instructs the commissioner to use the commissioner-generated-passcode flow for commissioning.
# mCommissionerPasscodeReady: false -> This flag indicates that the commissionee has not obtained the commissioner passcode from the user and
# thus is not ready for commissioning.
# Vendor ID: {COMMISSIONER_GENERATED_PASSCODE_VENDOR_ID} -> The initial VENDOR_ID of the casting player will be overridden to {COMMISSIONER_GENERATED_PASSCODE_VENDOR_ID}.
# Otherwise we will enter the commissionee-generated-passcode flow.
Step(app=App.TV_CASTING_APP, output_msg=['IdentificationDeclarationOptions::LogDetail()', 'IdentificationDeclarationOptions::mCommissionerPasscode: true',
'IdentificationDeclarationOptions::mCommissionerPasscodeReady: false', 'IdentificationDeclarationOptions::TargetAppInfos list:', f'TargetAppInfo 1, Vendor ID: {COMMISSIONER_GENERATED_PASSCODE_VENDOR_ID}']),

# Validate that we received the cast request from the tv-casting-app on the tv-app output.
Step(app=App.TV_APP,
output_msg=['------PROMPT USER: Test TV casting app is requesting permission to cast to this TV, approve?']),

# Validate that we received the instructions on the tv-app output for sending the `controller ux ok` command.
Step(app=App.TV_APP, output_msg=['Via Shell Enter: controller ux ok|cancel']),

# Send `controller ux ok` command to the tv-app subprocess.
Step(app=App.TV_APP, input_cmd='controller ux ok\n'),

# Validate that the tv-app sent a message to the tv-casting-app indicating that the tv-app is now displaying the passcode to the user.
Step(app=App.TV_APP, output_msg=['Commissioner Declaration Start',
'commissioner passcode: true', 'Commissioner Declaration End']),

# Validate that we received the cast request with the casting passcode on the tv-app output.
Step(app=App.TV_APP, output_msg=[
f'------PROMPT USER: Test TV casting app is requesting permission to cast to this TV. Casting passcode: [{COMMISSIONER_GENERATED_PASSCODE}].']),

# Validate that the tv-casting-app received the message from the tv-app indicating that the tv-app is now displaying the passcode to the user.
Step(app=App.TV_CASTING_APP, output_msg=['Commissioner Declaration Start',
'commissioner passcode: true', 'Commissioner Declaration End']),

# Validate that the user is prompted to input passcode from the tv-app on the tv-casting-app output.
Step(app=App.TV_CASTING_APP, output_msg=['Awaiting user input', 'Input the Commissioner-Generated passcode displayed on the CastingPlayer UX.',
f'cast setcommissionerpasscode {int(COMMISSIONER_GENERATED_PASSCODE, 16)}', 'Awaiting user input']),

# Send `cast setcommissionerpasscode {COMMISSIONER_GENERATED_PASSCODE}\n` to the tv-casting-app subprocess.
Step(app=App.TV_CASTING_APP, input_cmd=f'cast setcommissionerpasscode {int(COMMISSIONER_GENERATED_PASSCODE, 16)}\n'),

# Validate the commissioner passcode that the user entered on the tv-casting-app output.
Step(app=App.TV_CASTING_APP, output_msg=[
f'CommandHandler() setcommissionerpasscode user-entered passcode: {int(COMMISSIONER_GENERATED_PASSCODE, 16)}']),
shaoltan-amazon marked this conversation as resolved.
Show resolved Hide resolved

# Validate that the `IdentificationDeclaration` message sent from the tv-casting-app to the tv-app will contain the following entries:
# mCommissionerPasscode: true -> This flag instructs the commissioner to use the commissioner-generated-passcode flow for commissioning.
# mCommissionerPasscodeReady: true -> This flag indicates that the commissionee has obtained the commissioner passcode from the user and
# thus is ready for commissioning.
# Vendor ID: {COMMISSIONER_GENERATED_PASSCODE_VENDOR_ID} -> The initial VENDOR_ID of the casting player will be overridden to {COMMISSIONER_GENERATED_PASSCODE_VENDOR_ID}.
# Otherwise we will enter the commissionee-generated-passcode flow.
Step(app=App.TV_CASTING_APP, output_msg=['IdentificationDeclarationOptions::LogDetail()', 'IdentificationDeclarationOptions::mCommissionerPasscode: true',
'IdentificationDeclarationOptions::mCommissionerPasscodeReady: true', 'IdentificationDeclarationOptions::TargetAppInfos list:', f'TargetAppInfo 1, Vendor ID: {COMMISSIONER_GENERATED_PASSCODE_VENDOR_ID}']),

# Validate that pairing succeeded between the tv-casting-app and the tv-app.
Step(app=App.TV_APP, output_msg=['Secure Pairing Success']),

# Validate that the connection succeeded in the tv-casting-app output.
Step(app=App.TV_CASTING_APP, output_msg=['Successfully connected to CastingPlayer']),

# Validate that commissioning succeeded in the tv-app output.
Step(app=App.TV_APP, output_msg=['------PROMPT USER: commissioning success']),

# Validate that we are able to subscribe to the media playback cluster by reading the CurrentState value and that it matches {ATTRIBUTE_CURRENT_PLAYBACK_STATE}.
Step(app=App.TV_CASTING_APP, output_msg=[f'Read CurrentState value: {ATTRIBUTE_CURRENT_PLAYBACK_STATE}']),

# Validate the LaunchURL in the tv-app output.
Step(app=App.TV_APP,
output_msg=['ContentLauncherManager::HandleLaunchUrl TEST CASE ContentURL=https://www.test.com/videoid DisplayString=Test video']),

# Validate the LaunchURL in the tv-casting-app output.
Step(app=App.TV_CASTING_APP, output_msg=['LaunchURL Success with response.data: exampleData']),

# Signal to stop the tv-casting-app as we finished validation.
Step(app=App.TV_CASTING_APP, input_cmd=STOP_APP),

# Signal to stop the tv-app as we finished validation.
Step(app=App.TV_APP, input_cmd=STOP_APP)
]
Expand Down
30 changes: 23 additions & 7 deletions scripts/tests/run_tv_casting_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -270,12 +270,26 @@ def run_test_sequence_steps(
@click.command()
@click.option('--tv-app-rel-path', type=str, default='out/tv-app/chip-tv-app', help='Path to the Linux tv-app executable.')
@click.option('--tv-casting-app-rel-path', type=str, default='out/tv-casting-app/chip-tv-casting-app', help='Path to the Linux tv-casting-app executable.')
def test_casting_fn(tv_app_rel_path, tv_casting_app_rel_path):
@click.option('--commissioner-generated-passcode', type=bool, default=False, help='Enable the commissioner generated passcode test flow.')
def test_casting_fn(tv_app_rel_path, tv_casting_app_rel_path, commissioner_generated_passcode):
"""Test if the casting experience between the Linux tv-casting-app and the Linux tv-app continues to work.

Default paths for the executables are provided but can be overridden via command line arguments.
For example: python3 run_tv_casting_test.py --tv-app-rel-path=path/to/tv-app
--tv-casting-app-rel-path=path/to/tv-casting-app
By default, it uses the provided executable paths and the commissionee generated passcode flow as the test sequence.

Example usages:
1. Use default paths and test sequence:
python3 run_tv_casting_test.py

2. Use custom executable paths and default test sequence:
python3 run_tv_casting_test.py --tv-app-rel-path=path/to/tv-app --tv-casting-app-rel-path=path/to/tv-casting-app

3. Use default paths and a test sequence that is not the default test sequence (replace `test-sequence-name` with the actual name of the test sequence):
python3 run_tv_casting_test.py --test-sequence-name=True

4. Use custom executable paths and a test sequence that is not the default test sequence (replace `test-sequence-name` with the actual name of the test sequence):
python3 run_tv_casting_test.py --tv-app-rel-path=path/to/tv-app --tv-casting-app-rel-path=path/to/tv-casting-app --test-sequence-name=True

Note: In order to enable a new test sequence, we also need to define a @click.option() entry for the test sequence.
"""

# Store the log files to a temporary directory.
Expand All @@ -288,15 +302,17 @@ def test_casting_fn(tv_app_rel_path, tv_casting_app_rel_path):
# Get all the test sequences.
test_sequences = Sequence.get_test_sequences()

# Get the test sequence of interest.
test_sequence = Sequence.get_test_sequence_by_name(test_sequences, 'commissionee_generated_passcode_test')
# Get the test sequence that we are interested in validating.
test_sequence_name = 'commissionee_generated_passcode_test'
if commissioner_generated_passcode:
test_sequence_name = 'commissioner_generated_passcode_test'
test_sequence = Sequence.get_test_sequence_by_name(test_sequences, test_sequence_name)

if not test_sequence:
logging.error('No test sequence found by the test sequence name provided.')
handle_casting_failure(None, [])

# At this point, we have retrieved the test sequence of interest.
test_sequence_name = test_sequence.name
test_sequence_steps = test_sequence.steps

# Configure command options to disable stdout buffering during tests.
Expand Down
Loading