Splicing tests not yet defined

[rcottingham]

Splicing tests have not been defined. These are specified in §8.8, §8.13 and §8.14 in the Device Playback Capabilities Specification.

It has therefore not been possible to verify the OF logic for these tests.

For further details on the issue please see:
“Eurofins CTA WAVE Test Suite validation of the Observation Framework results including analysis 31 Jan 2023.xls”

[jpiesing]

I'm a little confused at the problem description here.
Is this saying that the splicing tests were left out of the .csv file that is used to combine template HTML+JS and test content into an actual test case?

[rcottingham]

Hi Jon - splicing tests results have not yet been provided by Fraunhofer, so we believe they are not ready.

There is some test splicing content in the avc_sets folder, but not in the required CMAF media profiles. (Also, the content in the avc_sets appears to be general playback and not splicing).

As soon as we get confirmation from Fraunhofer that the splicing tests have passed, we will run them in the Observation Framework.

I hope that clarifies and apologies to all for any confusion.

[FritzHeiden]

As far as I understand, the provided test results are supposed to help to validate the new test content (2022-10-17). Splicing content was not included in this content update, so the corresponding tests were not included.

[jpiesing]

@FritzHeiden @rbouqueau says in cta-wave/Test-Content#19 that splicing content was created in October 2022. Please can you find this content, update the .csv, generate the splicing tests with this October 2022 content and see if it works as far as you can test.

[rbouqueau]

I plan to update the streams and database.json today. Let me update you here when it's done (so that people work on the latest version of the content).

[rbouqueau]

That's updated.

[FritzHeiden]

I am unable to access the new splice content either online mpd or zip download.

[rbouqueau]

Indeed I had not updated everything. Could you please retry?

[FritzHeiden]

Tested splice current splice content (2023-04-28) and it works fine. Tests are still on separate branch (content-2023-04-28) and will be merged to master once all other tests work fine with the new content.

[FritzHeiden]

Spliced tests are now merged to master

[jpiesing]

@rcottingham Above you said that ..

As soon as we get confirmation from Fraunhofer that the splicing tests have passed, we will run them in the Observation Framework.

The above comments from Fritz suggest this is now the case. Do you need anything more from them?

[gitwjr]

@FritzHeiden to provide recordings to @rcottingham

[FritzHeiden]

@rcottingham recordings can be found here: https://drive.google.com/file/d/1_M_G-5rbiVnS6QQYcCUUEcggJ5reH_Ud/view?usp=sharing

[DannyKokkinos]

Hey @FritzHeiden could you please approve my request to access the recordings from the google drive.

[FritzHeiden]

@DannyKokkinos I updated access permissions for the link. Please try again

[rcottingham]

Splicing test ok. Limitation with current Master branch of Observation Framework with calculating the duration of the content with variable fragment durations. Issue fixed in DPCAT22 branch to be merged once development/testing complete. See issue cta-wave/device-observation-framework#46