Screen recording can't be opened natively on Mac

[timja]

Jenkins and plugins versions report

What Operating System are you using (both controller, and any agents involved in the problem)?

Reproduction steps

Open a ui-recording.mov from a test result
If you're on mac it will default to quicktime player, it fails to open

Expected Results

It opens

Actual Results

It doesn't

Anything else?

I'm using VLC to open them currently which works but it's the only thing I need VLC for these days...

Code is around here:

acceptance-test-harness/src/main/java/org/jenkinsci/test/acceptance/recorder/TestRecorderRule.java

Lines 72 to 87 in 200aafd

	String mimeType = FormatKeys.MIME_QUICKTIME;
	String videoFormatName = VideoFormatKeys.ENCODING_QUICKTIME_ANIMATION;
	String compressorName = VideoFormatKeys.COMPRESSOR_NAME_QUICKTIME_ANIMATION;
	Dimension outputDimension = gc.getBounds().getSize();
	int bitDepth = BIT_DEPTH;
	float quality = QUALITY_RATIO;
	int screenRate = FRAME_RATE_PER_SEC;
	Format outputFormatForScreenCapture = getOutputFormatForScreenCapture(videoFormatName,
	compressorName, outputDimension,
	bitDepth, quality, screenRate);
	this.screenRecorder = new JUnitScreenRecorder
	(gc, gc.getBounds(), getFileFormat(mimeType),
	outputFormatForScreenCapture, null, null, diagnostics);
	this.screenRecorder.start();

Found a blog post that's newer than our code here which may have some updated format:
https://www.ontestautomation.com/creating-a-video-capture-of-your-selenium-tests-using-monte-screen-recorder/

[jtnord]

You can not open it natively on Windows either and am also using VLC to view them.

[basil]

Found a blog post that's newer than our code here which may have some updated format: https://www.ontestautomation.com/creating-a-video-capture-of-your-selenium-tests-using-monte-screen-recorder/

I don't think it will be possible to fix this bug as long as we continue to use the Monte library. I tried all the video encodings listed in https://github.com/wrandelshofer/MonteMedia and the only one that worked in QuickTime Player for macOS was "Motion JPEG", but that created unusably large files (100+ MiB for a few seconds of recording). Sadly QuickTime Player just doesn't support old formats anymore, and the Monte library doesn't support newer formats either. On the plus side VLC can handle these old formats just fine.

[basil]

I suppose someone who is particularly motivated could add ffmpeg to our Docker image and then postprocess the video to convert it from QuickTime RLE to some more modern encoding that is supported by recent versions of QuickTime Player for macOS. I have no interest in working on this since VLC works well enough for me.

[wrandelshofer]

With the current version of the montemedia library you can create videos in the following formats, that will open natively:

• Windows: AVI with JPEG encoding.
• macOS: QuickTime with JPEG encoding.

I know that these settings produce impractically huge files.

I am planning to do a proof-of-concept with the Foreign Function & Memory API that was released in JDK 21.
Maybe it is possible to encode videos using the native Windows and macOS APIs.

[timja]

Thanks, if its useful, we record on linux and then view on mac and Windows mostly (probably some linux too)

[timja]

FWIW I was able to convert it to viewable in native player with both ffmpeg and handbrake:

ffmpeg:

ffmpeg -i ~/Downloads/ui-recording.mov -c:v libx264 -vf format=yuv420p -movflags +faststart ~/Downloads/ui-recording2.mp4

handbrake:

 handbrakecli -i ~/Downloads/ui-recording.mov -e x264 -E facc -o ~/Downloads/ui-recording.mp4

The codec name currently in use is qtrle which has had supported dropped from quicktime player

Both options are smaller than the original video:

junit-plugin [🌱 reduce-dependencies-2][☕ v17]
❯ du -h ~/Downloads/ui-recording.mp4
2.0M	/Users/timja/Downloads/ui-recording.mp4

junit-plugin [🌱 reduce-dependencies-2][☕ v17]
❯ du -h ~/Downloads/ui-recording2.mp4
2.0M	/Users/timja/Downloads/ui-recording2.mp4

junit-plugin [🌱 reduce-dependencies-2][☕ v17]
❯ du -h ~/Downloads/ui-recording.mov
6.8M	/Users/timja/Downloads/ui-recording.mov

[basil]

As stated in #1041 (comment), it probably wouldn't be too hard to install ffmpeg(1) into our Docker image and make a best-effort attempt to post-process the video into a modern format if the utility is available. If the utility is not available, the existing behavior can be retained.

[timja]

Yes, this was some initial research for that.

I also found this library which may simplify it: https://github.com/bramp/ffmpeg-cli-wrapper

[wrandelshofer]

| I am planning to do a proof-of-concept with the Foreign Function & Memory API that was released in JDK 21.
| Maybe it is possible to encode videos using the native Windows and macOS APIs.

I have looked briefly at the APIs in Windows and macOS. I think it is a lot of work (altough fun stuff).
The main issue is, that it adds a dependency on the OS and possibly on specific OS versions.

I am now doing a prototype with the MP4 H.264 codec in JCodec. The patents for H.264 have expired - as far as I know.
So it should be fine now to use it in open source projects.

[jtnord]

FYI, not that we are using test containers for webdriver but https://java.testcontainers.org/modules/webdriver_containers/#recording-videos

We have a largeish amount of home grown things in this project that now may have some OSS supported equivalent.