[REQ] FFMPEG's video capture (as source) support

[MarcoRavich]

As always thank you for your valuable work as well as your careful listening to users' requests.

Even if it probably don't falls within the project scope, it would be extremely interesting (and could make FF almost unique) to implement FFMPEG video capture as input.

We've got inspired by @Jessecar96's git on How to capture, deinterlace, and encode analog video which proposes an interesting method on how to set FFMPEG for video capture through dedicated hardware.

The most interesting (and recently made) resource is probably @StefanoLusardi's Video Capture wrapper for FFMPEG with Software/Hardware decoding and there's also vcr by @BurntSushi or - a bit dated - git by @joncampbell123 who has implemented "video capture, live feed display, and recording to an AVI file when recording activated" through FFMPEG in his Video capture and archiving utility for Linux.

If it helps to let you implement it easier, here are some simple FFMPEG capture scripts to get inspiration from:

• A collection of scripts to capture video with ffmpeg-python by @zfleeman
• FFMPEG scripts by @modeler
• FFMPEG-Capture by @pcntv

Last but not least @egemengulpinar's capture-device-list on Windows or @ericwooley's ffmpeg-devices on Mac may also help.

Hope that inspires !

[MarcoRavich]

OK here's the 1st quick test (played by Panasonic NV-HD630 on Blackmagic Intensity Pro).

commandline:
ffplay -pixel_format uyvy422 -framerate 25 -f dshow -i video="Decklink Video Capture":audio="Decklink Audio Capture" -aspect 4:3 -vf "yadif=1"

screenshot:

[zfleeman]

Hey all, thanks for the shoutout. The thing I could never figure out perfectly was the 'dshow' rtbufsize parameter with my video capture card. I was picking arbitrary values to prevent any frame skipping from the capture device, and I ended up landing on 2048M, here. That solved a frame skipping issue I had, but I was always left wondering if there was a better way to pick the best value or if there was another parameter altogether that I should have been using.

Other than that, working with dshow was surprisingly straight forward with the base ffmpeg as well as ffmpeg-python. Good luck, I'll be following this!

[MarcoRavich]

@zfleeman thanks for your scripts and hints !

There are a bunch of video capture parameters to consider in order to achieve good results in analog video digitization.

Many are nicely explained by @ViRb3 in his sdconv script page:

• Deinterlacing
• PAR
• chromaticity
• lossless encode
• etc.

We would love to put all these efforts into an FFMPEG commandline to let cool devs like @cdgriffith easily implements is in their GUIs.

About rtbufsize we do see different approaches: for example, in @pcntv record.bat is set to 1500M...
-rtbufsize 1500M -f dshow -pixel_format uyvy422 -framerate 29.97 -i video="Blackmagic WDM Capture":audio="Blackmagic WDM Capture"

...in this stackoverflow entry the author puts -rtbufsize 702000k in his commandline.

This means there's not a "standard" value to choose, so letting users freely set it up may be the best approach.

Hope to establish some kind of fruitful collaboration !

[MarcoRavich]

• according to many, yadif=0:-1:0 should give better results;
• someone uses -rtbufsize 1G;
• possible encoding live preview : "ffmpeg" -y -f dshow -rtbufsize INT_MAX -r <framerate> -i video="<capture_device>":audio="<capture_device>" ^ -c:v <video_codec> -c:a <audio_codec> <outputfile.ext> ^ -c:v rawvideo -c:a <audio_codec> -f nut - | "ffplay" -
• qv4l2 looks interesting

...still investigating...

[MarcoRavich]

After some research, here's a new test:

commandline used:

ffplay -rtbufsize 1G -pixel_format uyvy422 -framerate 25 -f dshow -i video="Decklink Video Capture":audio="Decklink Audio Capture" -aspect 4:3 -vf "bwdif=mode=send_field:parity=auto:deint=all"

(note: changed VCR in order to preserve the main one, test done on Samsung DVD-V6700 combo)

[MarcoRavich]

...still testing...

ffplay -rtbufsize 1G -pixel_format uyvy422 -framerate 25 -f dshow -i video="Decklink Video Capture":audio="Decklink Audio Capture" -aspect 4:3 -vf "scale=in_range=full:out_range=full,lut='y=(va l-16)*1.067',bwdif=mode=send_field:parity=auto:deint=all,crop=702:564:6:2"

Inspired by this reply @ videohelp.

[MarcoRavich]

Found an interesting Python code by @miguelfazenda called VCR:
https://github.com/miguelfazenda/VCR/blob/master/recorder.py

[MarcoRavich]

(still investigating)
...there are many interesting discussions about sws_flags usage to preserve video quality, but we honestly don't know if these parameters can also be useful in the acquisition phase:

Always add +accurate_rnd+full_chroma_int to sws_flags (by -sws_flags +accurate_rnd+full_chroma_int or sws_flags=+accurate_rnd+full_chroma_int in filtergraphs, or both), unless speed is prefered over quality.

Source: https://trac.ffmpeg.org/ticket/1582#comment:15

Another interesting - and funny - (re)source about this: https://kohana.fi/article/color-conversion-issues-with-ffmpeg

EDIT: just found another interesting - even if bit dated - git by @lauramossity containing python scripts to convert and edit captured videos from camcorder.

[MarcoRavich]

Very interesting finds to consider:

https://digitensions.home.blog/2019/03/18/moving-video-tape-capture-to-an-open-source-workflow/

https://blogs.library.duke.edu/bitstreams/2021/03/12/ffv1-the-gains-of-lossless/

[MarcoRavich]

[FUN] chatGPT "opinion"

Here's an FFMPEG command line that you can use on Windows to acquire an audio/video stream from a capture card using a lossless codec:

ffmpeg -f dshow -i video="Your Capture Card Name" -f dshow -i audio="Your Audio Device Name" -c:v utvideo -c:a pcm_s16le -pix_fmt yuv420p -preset ultrafast output.avi

This command assumes that you have installed FFMPEG on your Windows system and that you know the names of your capture card and audio device.

Replace "Your Capture Card Name" and "Your Audio Device Name" with the actual names of your capture card and audio device. You can find the names of your devices by running the command ffmpeg -list_devices true -f dshow -i dummy.

The command uses the dshow input format to capture video and audio from the specified devices. It encodes the video using the lossless utvideo codec and the audio using the uncompressed pcm_s16le codec. The output file format is AVI, and the output video is in yuv420p pixel format. The ultrafast preset is used to encode the video quickly, but you can choose a different preset if you want to trade off encoding speed for file size.

The output is saved to a file named "output.avi" in the current directory. You can change the filename and path to save the output file to a different location.

EDIT: refinement (after some observations)

ffmpeg -f dshow -i video="Your Capture Card Name" -f dshow -i audio="Your Audio Device Name" -c:v huffyuv -c:a pcm_s16le -pix_fmt yuv420p -preset ultrafast -bufsize 2G -f matroska output.mkv

[cdgriffith]

These are fun ideas, but way outside anything I was thinking when making this program and is very ill suited for from a coding perspective. It's all open source, so anyone else can have at it if they wish, but this is not something I could see being added myself.

Other discussion: #342