Support Device and Named Pipes URLs as Source

[charrsId]

how to do?tkx

[MrBean2016]

What do you mean with byte[] video?
Is is a file with a correct header or something like a stream?
Or is the question how to set fme:MediaElement to use the file, open it and play the video?

Check the sample app FFME.Sample, it is all in there.

[mariodivece]

Yeah, I don't know what @charrsId means either. If he means setting a byte[] as the media container source, I don't think this will be available (at least not soon). What I am planning on supporting later (3.0 maybe?) are dshow devices and named pipes so that the URLs look like device://dshow?video="My WebCam"&audio="My Microphone" or pipe://named_pipe_here
In that way you can feed the byte[] into the named pipe

[charrsId]

Thank you for your reply.
Yes, I want to use ffme to play video stream. But now it seems impossible, right?

[tenisak]

Thank you very much for your hard work on this project!
Please, how do you see this with implementation of capture devices? I'm looking forward to it. That is the only thing that is missing to perfection :-)

[mariodivece]

yep. This is most likely the first item for the next version, 3.0. Still trying to get a final release of 2.0

[tenisak]

That's great! Thanks so much!

[mariodivece]

Latest Commit: It is now possible to open devices with URIs like the following:

device://dshow/?audio=Microphone (Vengeance 2100)
device://gdigrab?title=Command Prompt
device://gdigrab?desktop

[tenisak]

Thanks for implementing this feature! Can you please advise how to do this ffplay example? (#128)
ffplay -f dshow -video_size 768x576 -framerate 25 -pixel_format yuyv422 -i video="Osprey-460e Video Device 1C"

[mariodivece]

Try this and let me know the results

Media.OnMediaOpening(s, e) =>
{
    e.Options.Input["framerate"] = "25";
    e.Options.Input["video_size"] = "768x576";
    e.Options.Input["pixel_format"] = "yuyv422";
};
Media.Source = new Uri("device://dshow/?video=Osprey-460e Video Device 1C");

[tenisak]

If I open capture device without any other parameters, it works:
Media.Source = new Uri("device://dshow/?video=Logitech HD Webcam C270");

But parameters can not be inserted:

Error message: 'MediaElement' does not contain a definition for 'OnMediaOpening' and no extension method 'OnMediaOpening' accepting a first argument of type 'MediaElement' could be found (are you missing a using directive or an assembly reference?)

Or another option also does not work:

Error message: 'MediaOptions' does not contain a definition for 'Input' and no extension method 'Input' accepting a first argument of type 'MediaOptions' could be found (are you missing a using directive or an assembly reference?)

[mariodivece]

sorry, I meant the MediaInitializing event, not the mediaopening. Try the MediaInitializing instead (i.e. before opeining the input that is)

[tenisak]

Thanks, it works!
I tested it with an ordinary webcam and it works. During the week I will test it with two capture cards. I report the result.

I have two more questions:

1. If the capture device fails to open (eg name error, disconnected, etc.), FFME only displays a blank area. How to recognize why it failed to connect?

2. BTW, is it possible to display a list of capture devices in system using FFME?
   FFplay example:
   ffplay -list_devices true -f dshow -i dummy
   ...and their modes of operation?
   ffplay -list_options true -f dshow -i video="Logitech HD Webcam C270"

[mariodivece]

please examine the ffmeg.c code and tell me where list_devices takes effect so I can call the FFmpeg API that way

[mariodivece]

Another way I have done this in the past is to use System.Diagnostics.Process to call ffmpeg.exe. You can also use the SWAN library: https://github.com/unosquare/swan#the-processrunner-class

[tenisak]

I confirm that the capture device works! Tested with the Osprey Card. It works exactly as in the above examples.

I would like to try to test with a Blackmagic Decklink card. These cards are much prone to precise parameter settings. (This can be seen, for example, from discussions on the Blackmagic forum.) However, the current FFmpeg compilations from ZZeranoe for Windows no longer support the decklink parameter.

Do you have archived ffmpeg version older than October 2016? In these builds support for DeckLink yet been compiled. If you have this old build in the archive, I can test it. (At this point I have no idea how complicated it is to make my own compilation ffmpeg and I guess it will not be that easy.)

[mariodivece]

Sorry I don't keep older versions of ffmpeg

[Mihaylov93]

@tenisak With this script you could compile your own FFmpeg to your needs (Thats what I did)
Basically it ask you which license you want, which libs and codecs (with yes or no like questions) and does everything for you, setting up the environment, downloading the dependencies and compiling it.

[tenisak]

@Mihaylov93 Thank you very much for the tip. There is a pile of parameters that he asks :-) I do not have much experience with it. Do you have a tip please which components are needed to compile?

[nanus]

@mariodivece Hello Mario, thanks for this great piece of software :) I'm currently trying to use a webcam and it works but FFMediaElement automatically selects the lower resolution.
I'm trying to hook to the MediaInitializing event as you suggested to another user but in the current version 4.0 the MediaInitializingRoutedEventArgs are totally different and this piece of code does not work anymore:

Media.MediaInitializing += (s, e) =>
{
e.Options.Input["framerate"] = "25";
e.Options.Input["video_size"] = "1920x1080";
e.Options.Input["pixel_format"] = "yuyv422";
};

How can I achieve the same results in the current release?

Thanks a lot.
Angelo Casalini

[nanus]

update:
I've found the answer looking at Sample project.

Media.MediaInitializing += (s, e) =>
{
e.Configuration.PrivateOptions["framerate"] = "30";
e.Configuration.PrivateOptions["video_size"] = "3840x2160";// "1920x1080";
};

Thanks and keep up the good project!
Angelo

[jswenwu]

Thanks for implementing this feature! Can you please advise how to do this ffplay example?

ffplay.exe -f dshow -i video="HP 720p HD Monitor Webcam":audio="Microphone (HP 720p HD Monitor Webcam)"

Thanks a lot.

[mariodivece]

As in multiple examples above:

Media.Source = new Uri("device://dshow/?video=\"HP 720p HD Monitor Webcam\":audio=\"Microphone (HP 720p HD Monitor Webcam)\"");

Recommended:

e.MediaOptions.IsTimeSyncDisabled = true;

[jswenwu]

thank you very much!
jswenwu

[jswenwu]

Sorry
in my case,
Media.Source = new Uri("device://dshow/?video=\"HP 720p HD Monitor Webcam\"");
it do not work.
Media.Source = new Uri("device://dshow/?video=HP 720p HD Monitor Webcam");
it is OK.
jswenwu

[mariodivece]

yeah... why would the first one work? I am surprised it's not giving you a compile-time error... Double quotes need to be escaped

[jswenwu]

yeah... why would the first one work?
it means :
Media.Source = new Uri("device://dshow/?video=\"HP 720p HD Monitor Webcam\"");
but it do not work.
please try it, Thanks a lot.
jswenwu

[jswenwu]

this web site auto hide "\".

[mariodivece]

I see the issue -- the Command Prompt is unescaping the double quotes as they are read as arguments.
Have you tried:

Media.Source = new Uri("device://dshow/?video=HP 720p HD Monitor Webcam:audio=Microphone (HP 720p HD Monitor Webcam)");

That should work fine.

[jswenwu]

Hi
Could you please tell me how can I get thumbnails or screenshots of video?
Thanks a lot.

[mariodivece]

There are plenty of examples of haw to do things in the sample project provided with the source code.

[jswenwu]

Thank you so much for so fast reply! Let's me have a look now.

[ssoreide]

Thanks, this is working really great! However I'm not able to select the vcodec via the PrivateOptions.

I have a dshow source that supports mjpeg encoding but I always get rawvideo even though I specify mjpeg like this:
e.Configuration.PrivateOptions["vcodec"] = "mjpeg";

All the other properties work great, like framerate and video_size. Is there special handling of the vcodec needed?

[mariodivece]

vcodec is not a private option
you need to handle the MediaOpening event and set the codec on the stream like this:

var streamIndex = e.Info.Streams.FirstOrDefault(s => s.Value.CodecType == AVMediaType.AVMEDIA_TYPE_VIDEO).Value.StreamIndex;
e.Options.DecoderCodec[streamIndex] = "mjpeg";

[ssoreide]

Thanks for the very quick reply! I tested your code, but I still get rawvideo as the codec of the incoming stream. The DecoderCodec is sent to ffmpeg as the codec main option, not vcodec main option, is that correct? I am able to get the correct codec when using ffmpeg on the command line with the vcodec main option set to mjpeg.

BTW, I can trick it to mjpeg through FFME as well, by setting the PrivateOptions for frame rate and resolution combination to a higher number than what can be delivered as uncompressed video (bandwidth wise) but this doesn't seem like a proper solution.

[mariodivece]

ffmpeg is a decoder/encoder. Could you provide the full commandline arguments for ffplay.exe instead please?

[ssoreide]

I tried with ffplay, but I can't make it choose the codec from the dshow source. I could only show this with ffmpeg command lines.

Here is the command I can use to get rawvideo from the source:
ffmpeg -f dshow -video_size 1280x720 -framerate 5 -i video="Logitech Webcam C930e" -y output.mp4

I then see following input stream info:

Input #0, dshow, from 'video=Logitech Webcam C930e':
  Duration: N/A, start: 266326.190000, bitrate: N/A
    Stream #0:0: Video: rawvideo (YUY2 / 0x32595559), yuyv422, 1280x720, 5 fps, 5 tbr, 10000k tbn, 10000k tbc

If I add the -vcodec mjpeg or -codec:v mjpeg to the command line like this:
ffmpeg -f dshow -video_size 1280x720 -framerate 5 -codec:v mjpeg -i video="Logitech Webcam C930e" -y output.mp4

then I get the expected result, using HW compression from the camera:

Input #0, dshow, from 'video=Logitech Webcam C930e':
  Duration: N/A, start: 266629.410000, bitrate: N/A
    Stream #0:0: Video: mjpeg (Baseline) (MJPG / 0x47504A4D), yuvj422p(pc, bt470bg/unknown/unknown), 1280x720, 5 fps, 5 tbr, 10000k tbn, 10000k tbc

It's the same result I'd like to achieve with FFME, so that I can record directly the compressed stream to file (using your TransportStreamRecorder class) at the same time as showing it in the WPF client. If the camera provides rawvideo then the TransportStreamRecorder will copy invalid codec parameters, and also I want to avoid having to compress the stream on CPU for recording to file.

Thanks again.

[mariodivece]

The problem is that FFME is based off ffplay -- not ffmpeg. FFmpeg does stuff somewhat differently as it is a decoder/encoder application. Without ffplay arguments it's impossible for me to provide help with a workaround.

[ssoreide]

Understood. I haven't been able to make the right ffplay command, unfortunately. I can make it interpret the incoming source as mjpeg but the actual stream is not affected and comes in as rawvideo. Thanks anyway for your support.

[mariodivece]

You could modify the TransportStreamRecorder and manually specify output parameters...
Something like: var codecParams = ffmpeg.avcodec_parameters_alloc(); and set the output parameters manually -- I don't know how simple or how complicated this would be.

[ssoreide]

Yes, I did experiment with this, and was able to record with my own output parameters. But it's costing too much resources to encode the raw frames. That's why I'm trying to make the dshow device deliver the frames already compressed and just store those to file.

[ghost]

Hi. I want to ask. Is it normal that for some reason when I use my webcam as output device the video is like 1,5 seconds delayed?

[mariodivece]

Hi. I want to ask. Is it normal that for some reason when I use my webcam as output device the video is like 1,5 seconds delayed?

Related to #420
Latency will be addressed

[qwerty13]

How to get name of available video (or audio) devices?

[LazyCuteLion]

I tried with ffplay, but I can't make it choose the codec from the dshow source. I could only show this with ffmpeg command lines.

Here is the command I can use to get rawvideo from the source: ffmpeg -f dshow -video_size 1280x720 -framerate 5 -i video="Logitech Webcam C930e" -y output.mp4

I then see following input stream info:

Input #0, dshow, from 'video=Logitech Webcam C930e':
  Duration: N/A, start: 266326.190000, bitrate: N/A
    Stream #0:0: Video: rawvideo (YUY2 / 0x32595559), yuyv422, 1280x720, 5 fps, 5 tbr, 10000k tbn, 10000k tbc

If I add the -vcodec mjpeg or -codec:v mjpeg to the command line like this: ffmpeg -f dshow -video_size 1280x720 -framerate 5 -codec:v mjpeg -i video="Logitech Webcam C930e" -y output.mp4

then I get the expected result, using HW compression from the camera:

Input #0, dshow, from 'video=Logitech Webcam C930e':
  Duration: N/A, start: 266629.410000, bitrate: N/A
    Stream #0:0: Video: mjpeg (Baseline) (MJPG / 0x47504A4D), yuvj422p(pc, bt470bg/unknown/unknown), 1280x720, 5 fps, 5 tbr, 10000k tbn, 10000k tbc

It's the same result I'd like to achieve with FFME, so that I can record directly the compressed stream to file (using your TransportStreamRecorder class) at the same time as showing it in the WPF client. If the camera provides rawvideo then the TransportStreamRecorder will copy invalid codec parameters, and also I want to avoid having to compress the stream on CPU for recording to file.

Thanks again.

I had the same problem,is there a solution? @ssoreide @mariodivece
there is ffplay command:
ffplay -f dshow -i video="USB Camera" -video_size 1920x1080 -framerate 30 -codec:v mjpeg
result:
Input #0, dshow, from 'video=USB Camera':= 0KB sq= 0B
Duration: N/A, start: 1097443.913775, bitrate: N/A
Stream #0:0: Video: mjpeg (Baseline) (MJPG / 0x47504A4D), yuvj422p(pc, bt470bg/bt709/unknown), 1920x1080, 30 fps, 30 tbr, 10000k tbn

if not set -codec:v,the result:
Input #0, dshow, from 'video=USB Camera':= 0KB sq= 0B
Duration: N/A, start: 1097587.392306, bitrate: N/A
Stream #0:0: Video: rawvideo (YUY2 / 0x32595559), yuyv422(tv, bt470bg/bt709/unknown), 1920x1080, 5 fps, 5 tbr, 10000k tbn