Skip to content
This repository has been archived by the owner on Feb 10, 2023. It is now read-only.

Widevine DRM enabled for streams #233

Open
maturd opened this issue Apr 8, 2022 · 69 comments
Open

Widevine DRM enabled for streams #233

maturd opened this issue Apr 8, 2022 · 69 comments
Assignees
Labels
bug Something isn't working

Comments

@maturd
Copy link

maturd commented Apr 8, 2022

Please check the FAQ and search existing issues before you submit a new one!

Describe the bug
Whatever live feed I try to open (MPV, IINA, VLC) I only get a grey screen with no audio. If I try to copy to clipboard and open the link in Safari I don't get a video there too but it download a .mpd file that is unplayable.

To Reproduce
Steps to reproduce the behavior:

  1. Select any live feed 2. Try to open it with any player or copy the link to open it in a browser

Expected behaviour
Play feed

Screenshots
If applicable, add screenshots to help explain your problem.

F1TV account plan
Pro Account. No VPN

Desktop (please complete the following information):

  • macOS
  • Brew
  • 2.7.0

Logs
If applicable please provide the relevant portion of your logs. You can find them by running f1viewer -logs.

@maturd maturd added the bug Something isn't working label Apr 8, 2022
@nicholascw
Copy link

Same thing here. Gray screen, flood of ffmpeg errors together with some 404 error from F1viewer itself when playing the Australian GP Practice 1 LIVE.

@pizza461
Copy link

pizza461 commented Apr 8, 2022

Yes, unfortunately this seems to be a problem with DRM, which they have started implementing. That really sucks, because there is no fix.

Better explained by the creator of Race Control (which also suffers from this problem):

robvdpol/RaceControl#413

@crater9893
Copy link

What a shame, I had my own build that I could watch from my Ipad , the only reason is use it is because the f1 player is so terrible and using f1viewer proxy ensures I always get 4k. Now the mpd file doesnt open in safari and on VLC / MPV just a grey screen with garbled vertical lines.

@crater9893
Copy link

To whom it may concern, **** you!
Seriously I know F1 is reading this, at least before you disable this app make the IOS app workable , its so terrible I will be happy if you get rid of the adaptive quality that never works even on my 10Gig connection. When I use custom players I can watch in 4k without a hitch.

@pizza461
Copy link

pizza461 commented Apr 8, 2022

1080p50 is the highest quality, my friend.

But I agree, we need the 3rd party players! This is so not right xD

@DoeEensGek
Copy link

https://www.bento4.com/documentation/mp4decrypt/
From the index.mpd ContentProtection schemeIdUri="urn:mpeg:dash:mp4protection:2011" value="cenc" cenc:default_KID="1*****5-2**6-4***4-B**7-4*********7" xmlns:cenc="urn:mpeg:cenc:2013"

Don't have much time, but think there are still some options. But they make it harder.

@enginefeeder101
Copy link

Same here. MPD files with no audio and 'gray' video....

@SoMuchForSubtlety SoMuchForSubtlety changed the title [BUG] No content loading up [BUG] Widevine DRM enabled for streams Apr 8, 2022
@SoMuchForSubtlety SoMuchForSubtlety changed the title [BUG] Widevine DRM enabled for streams Widevine DRM enabled for streams Apr 8, 2022
@arpiecodes
Copy link

arpiecodes commented Apr 8, 2022

https://www.bento4.com/documentation/mp4decrypt/ From the index.mpd ContentProtection schemeIdUri="urn:mpeg:dash:mp4protection:2011" value="cenc" cenc:default_KID="1*****5-2**6-4***4-B**7-4*********7" xmlns:cenc="urn:mpeg:cenc:2013"

Don't have much time, but think there are still some options. But they make it harder.

Sorry to say but this method still requires a way to actually get the master content key from Widevine DRM to make it work.

Basically, the media is encrypted using a static key. This key is processed by Widevine CDM (content decryption module) on the client device by talking with the Widevine server or doing some secret key stuff (this part is typically heavily obfuscated so difficult to reverse engineer) and actually performs the AES-128 CTR decryption on the content so the master key does not leak.

If you can get the content's master key (there used to be ways, Google it - Widevine had a huge 'security issue' not too long ago which allowed extracting the master keys by chrome extension (!), but now it's very hard again) you can decrypt the content without doing the Widevine stuff (its actually built-in for some players, including ffmpeg). That's also how those 'illegal streaming websites' keep on popping up using the actual CDN of the streaming service itself. But then again; this assumes those master content keys are not random and they never change..

Alternatively, instead of trying to find the master keys skipping Widevine completely, you may be able to implement some version of the CDM library like https://gist.github.com/ruario/3c873d43eb20553d5014bd4d29fe37f1 and then make a wrapper that calls the library to use the decryption part, the same way they do it for Kodi; https://github.com/emilsvennesson/script.module.inputstreamhelper/blob/master/lib/inputstreamhelper/widevine/widevine.py. But I wouldn't get my hopes up. It will be painstakingly hard.

@pvanb
Copy link

pvanb commented Apr 8, 2022

But I agree, we need the 3rd party players! This is so not right xD

I see what you did there 💥🎧💥

@notarobot1337
Copy link

Let's say I had the keys (completely possible and it's one WV master key), anyone aware of any players for Mac that support playback with key? Something like the adaptive.license plugin for Kodi?

@CastorSmith89
Copy link

Is this a global issue or a european issue due to DSGVO? Just observed that If I start a screenshot tool like greenshot the stream in the webbrowser of F1TV get black to prevent screenshots from the screen. I expect this is something regarding Copyright (Urheberrecht) which is huge changed in DSGVO.

@nicholascw nicholascw mentioned this issue Apr 9, 2022
@arpiecodes
Copy link

arpiecodes commented Apr 9, 2022

Let's say I had the keys (completely possible and it's one WV master key), anyone aware of any players for Mac that support playback with key? Something like the adaptive.license plugin for Kodi?

If you have the master keys, you should use a player that supports CENC decrypting (common encryption, AES-128 CTR) with the key. Alternatively, you could use ffplay/ffmpeg to open the stream, then pipe the audio/video output to another player of your choice (see https://stackoverflow.com/questions/44939166/ffmpeg-how-to-produce-mp4-cenc-common-encryption-videos/44949160#44949160 -decryption_key argument).

You can also let ffmpeg write the stream into an UDP Multicast group, so you can also access it from other players in your network. Or to an RTMP stream for example.

@notarobot1337
Copy link

notarobot1337 commented Apr 9, 2022

https://stackoverflow.com/questions/44939166/ffmpeg-how-to-produce-mp4-cenc-common-encryption-videos/44949160#44949160

Unfortunately not that easy as it falls over once it starts pulling the mp4 segments (HTTP Error 400, Bad Request), I'm guessing it's missing some header information, I'll have a play about with it further and report back

@arpiecodes
Copy link

arpiecodes commented Apr 9, 2022

Unfortunately not that easy as it falls over once it starts pulling the mp4 segments (HTTP Error 401), I'm guessing it's missing some header information, I'll have a play about with it further and report back

IIRC streams gotten through f1viewer might give you a 401 if the kid GET argument is missing (it seems the value does not really matter, even empty works in my testing).

Might be easier to copy a link from a browser session into ffmpeg and see if that works? Or changing the user-agent from ffmpeg to something less trivial to block. :-)

EDIT: Nevermind above. I see what you mean now. DASH streaming is protected by setting a cookie from the manifest request forward. I guess you need cookies support in ffmpeg (which does not exist), or a proxy that adds the cookies for you. You could also try to put the cookies in manually by looking at the cookies set for your browser session.

@notarobot1337
Copy link

Ok with one massive line into FFplay with all the headers it attempts to play but I'm getting 'Option decryption_key not found', perhaps I need a different version of FFplay, again I'll report back.

@arpiecodes
Copy link

Ok with one massive line into FFplay with all the headers it attempts to play but I'm getting 'Option decryption_key not found', perhaps I need a different version of FFplay, again I'll report back.

Guess you need the module 'crypto' built in. Which might also require building from source. ffmpeg -protocols should tell you if your version has the crypto protocol built-in.

@notarobot1337
Copy link

it has the crypto module, I'll build a new version

@notarobot1337
Copy link

sooo close

So FFmpeg starts doing its thing, then spits out a ton of these...

Stream mapping:
Stream #0:5 -> #0:0 (copy)
Stream #0:6 -> #0:1 (copy)
Press [q] to stop, [?] for help
[dash @ 0x7fb417004280] No longer receiving stream_index 02 bitrate=347523.4kbits/s speed=0.0237x
[dash @ 0x7fb417004280] No longer receiving stream_index 1
[dash @ 0x7fb417004280] No longer receiving stream_index 2
[dash @ 0x7fb417004280] No longer receiving stream_index 3
[dash @ 0x7fb417004280] No longer receiving stream_index 4
[NULL @ 0x7fb41704fc80] illegal reordering_of_pic_nums_idc 17
[NULL @ 0x7fb41704fc80] non-existing PPS 1 referenced
[NULL @ 0x7fb41704fc80] non-existing PPS 2 referenced
[NULL @ 0x7fb41704fc80] non-existing PPS 1 referenced
[NULL @ 0x7fb41704fc80] non-existing PPS 42 referenced
[NULL @ 0x7fb41704fc80] non-existing PPS 2 referenced
[NULL @ 0x7fb41704fc80] non-existing PPS 7 referenced
[NULL @ 0x7fb41704fc80] non-existing PPS 2 referenced
[NULL @ 0x7fb41704fc80] non-existing PPS 1 referenced
[NULL @ 0x7fb41704fc80] non-existing PPS 2 referenced

@arpiecodes
Copy link

arpiecodes commented Apr 9, 2022

Those warnings could also well be non-fatal. For example; sometimes in the stream audio tracks can be added or removed dynamically, depending on the broadcast. In such case, track indexes might change.

Does it actually produce a playable video/audio output if you try to write it to a file?

Maybe you can try -ss 0 to have it start at the beginning of the broadcast/stream?

@notarobot1337
Copy link

Those warnings could also well be non-fatal. For example; sometimes in the stream audio tracks can be added or removed dynamically, depending on the broadcast. In such case, track indexes might change.

Does it actually produce a playable video/audio output if you try to write it to a file?

Maybe you can try -ss 0 to have it start at the beginning of the broadcast/stream?

Excellent explanation, I sent the output to a local rtmp server so I could watch the stream as it came in, unfortunately it just gave a green screen however the bitrate, resolution and fps were all correct, which leads me to believe the code is correct except for how it's handling the WVkey, so next I'll dump it to a .mp4 and see if mp4decrypt can process it.
I'm 99% certain the key is correct as it matches up to the KID etc and I've obtained keys using the same method many times before.

@arpiecodes
Copy link

arpiecodes commented Apr 9, 2022

For some weird reason, the HLS variant (if you use Safari as playback client) seems to be unencrypted again now for Qualifying Australia..

I also noticed it uses the same kid. So if DASH does not work, it might be possible to use the same key for the HLS variant as they are usually encrypted using the same content key within the same origin.

EDIT: Nevermind, this is probably just the content moving from 'restart' (e.g. recording of the live stream) to 'video-on-demand' after processing. However I noticed kid 1042, which might be interesting to try with a future live stream. Though that probably won't really work.

Above comment about trying HLS if the kid matches does still stand, though.

@notarobot1337
Copy link

notarobot1337 commented Apr 9, 2022

the WV kid is what I'm referring to from within the mpd, from what I've tested so far they seem to use 1 kid, pssh and master key for all streams and it changes daily.
Am I right in thinking the HLS url is the same but with m3u8 extension?

EDIT: Yeah the m3u8's play any, they're not using DRM encryption.

@arpiecodes
Copy link

Usually, there is one master content key (the same origin software is usually responsible for encrypting and packaging HLS/DASH variants of the same source).

The DRM system is actually only in charge of giving you the correct content decryption key, it does not really do more than that (but in reality it has to because it cannot risk to leak the master key, so hardware/software implementations usually also take care of decryptingthe media and securely playing back the video - which is why you cannot screenshot/record the framebuffer output as it's virtually not there and using a 'secure' path to the graphics card).

Thanks to CENC (=Common Encryption) standard almost all streaming formats could in theory use the same encryption method. Usually, it is quite common that content is using the same master key across all streaming formats. But, these are all assumptions of course.

Am I right in thinking the HLS url is the same but with m3u8 extension?

Also unable to check right now as the platform does not seem to deliver the URL's for the actual 'live' stream variants anymore, just the replay versions, which makes comparing them hard..

EDIT: Yeah the m3u8's play any, they're not using DRM encryption.

I guess when they 'archive' something as 'replay' content, they are using another way of serving the content. In Chrome, it also gives you a HLS type stream. This content hasn't got any DRM on it indeed.

Though, when a broadcast is 'live', they seem to use Fairplay on the HLS stream. Which is Apple's Widevine responsible of giving the client the key. Which might coincidentally be the same CENC key you extracted from the DASH WV stream.

@arpiecodes
Copy link

The real problem however is that sharing the master content key is actually considered circumventing 'effective' DRM which is illegal to do in most countries. Having said that, kinda in it to see if you can get it working. Which would be awesome. But still, not really practically usable for a typical f1viewer user.

@notarobot1337
Copy link

By the way, yesterday's token was no longer valid today, it seems that it also changes randomly.

They expire every 24 hours, that part is controlled via HMAC

@notarobot1337
Copy link

That's exactly what I meant about implementation, legality must be taken into account, but we could get around this by introducing an additional field where we could enter the key, each of us in our own way.

First things first lets get it to stream live, if ffmpeg doesn't support 'on the fly' decryption then we could find something that does, or the really long method would be to script it to download segments, decrypt then then create a new stream with the decrypted segments.

@crisgsm33
Copy link

By the way, yesterday's token was no longer valid today, it seems that it also changes randomly.

They expire every 24 hours, that part is controlled via HMAC

In my case it was unchanged from Saturday until today.

@notarobot1337
Copy link

By the way, yesterday's token was no longer valid today, it seems that it also changes randomly.

They expire every 24 hours, that part is controlled via HMAC

In my case it was unchanged from Saturday until today.

Not sure on that then, from experience it's been 24hours for me

@crisgsm33
Copy link

This is the smallest problem, I would say, Bento4-SDK needs to be integrated into the script, the order of events being download, decrypt, play...

@crisgsm33
Copy link

The script could also extract the token automatically, without our intervention, using the Firefox library.

@notarobot1337
Copy link

This is the smallest problem, I would say, Bento4-SDK needs to be integrated into the script, the order of events being download, decrypt, play...

Yes, but that still requires knowing how the mpd segments fall, as this seems different in archive to race day

@crisgsm33
Copy link

The information is in the header, only the decryption key can decipher the order and size of the chunks, which is between 2 and 4 sec for .mpd files.
Also, .mpd files do not have a codec, it is set by FFmpeg after the script, so it does not work with a simple video and audio copy.

@crisgsm33
Copy link

There are some dowloader software, like youtube-dl, that works with command prompt, that would know how to download .mpd in chunks, I don't know how much and how it would work live.

@maturd
Copy link
Author

maturd commented Apr 12, 2022

Is tvOS (Apple TV) working .mpd files differently (and in an easier way)? VroomTV (F1AppleTV) stopped to work after the token issue first, and re-introduction of DRM after, but @NoahFetz worked out a solution in such short time. Issue 55

@NoahFetz
Copy link

Is tvOS (Apple TV) working .mpd files differently (and in an easier way)? VroomTV (F1AppleTV) stopped to work after the token issue first, and re-introduction of DRM after, but @NoahFetz worked out a solution in such short time. Issue 55

All Apple clients (Safari Browser, iPhone, AppleTV etc.) get m3u8 encrypted with Apple's DRM called FairPlay. As far as I know this only works on Apple devices.

@anthonylavado
Copy link

Noah is correct. F1TV puts out two DRM streams. One is Widevine, the other is FairPlay. Widevine is used for most clients (web, Android, Android TV, Fire TV devices), and FairPlay is used exclusively for Apple devices/software (Safari, iOS/iPadOS/tvOS app).

On the Apple side, it's a little bit easier to deal with since there are system modules you can call to handle it.

@maturd
Copy link
Author

maturd commented Apr 19, 2022

Noah is correct. F1TV puts out two DRM streams. One is Widevine, the other is FairPlay. Widevine is used for most clients (web, Android, Android TV, Fire TV devices), and FairPlay is used exclusively for Apple devices/software (Safari, iOS/iPadOS/tvOS app).

On the Apple side, it's a little bit easier to deal with since there are system modules you can call to handle it.

So right now, it could be possible to let f1viewer users on Mac to use this different DRM stream easily? Could be a possible scenario @SoMuchForSubtlety ? Or is a difficult workaround to introduce just for Mac users?

@anthonylavado
Copy link

@maturd Not to respond for the maintainer but, when I say "system modules", I'm referring to an app written in Apple's own Objective-C or Swift languages. VROOM TV is a native app on Apple TV, that's why it can do that. It's not as easy as enabling something for a Mac build. F1Viewer just launches playback though - it's not trying to be the player. It's not the stream URL retrieval that needs DRM support, it's the player.

I wonder if setting the useragent to be the iOS F1TV app will provide a FairPlay stream that could be viewed in QuickTime though.

@howlett
Copy link

howlett commented Apr 19, 2022

Is there a branch to test for this weekend?

@PostMalone
Copy link

Is there a branch to test for this weekend?

Probably not. DRM is probably extremely hard to bypass

@notarobot1337
Copy link

Is there a branch to test for this weekend?

Probably not. DRM is probably extremely hard to bypass

DRM isn't hard to bypass (WV at least isn't), getting ffmpeg to playback encrypted with the key is where I'm falling down, but I know it will work if someone with FFmpeg knowledge has some time to take a look.

I was able to download an existing mpd, decrypt it, then play it back with no issues.

@SamPro910
Copy link

@notarobot1337 that means good things, no? Not an expert, but I mean we're at least 10% there, surely?

@Psidium
Copy link

Psidium commented Apr 25, 2022

Can we can use mpv to view it with --demux-lavf-o=decryption_key=<key>? (--mpv-demux-lavf-o=decryption_key=<key> on iina) mpv-player/mpv#8286

@notarobot1337
Copy link

mpv-player/mpv#8286

Not testing, there's been some progress made elsewhere though, perhaps the author could get in touch.

If you wanna test this with key msg me on TG

@Vallesuski
Copy link

mpv-player/mpv#8286

Not testing, there's been some progress made elsewhere though, perhaps the author could get in touch.

If you wanna test this with key msg me on TG

This is good news. You should provide more details so author can review it.

@notarobot1337
Copy link

Alternatively, you could use ffplay/ffmpeg to open the stream, then pipe the audio/video output to another player of your choice (see https://stackoverflow.com/questions/44939166/ffmpeg-how-to-produce-mp4-cenc-common-encryption-videos/44949160#44949160 -decryption_key argument).

@synegic please don't suggest this option. From my research, no one has been able to successfully use FFmpeg to decrypt any Widevine, even with a valid key. If you have done it, please post proof.

He’s completely right for this suggestion as it does work with the correct set up.

As for posting proof on a public GitHub… I don’t see that as a good idea….

@f1multiviewer
Copy link

disclosure: I'm the developer of MultiViewer for F1, hope this is ok to post here

Since other 3rd-party apps were rendered useless for live broadcasts, I started working on my own application, which uses Electron (with content decryption module, from CastLabs). It uses the same player as F1TV, but it uses a different configuration that leads to a better sync, less stalling/buffering, and a more pleasant viewing experience overall. I was also able to add Live Timing overlays (on live sessions for now) that can be synced with the driver on-boards to show live telemetry on top of the OBC player. Still working on a better sync during replays and to allow live-timing to work on replays.

I hope to open-source it at some point, but it's already available to try (and many have tried last race):

https://beta.f1mv.com/
https://www.reddit.com/r/F1TV/comments/v35muj/sneak_peak_of_f1_multiviewer_a_desktop_app/
https://www.reddit.com/r/F1MultiViewer/

I'm also on Discord (invite on the beta site), to help other 3rd-party apps if they are using the same (web-based) approach.

@Vallesuski
Copy link

disclosure: I'm the developer of MultiViewer for F1, hope this is ok to post here

Since other 3rd-party apps were rendered useless for live broadcasts, I started working on my own application, which uses Electron (with content decryption module, from CastLabs). It uses the same player as F1TV, but it uses a different configuration that leads to a better sync, less stalling/buffering, and a more pleasant viewing experience overall. I was also able to add Live Timing overlays (on live sessions for now) that can be synced with the driver on-boards to show live telemetry on top of the OBC player. Still working on a better sync during replays and to allow live-timing to work on replays.

I hope to open-source it at some point, but it's already available to try (and many have tried last race):

https://beta.f1mv.com/ https://www.reddit.com/r/F1TV/comments/v35muj/sneak_peak_of_f1_multiviewer_a_desktop_app/ https://www.reddit.com/r/F1MultiViewer/

I'm also on Discord (invite on the beta site), to help other 3rd-party apps if they are using the same (web-based) approach.

Are you able to download the video right after live session ends with no DRM?

@f1multiviewer
Copy link

disclosure: I'm the developer of MultiViewer for F1, hope this is ok to post here
Since other 3rd-party apps were rendered useless for live broadcasts, I started working on my own application, which uses Electron (with content decryption module, from CastLabs). It uses the same player as F1TV, but it uses a different configuration that leads to a better sync, less stalling/buffering, and a more pleasant viewing experience overall. I was also able to add Live Timing overlays (on live sessions for now) that can be synced with the driver on-boards to show live telemetry on top of the OBC player. Still working on a better sync during replays and to allow live-timing to work on replays.
I hope to open-source it at some point, but it's already available to try (and many have tried last race):
https://beta.f1mv.com/ https://www.reddit.com/r/F1TV/comments/v35muj/sneak_peak_of_f1_multiviewer_a_desktop_app/ https://www.reddit.com/r/F1MultiViewer/
I'm also on Discord (invite on the beta site), to help other 3rd-party apps if they are using the same (web-based) approach.

Are you able to download the video right after live session ends with no DRM?

No, MultiViewer is meant to be a viewer only, not to aid downloading, for this, RaceControl and F1Viewer are still both great options :)

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests