ustream stopped completely on April 19

[FredThompsonII]

none of the CreativeLive.com ustream feeds work for me, even with a virgin Windows installation for which the streams play properly with Internet Explorer. I tried yesterday's daily build then tried it with the changes from #1122 and #1227 but could not grab the streams.

Attached is the ustream.py file (extension changed to txt)

ustreamtv.txt

At some point on the afternoon of April 19 livestreamer was displaying mismatched/missing arguments in the ustream plugin. I did NOT capture any of them.

Can other people verify this problem?

Does anyone have an idea how to fix this?

Here are the stream URLs:

live1 https://www.ustream.tv/embed/4307895
live2 https://www.ustream.tv/embed/11655601
live3 https://www.ustream.tv/embed/12952587
live4 https://www.ustream.tv/embed/12952592
live5 https://www.ustream.tv/embed/15751699

watch1 https://www.ustream.tv/embed/952665
watch2 https://www.ustream.tv/embed/12858195
watch3 https://www.ustream.tv/embed/12858204
watch4 https://www.ustream.tv/embed/3696988
watch5 https://www.ustream.tv/embed/14313521

watch6 https://www.ustream.tv/embed/14313537
watch7 https://www.ustream.tv/embed/16551254
watch8 https://www.ustream.tv/embed/16551260
watch9 https://www.ustream.tv/embed/16551290
watch10 https://www.ustream.tv/embed/16551294

watch11 https://www.ustream.tv/embed/16561985
watch12 https://www.ustream.tv/embed/16608869
watch13 https://www.ustream.tv/embed/20712998
watch14 https://www.ustream.tv/embed/20712999
watch15 https://www.ustream.tv/embed/20713000

[L2501]

works for me using latest git + #1197 & #1278

[FredThompsonII]

Well, I'm stumped. Grabbed the 2016-02-02 build of ustream.py from http://livestreamer-builds.s3.amazonaws.com/livestreamer-latest-win32.zip then rolled 1197 and 1278 into it. Still no display of anything working properly. ustream.py is attached. Would you (or someone else) please reply with a working ustream.py?
ustreamtv.txt

[FredThompsonII]

tried the 2015-05-02 version with same results (probably same source, I didn't compare, just noted file dates...)

[RuKeBo]

How do you compile the ".py" file. I used "python -m py_compile ustreamtv.py" and it compiles to a "pyc" but it doesn't work correctly.

[scottbernstein-zz]

Are you using Python 2.7 for compiling (as opposed to Python 3.x)? 2.7.x is the required version.

Date: Wed, 27 Apr 2016 11:09:27 -0700
From: [email protected]
To: [email protected]
CC:
Subject: Re: [chrippa/livestreamer] ustream stopped completely on April 19 (#1295)

How do you compile the ".py" file. I used "python -m py_compile ustreamtv.py" and it compiles to a "pyc" but it doesn't work correctly.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub

[RuKeBo]

I was originally using 2.6, then installed 3.x, didn't work with either. Just tried with 2.7 and does the same thing. I am just trying to compile the ustreamtv.py to ustreamtv.pyc.

Output from livestreamer when using compiled ustreamtv.py plugin:
Traceback (most recent call last):
File "", line 6, in
File "main.py", line 128, in
File "main__livestreamer.py", line 9, in
File "livestreamer_cli\main.py", line 886, in main
File "livestreamer_cli\main.py", line 469, in handle_url
File "livestreamer_cli\main.py", line 395, in fetch_streams_infinite
File "livestreamer_cli\main.py", line 388, in fetch_streams
File "livestreamer\plugin\plugin.py", line 313, in get_streams
File "livestreamer\plugin\plugin.py", line 233, in streams
File "ustreamtv.py", line 550, in _get_live_streams
File "ustreamtv.py", line 513, in _get_desktop_streams
KeyError: 'url'

[FredThompsonII]

I've never compiled. Each time I modified the .py file, the results changed. The changes L2501 suggested worked for a few days then didn't. I've tried various combinations of 1122 1197 and 1227 and keep getting failures. Some of those are reported as being indentation issues but nothing looks out of place.

Is there a significant speed advantage to compiling?

Even so, if it pukes in run-time from source, it would puke from compilation...if it even got that far...

is ustream working for either of you?

I attached my ustreamtv.py above with the extension changed to txt. Do you see any errors?

[RuKeBo]

I was assuming it needed to be complied since all the plugins in the plugins folder are complied. But now that I look at it, the working one I have is not complied, go figure. It is working for me but not perfect. None of the ones posted here work for me. Not sure where I got the one I am using, I keep backing them up in case a new version messes it up. Here is a working one.
ustreamtv.txt

[snifer1981]

Maye be #1171 (comment) works for you.

[RuKeBo]

This seems to work a little better but it is doing some strange things. Basically, the one I had worked fine except that it, for the most part, cut off the end of the stream, consistently. It also showed some errors but the stream did not skip, offset the audio, etc. The one you have does not seem to cut the end off but is is dropping some information in between the middle at times and on one file the audio seemed off at the end. Funny thing is, I can save the same stream and one file will be larger than the other, I assumed the larger one contained more information, but it skips or drops out in places. Still testing.

[FredThompsonII]

The vast majority of stream are starting properly and audio is synced. If I save to a file then remux to MP4 using ffmpeg without any transcoding, sync stays proper.

I have seen quite a few streams where the video disappears but the audio remains. This leads me to suspect livestreamer is prematurely concatenating the last GOP. Maybe there's a little typo or oversight somewhere which causes a buffer to be flushed as soon as the end-of-stream condition is detected.

My hunch is streams which stop early (and they almost always do) have incredibly long GOPs or forward-referencing frames which don't have the GOP from which they are to draw.

I tried adding these to my CLI calls but they didn't fix the issue:

--stream-timeout 255
--http-timeout 255
--http-stream-timeout 255

Now I have another idea; keep reading regardless of being told the source stream has stopped. Basically, make a large file which continues until the livestreamer instance is cancelled.

Segments can always be created by cutting later.

Attached are 3 variations of ustreamtv.txt; the one which I am using that truncates the end of files, one that didn't work for me, and one that worked then stopped working which lead to the OP.

ustreamtv.txt
ustreamtv - bad.txt
ustreamtv - old.txt

[FredThompsonII]

Another idea; what if rtmpdump is the culprit? I'm testing rtmpdump 2.4 with KSV 2015-12-15 customizations

https://github.com/K-S-V/Scripts/releases

[FredThompsonII]

Changing rtmpdump didn't help.

Neither did adding these: --stream-timeout 255 --http-timeout 255 --http-stream-timeout 255 --player-no-close --ringbuffer-size 128M

Viewed with a web browser, there is no lost data.

livestreamer is dropping multiple GOPs at the end of a segment.

I wonder if the problem is in livestreamer's file writing routine...

[RuKeBo]

My plugin seems to work fine most of the time except for the one thing you mention, it truncates the end of the stream. I do get the audio out of sync on occasion, but the truncate issue is every time. I have no idea how this works, but if you turn debug on you can see it downloads chunks. I have wondered, as you have, if it is not writing that last chunk to the file. Like maybe it sees the end of the stream before it writes the last part of the file. Again, I have no idea how it works, just speculating.

It appears to add the chunk(s) to the queue, then write them. You would think it would have to write to the file before it gets to the code to exit.

[L2501]

it should be possible to rewite the ustream module to use their http api instead of the rtmp, if you block port 1935 you can see all the chunk infos can be got over the http.

[L2501]

yes and python-librtmp doesn not work with KSV patched librtmp, you need the vanilla one.

[FredThompsonII]

The more I look at this, the more it looks like the file writer is being instructed to close early. chrippa alluded to that in issue 265 #265

Is that in the ustreamtv.py file? Seems most likely.

[RuKeBo]

I tried to do some testing on this last night, very rudimentary. I used livestreamer to capture a stream to a file, to VLC, and I viewed the stream online. I know livestreamer cut the file stream short compared to when the ustream stream ended but I thought that it also ended the VLC stream short, which I would think does not involve writing to a file. I will try and test this more tonight just to be sure, not sure if means anything or not.

[FredThompsonII]

This stream is running WITHOUT the end-of-segment portion today, Aug 14:

https://www.ustream.tv/embed/14313537