Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for BBB with authentification #41

Closed
dmrbatuhan opened this issue Apr 3, 2023 · 28 comments
Closed

Add support for BBB with authentification #41

dmrbatuhan opened this issue Apr 3, 2023 · 28 comments

Comments

@dmrbatuhan
Copy link

Hello, I could download the videos with bbb-dl a week ago. But today, it gives a error like this:
bbb-dl https://scalelite.sdu2bbb.online/playback/presentation/2.3/
f7932ffc29b496194e314fab60b388ca8978a87e-1678428387252
Output directory for the final video is: /root
Directory for the temporary files is: /root/.local/share/bbb-dl/383011d326cfa600
3148de9fd405d1c4
Downloading meta information
Downloading metadata.xml...
Downloading shapes.svg...
shapes.svg could not be downloaded: 404 Not Found
metadata.xml could not be downloaded: 404 Not Found
Error: metadata.xml is essential. Abort! Please try again later!
Can you help me with this situation?

@C0D3D3V
Copy link
Owner

C0D3D3V commented Apr 3, 2023

Your playback url seems to be broken

@C0D3D3V
Copy link
Owner

C0D3D3V commented Apr 3, 2023

At least for me the URL returns 404

@C0D3D3V
Copy link
Owner

C0D3D3V commented Apr 5, 2023

Please open a new issue if this error persists with a correct URL

@C0D3D3V C0D3D3V closed this as completed Apr 5, 2023
@dmrbatuhan
Copy link
Author

Hello, as I mentioned before, the link is not broken. It is giving a metadata error. However, today I discovered something interesting. When I try to download an old recorded lesson using the link, it works without any problems, but it doesn't work with the new links. You will find the links below. These links are obtained from the live lesson recordings section in my university's Learning Management System.
Working link (March 6th, 2023)
https://scalelite.sdu2bbb.online/playback/presentation/2.3/7cb787fa0c008c41753dbab180f89208cf2eac80-1678081914350
Link that gives a metadata error (April 3rd, 2023)
https://scalelite.sdu2bbb.online/playback/presentation/2.3/7cb787fa0c008c41753dbab180f89208cf2eac80-1680501512019
Thanks in advance.

@C0D3D3V
Copy link
Owner

C0D3D3V commented Apr 5, 2023

Are you kidding me? Make a screenshot of the second link to show me it does not give an error for you.

@C0D3D3V
Copy link
Owner

C0D3D3V commented Apr 5, 2023

My guess is, that second session was not recorded, thats why the link is broken.

@dmrbatuhan
Copy link
Author

If I couldn't watch the link that gives a metadata error on the bbb-dl through the university website, I would have thought the same as you. I am trying to take a screenshot, but as I am visually impaired, I may not be able to provide a very clear screenshot.

@C0D3D3V
Copy link
Owner

C0D3D3V commented Apr 5, 2023

Do you have to login on that website? You can also send the screenahot via mail

@C0D3D3V C0D3D3V reopened this Apr 5, 2023
@dmrbatuhan
Copy link
Author

Yes, I log in with my university email address and password, and then I click on this link. I hope I got it correctly. I'm blind.
scalelite sdu2bbb online_playback_presentation_2 3_7cb787fa0c008c41753dbab180f89208cf2eac80-1680501512019(Samsung Galaxy A51_71)

@C0D3D3V
Copy link
Owner

C0D3D3V commented Apr 5, 2023

I guess you can only view the URL when you have loged in. Can you send me the website where you log in? Is it moodle or directly on Big Blue Button? If it is moodle maybe it is some sort of LTI module for authentification. In any case... I can add an option that allows you to load cookies. So you only have to save your browser cookies to file and bbb-dl will load them.

@C0D3D3V C0D3D3V changed the title Metadata Problem Add support for BBB with authentification Apr 5, 2023
@dmrbatuhan
Copy link
Author

Please accept my apologies for the delayed response. As you may recall, I had previously sent a screenshot of the relevant window. To access the university's learning management system, users can use the login link provided at https://oys.sdu.edu.tr, where they will be prompted to enter their university email address and password. Upon successful login, the user will be directed to a control panel screen displaying a list of courses. By selecting any of the listed courses, users can access a link that redirects them to the Big Blue Button platform located under the "Live Classes and Recordings" section. Clicking on the link will launch a window that is similar to the one shown in the screenshot I provided earlier. The class can then be initiated by pressing the "Play Video" button.

@C0D3D3V
Copy link
Owner

C0D3D3V commented Apr 7, 2023

The link you provided is a moodle instance. As I guessed correctly ^^ Very interesting, maybe I could even integrate a mechanic to send the cookies from moodle-dl to bbb-dl, or since moodle-dl will use yt-dlp for LTI Modules I would need to send the cookies from yt-dlp to bbb-dl...
But that is something I will maybe do someday in future. For now I will keep it very simple and just allow you to login via a simple cookie file or a simple browser window...
I do not think I will have time over eastern for this, but next week probably...

@C0D3D3V
Copy link
Owner

C0D3D3V commented Apr 13, 2023

Please update and read the instructions: https://github.com/C0D3D3V/bbb-dl/releases/tag/v1.0.3

Please report if it works!

@C0D3D3V
Copy link
Owner

C0D3D3V commented Apr 18, 2023

Have you tried it?

@dmrbatuhan
Copy link
Author

I had a busy week, so I could only try now. Unfortunately, the result is still the same. First, I logged in to the browser window that appeared using the bbb-dl-browser command on Windows, but the result was negative. Then, I used a Google Chrome extension to save the cookies to a file named cookies.txt and copied it to the specified default folder. I even tried the cookies.txt file on Linux, but I still got the same error: "Downloading meta information
Downloading metadata.xml...
metadata.xml could not be downloaded: 404 Not Found
Error: metadata.xml is essential. Abort! Please try again later!"

@C0D3D3V
Copy link
Owner

C0D3D3V commented Apr 18, 2023

Have you opened the BBB session in the bbb-dl-browser? So that you could watch the recording in the browser? And after that closed the browser and tried to download the URL that was shown in bbb-dl-browser?

@C0D3D3V
Copy link
Owner

C0D3D3V commented Apr 18, 2023

Mh I can add a button to the browser to directly start the download of a detected session. If that does not help, we can have a discord call and maybe then I see what is going wrong...

@Patrick92bs
Copy link

I have the same problem. How does it work exectly with the download of the BBB lesson is behind login? I need to download some lessons from my university 😅

@C0D3D3V
Copy link
Owner

C0D3D3V commented Apr 29, 2023

I have currently no way to test it. Since I have no such login page.

But the idea is as follow:
run bbb-dl-bowser. Login and open your BBB-Session for that you need login. Copy the URL that is displayed inside bbb-dl-browser. Close bbb-dl-browser. Run bbb-dl with the copied URL.

@yilmazali32
Copy link

@C0D3D3V
Copy link
Owner

C0D3D3V commented Jun 23, 2023

If one of you would give me your credentials I could test the download using login and fix the bug. You can contact me via email (on my profile) or discord (c0d3d3v)

@varuneranki
Copy link

I would like to add some more details as i face similar issue. it might not be authentication issue. It connects to the server and then does nothing.
Screenshot from 2024-07-01 10-15-36
Screenshot from 2024-07-01 10-17-24
tired changing the working and output directories. that's not the issue.
Screenshot from 2024-07-01 10-23-24
navigated directly to work dir and used wget to download like a normal file. it didn't download after getting connected to server.
Screenshot from 2024-07-01 10-28-19

I have stumbled upon this repo just because soon the server will be made offline and we are asked to take our data with us. I was able to download manually certain files like videos and presentation slide images using wget. what i wanted was the additional data and automated tool to combine like yours. Something is wrong with the chromium instance in venv. without venv i couldn't even get to this stage. it gives traceback errors saying syntax is wrong.

@C0D3D3V
Copy link
Owner

C0D3D3V commented Jul 1, 2024

@varuneranki this is a diffrent issue. I can test it tomorrow. But please open a new issue. If it would be a problem with authentification you could use a cookies.txt as described above.

@varuneranki
Copy link

@varuneranki this is a diffrent issue. I can test it tomorrow. But please open a new issue. If it would be a problem with authentification you could use a cookies.txt as described above.

I have solved the problem. It is related to the above issue. I'm not opening a new issue.
Screenshot from 2024-07-02 17-10-14
All it needed was a VPN connected to our Uni network and was able to download easily. I remember wget worked with RDP uni server but not with my local machine. Turns out it requires NAT hole punching support for application or a VPN tunnel for accessing certain servers hosed on secure networks. All across Europe universities use Eduroam network and it has additional layers of security.

@C0D3D3V
Copy link
Owner

C0D3D3V commented Jul 2, 2024

@varuneranki It is not related to this issue. But its nice to hear that you got it working. The problem is that the server is configured wrongly, and does not allow connections with a TLSv1.0 Client Hello message.
But thats also a problem of aiohttp, because you should be able to connect to such misconfigured servers. With the python requests library it works without problems.
See aio-libs/aiohttp#8478

@C0D3D3V
Copy link
Owner

C0D3D3V commented Jul 2, 2024

@varuneranki If you want to download a lot of sessions, have a look at "Batch processing" and "How can I speed up the rendering process?" in the Readme

@varuneranki
Copy link

@varuneranki If you want to download a lot of sessions, have a look at "Batch processing" and "How can I speed up the rendering process?" in the Readme

I only have 2 sessions to download so it wouldn't bother me. It was good to know as networking is quite a complex topic to understand. I wouldn't have even used this github repo if our admin has configured the server properly and if BBB download option worked. Since server should have been shutdown yesterday, it was quite a task to figure out in a complex way. Thank you once again. 👍

@C0D3D3V C0D3D3V closed this as completed Jul 3, 2024
C0D3D3V added a commit that referenced this issue Jul 4, 2024
@C0D3D3V
Copy link
Owner

C0D3D3V commented Jul 4, 2024

@varuneranki I fixed your problem in the new version. Now it also works without VPN.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants