Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

deviantart failed to rip #1356

Closed
tayser1 opened this issue Jun 27, 2019 · 6 comments
Closed

deviantart failed to rip #1356

tayser1 opened this issue Jun 27, 2019 · 6 comments

Comments

@tayser1
Copy link

tayser1 commented Jun 27, 2019

Ripme version: 1.7.85
Java version: 10.0.2 64 bit
Operating system: Windows 10
Exact URL you were trying to rip when the problem occurred: https://www.deviantart.com/mewitti/gallery/?catpath=/
Please include any additional information about how to reproduce the problem:

insert own deviantart cookie in rip.properties located at local appdata
"auth=cookie;auth_secure=cookie;userinfo=cookie"
try rip deviantart user gallery

Expected Behavior

Rip deviantart gallery

Actual Behavior

Not downloading -> Error: null

@MrPlaygon
Copy link
Contributor

I am currently experiencing the same problem and working on a solution. This null error comes from DA thinking you're not a human (which the program is not) and you would have to solve a captcha. That however is impossible when you are not a human (at least for me).

@cyian-1756 I want to know your opinion what to do with the deviantart ripper. I see two options atm (correct me if I'm wrong pls):

  1. Change the DA Ripper back to use the Deviantart API -> Makes it stable again
  2. Change the current DA Ripper to not log in. This might (did not try in depth yet) solve some problems regarding downloads but will result in sometimes incomplete galleries because you have to be logged in to view certain art pieces.

I personally would recommend the first for now because I get more time to build a new DA Ripper and to test some things on DA Website. Especially the weird 401 Errors I encoutered recently.

Cheers and a great day
MrPlaygon

@cyian-1756
Copy link
Collaborator

@MrPlaygon

Change the DA Ripper back to use the Deviantart API -> Makes it stable again

I think this would be the best option

@rautamiekka
Copy link
Contributor

@MrPlaygon

Change the DA Ripper back to use the Deviantart API -> Makes it stable again

I think this would be the best option

That was already discussed, concluding that using the API makes zero sense, and gives very limited amount of data.

@MrPlaygon
Copy link
Contributor

I know we already discussed that but I still think it is worth considering to go back because the HTML website has so many weird things going on.
For example the login problem. With that many users who use ripme there is no way to prevent DeviantArt from noticing that the user is a bot.

However I will still try to make an html ripper work without login and see how efficient it is.

@MrPlaygon
Copy link
Contributor

I found the root of the problem. Deviantart has some upcoming design changes in their website (new HTML and CSS).
Sometimes the new and sometimes the old design loads. Therefore it sometimes works sometimes not.
I will pause development of an HTML Ripper until DA fully switches to the new design.

@MrPlaygon MrPlaygon mentioned this issue Sep 13, 2019
12 tasks
@metaprime
Copy link
Contributor

[Mega-Thread] DeviantArt ripper is broken; yes we know -- de-duping all other DeviantArt issues to this one. #2063

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants