Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FUSE? #38

Open
rcfa opened this issue Feb 15, 2022 · 12 comments
Open

FUSE? #38

rcfa opened this issue Feb 15, 2022 · 12 comments

Comments

@rcfa
Copy link

rcfa commented Feb 15, 2022

Thanks! This looks great, as I theoretically 13TB lifetime with them, which I can't use because they changed/abandoned their client software. (Don't get, why they don't at least expose a WebDAV interface to their storage.)

That said, 13TB, even with CLI tools aren't usable for much without lots of wasted time uploading, checking, etc.

Ideal would be a FUSE interface ( https://wiki.osdev.org/FUSE ), so the storage can be mounted as a network drive, and then used with random backup, file-sync, etc. software, without degoo specific code. Is that a direction you plan to take your code in?

The reason I ask: you seem to have already implemented the type of operations that sftp offers, and there's a FUSE abstraction layer based on those ( https://github.com/libfuse/sshfs )

@rcfa
Copy link
Author

rcfa commented Feb 21, 2022

Actually, there's another way this project's effort could be used.
There's the rClone https://rclone.org https://github.com/rclone/rclone cloud sync effort, which also allows for FUSE mounting of cloud storage.

They don't support degoo however, for lack of an API.

Maybe, with your work and their work combined, rClone could bring support for degoo...

@bernd-wechner
Copy link
Owner

You took the words right out of my mouth, actually. I have been meaning to drop rclone a line to see if they will take the reverse engineering I have captured here in order to add the Degoo service to the list of services they support. As I have many distracting priorities at present if you want to follow that up please do. We can only hope that some rclone contributor(s) might be excited by the opportunity to add another provider to their list with the API, while unspecified, deconstructed at least in part here:

https://github.com/bernd-wechner/Degoo/blob/master/degoo/API.py

@rcfa
Copy link
Author

rcfa commented Feb 21, 2022

rclone/rclone#4424

There's this thread. They seem to be hesitant about reverse engineering APIs, but non-Windows platforms support supposedly pluggable backends through shared libraries. So you could create one of these, and at least all the *IX platforms would be able to use it. And if things are reasonably stable, they might absorb it.
Or they might do it from the start, if someone else is maintaining the relevant code and they don't have to worry about it.

@rcfa
Copy link
Author

rcfa commented Feb 21, 2022

Just also found this, which seems to build on your work:

https://github.com/MDKPredator/degoo_drive

@forrie
Copy link

forrie commented Feb 21, 2022

Re: rclone

I use this from time to time, and I can't believe I didn't think of suggesting this LOL I think it would be a perfect fit for rclone's model.

Regarding the reverse engineering of protocol, if you have a relationship with someone at Degoo, perhaps you can include that mention -- or even ask your contact at Degoo how they feel about it. That statement could be more convincing. But if Degoo isn't cooperating, that might be viewed differently by the rclone folks?

In any case, great idea rfca :)

@bernd-wechner
Copy link
Owner

Alas, no contact at Degoo. I have had email exchanges with support but they have declined to offer any technical information on their API. They are aware of this reverse engineering.

@forrie
Copy link

forrie commented Feb 21, 2022

I have been utterly puzzled by Degoo's apparent apathy. I bought the 10TB "forever" deal, from Cult of Mac, a long time ago and have had a lot of frustration with them. I understand they may be a small shop; however, it's really in their best interests to make the platform more accessible, usable, which this project being important in context.

If the rclone folks agree to incorporate interfacing code, then someone is going to need to keep on it and analyze the changes Degoo makes, updating the rclone code accordingly. Seems like a daunting, frustrating task... especially if Degoo couldn't care less. shrug

@sskras
Copy link

sskras commented Aug 14, 2022

@bernd-wechner commented on Feb 21:

We can only hope that some rclone contributor(s) might be excited by the opportunity to add another provider to their list with the API, while unspecified, deconstructed at least in part here:

https://github.com/bernd-wechner/Degoo/blob/master/degoo/API.py

@rcfa commented on Feb 21:

non-Windows platforms support supposedly pluggable backends through shared libraries. So you could create one of these, and at least all the *IX platforms would be able to use it. And if things are reasonably stable, they might absorb it.

@forrie commented on Feb 21:

If the rclone folks agree to incorporate interfacing code, then someone is going to need to keep on it and analyze the changes Degoo makes, updating the rclone code accordingly. Seems like a daunting, frustrating task...

Folks, I would summarize you the following way:

it would be nice if someone implements this as a standalone, dedicated lib, eg. libdegoo, and then optionally maintains it – continuously tests for breakages in the De-facto API and reimplements the changes back into the lib.

So this needs like at least one dedicated person who:

  • has passion of using Degoo,
  • understands Python,
  • understands modern Rest APIs,
  • optionally understands Angular JS,
  • optionally understands Webpack,
  • understands graphQL

... right?

@ItachiSan
Copy link

Hi all,
I am quite interested in this chat.
I found this project (was thinking of reverse engineering Degoo myself for a small Dropbox-like app).
The reason I didn't start doing it is because of the license; while I wholesomely agree, I am unsure if it is copyleft or not.

In short, my idea is:

  • Re-implement / improve the code in api.py so that it can be a degoo.py (honestly, I wanted to use Rust... why not both 😉)
  • Write the tray icon separately (or not, depending on the next point)
  • Submit a request to rclone (not doing the GUI is fine for me too)

I thus want to verify the following whether the license is GPLv3 like and thus everything is should be under this license.
The reason is that, if I reimplement stuff, I will properly give credits and such, but having the responsibility to verify contributors to e.g. not having criminal records that are against human rights sounds hard as an individual (I am considering the extremes of the license here)

@bernd-wechner
Copy link
Owner

@sskras, nice summary. I cover some of those qualities but alas not the first one: "dedicated" ;-) . Mainly as I'm stretched across far too many commitments. It seems reverse engineering the API motivated me a while ago but I've got distracted and stuck on the login issue (unable thus far to solve that in Python as we now understand its causes just not a Python solution). Yet. Once that's solved, woo hoo. Of course yes, then a basic test suite would be a priority so we can keep up to date on status of the API modelling.

But ...

@ItachiSan, you are right all round, I think a FUSE interface and a sync client would both rock and help us get value out of our TB of Degoo storage that Degoo are selling (still) but have no useful interface too enabling us to do backups to it. And why not Rust, go for it, anyone with time and enthusiasm to work on a FUSE interface and get syncing working better has my blessings (one of the sync issues I've had is that Degoo don't provide server side file hashes and so change detection is not slick).

I'm not sure what you mean by license. The Degoo API has no license that I'm aware of it's a publicly exposed API, their website uses it and exposes it, and I see no licensing notes anywhere, having assumed that if you have an account and can log in, you are licensed to use their storage services. It would surprise me if they were so obstructionist as to encode in a license some restriction to using clients they write, least of all when they won't write a decent one.

As to my reverse engineering the license is here:

https://github.com/bernd-wechner/Degoo/blob/master/LICENSE.md

more liberal than the GPL by a ways ;-) and yes, mainly concerned with ethical application ....

P.S. Your username conjures unshakeable images of Japanese in my mind (Itachi san ...) and yet you're apprently Swedish with an Italian name ... love the diversity (trumps mine even ;-).

@vinibali
Copy link

Hi!
I'm here to push you guys towards to talk about rclone again.
Degoo's plan is super impressive, we could benefit a lot if we could use rclone for this provider.
What do you think?

@bernd-wechner
Copy link
Owner

I found this in the interim:

https://thepythoncorner.com/posts/2017-02-27-writing-a-fuse-filesystem-in-python/

And I got login working again. But have yet to finish testing. Have my fingers it way too many pies. This is slow and on my backburner but with login fixed I found the API had progressed and so had to update some of the API interface and am not done with that. Tested all utils bar put and get (the important ones ;-).

Looks like pyfuse can help us, though I do wonder about how laggy and sluggish a fuse mounted drive will be ;-). Will implement it some time and see.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants