Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Google Drive remote documentation #381

Closed
ei-grad opened this issue May 25, 2019 · 5 comments · Fixed by #833
Closed

Google Drive remote documentation #381

ei-grad opened this issue May 25, 2019 · 5 comments · Fixed by #833
Labels
A: docs Area: user documentation (gatsby-theme-iterative) type: enhancement Something is not clear, small updates, improvement suggestions

Comments

@ei-grad
Copy link
Contributor

ei-grad commented May 25, 2019

Just creating the docs issue for the upcoming iterative/dvc#2040 which would bring the Google Drive support.

Some things to mention:

  1. There are 3 types of URLs:

    1. Subfolder in the "My Drive" folder:

      gdrive://root/Datasets/my-dataset
      
    2. Folder by ID (recommended):

      gdrive://1r3UbnmS5B4-7YZPZmyqJuCxLVps1mASC
      

      (get it from the last part of URL https://drive.google.com/drive/folders/{here})

    3. Dataset named "my-dataset" in the hidden application folder:

      gdrive://appDataFolder/my-dataset
      

      (this one wouldn't be visible through Google Drive web UI and
      couldn't be shared)

  2. OAuth2, client ID and quotas, we need something like this: https://rclone.org/drive/#making-your-own-client-id

  3. There is an oauth_id option in remote config section, which could contain arbitary identifier and is meant to be used to allow different Google user accounts usage for different remotes.

@shcheklein shcheklein added the A: docs Area: user documentation (gatsby-theme-iterative) label May 27, 2019
@shcheklein shcheklein changed the title Google Drive remote documentation support Google Drive remote documentation May 27, 2019
@shcheklein shcheklein added command-reference type: enhancement Something is not clear, small updates, improvement suggestions labels May 27, 2019
@shcheklein shcheklein changed the title support Google Drive remote documentation Google Drive remote documentation May 27, 2019
@shcheklein
Copy link
Member

rclone already has a high quota and I will continue to make sure it is high enough by contacting Google. - @ei-grad do you know what the default limits are? Should we contact google to raise these limits?

@ei-grad
Copy link
Contributor Author

ei-grad commented Jun 10, 2019

image

Should we contact google to raise these limits?

I don't think it would be needed in near future. Usecase of rclone is a bit different from DVC's. But it may be a good idea to setup a monitoring somehow to see if DVC's default quota is overused, so we could contact them early.

@ei-grad
Copy link
Contributor Author

ei-grad commented Jun 10, 2019

@dmpetrov may have a better understanding if GDrive backend would be heavily used by something like a webinar or class lessons in the groups of students... Probably 10+ users syncing a big repo would affect each other, so maybe we have to give the instructions for the teachers to recommend to use the separate client tokens per student in such scenarios.

@ei-grad
Copy link
Contributor Author

ei-grad commented Jun 10, 2019

We also can go to the dark side and create some client tokens in advance, and chose between them randomly :).

// but this is why google wouldn't be restrictive in quota extension inquiries, so never mind

@shcheklein
Copy link
Member

@ei-grad makes sense, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A: docs Area: user documentation (gatsby-theme-iterative) type: enhancement Something is not clear, small updates, improvement suggestions
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants