Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Collections fail to load if any of the Github request is above 1mb #1092

Closed
peduarte opened this issue Feb 8, 2018 · 4 comments · Fixed by #1224
Closed

Collections fail to load if any of the Github request is above 1mb #1092

peduarte opened this issue Feb 8, 2018 · 4 comments · Fixed by #1224

Comments

@peduarte
Copy link

peduarte commented Feb 8, 2018

- Do you want to request a feature or report a bug?
Bug

- What is the current behavior?
Netlify CMS queries the Github API for files to add to its Collections. If any of those requests is above 1mb, Netlify CMS fails to render any of the other files.

- If the current behavior is a bug, please provide the steps to reproduce.
Create a Markdown file above 1mb and notice how the Netlify CMS won't render any of the other files in a collection.

- What is the expected behavior?
In my opinion, Netlify CMS should be able to catch this error, display the file in a disabled state, and let the end user know why it was not able to load.

Note

The reason Netlify CMS cannot query files above 1mb is a Github API limitation. More info: https://developer.github.com/v3/repos/contents/#get-contents

@tech4him1
Copy link
Contributor

Looks like GitHub is throwing a 403 and returning this error:

{
  "message": "This API returns blobs up to 1 MB in size. The requested blob is too large to fetch via the API, but you can use the Git Data API to request blobs up to 100 MB in size.",
  "errors": [
    {
      "resource": "Blob",
      "field": "data",
      "code": "too_large"
    }
  ],
  "documentation_url": "https://developer.github.com/v3/repos/contents/#get-contents"
}

@tech4him1
Copy link
Contributor

tech4him1 commented Feb 8, 2018

Tracing the load failure, it looks like it is failing to show any entries for the same reason that I outlined in #929 (comment).

In my opinion, Netlify CMS should be able to catch this error, display the file in a disabled state, and let the end user know why it was not able to load.

I think just hiding or disabling the single file is a good first step ("patch"). Later, we'll probably want to load files via the alternative API so that we can support files up to 100 MB.

@tech4him1
Copy link
Contributor

This has been partially fixed in #1093 -- if a file fails to load, all other entries in a collection will still work.

@tech4him1
Copy link
Contributor

tech4him1 commented Mar 5, 2018

@Benaiah Do you think we should fetch files by SHA all the time, or just when we get a failure like this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants