[RRFC] Partial document access when reading from _changes

[eridal]

Motivation ("The Why")

The current /_changes/ allows full document retrieval using the param include_docs=true, but sometimes such documents contain extra data, that make downloading such sequences unnecessary costly.

I have been reading about the available filters and views and seems like there is no way to achieve what I'm trying to do. Please let me know if this is already doable!

Example

For instance, the following sequences with their file sizes:

• 5524439: 36M
• 5524449: 23M
• 5526384: 45M
• 5527795: 45M

How

Current Behaviour

You can quickly test this by downloading a few files:

seq=5524449

curl "https://replicate.npmjs.com/_changes?since=$seq&limit=1&include_docs=true" > /dev/null

Desired Behaviour

Upon closer inspection of those sequences, the bloat was never caused by a document's property that we were looking for, but rather on many different properties which sometimes are not even part of the package standard.

The desired behavior would be to been able to download only the pieces of the document that we need.

I'm not sure about what is the best way to express but one idea could be to follow the selector api

POST /_changes?since=123&limit=1&include_docs=true HTTP/1.1
{
  "_partial": {
    "doc": {
      "description": true, // explicitly ask
      "dist-tags": false,  // or refuse fields
    }
  }
}

References

• Couchdb changes: http://docs.couchdb.org/en/stable/api/database/changes.html
• Couchdb selectors: http://docs.couchdb.org/en/stable/api/database/changes.html#selector
• Couchdb _view: http://docs.couchdb.org/en/stable/api/database/changes.html#changes-filter-view