feat: add support for list pagination in list capability invocations #184

vasco-santos · 2022-11-18T10:14:54Z

Adds support for list pagination in store/list and upload/list.

Relevant notes on decisions made:

unfortunately, Databases like DynamoDB does not support pagination by just giving pageNumber and size. Aiming to have something generic (that works in DynamoDB, but also other DBs), chosen approach is to rely on cursor and size .
at first, attempted to make cursor a Link property of the capability. And it worked well for store/list, but then I got to use cases (upload/list for instance) where it cannot be a link. This is because Upload Table primary key contains ${dataCid}#${carCid} to be unique. So, in this case that is the cursor we need to have. There might be other kind of use cases, so having this as string is really the most flexible approach to support pagination.

Note:

it would be great to get a fix for TheCapabilityParser Interface should infer nb is optional if all its properties are optional ucanto#140 before merging this

BREAKING CHANGE: store/list and upload/list types now require nb object with optional properties

cloudflare-workers-and-pages · 2022-11-18T10:15:47Z

Deploying with Cloudflare Pages

Latest commit:	`2b20085`
Status:	✅ Deploy successful!
Preview URL:	https://2475a6b0.ucan-protocol.pages.dev
Branch Preview URL:	https://feat-add-support-for-list-pa.ucan-protocol.pages.dev

View logs

vasco-santos · 2022-11-18T10:27:51Z

packages/access-client/test/capabilities/upload.test.js

@@ -328,6 +328,7 @@ describe('upload capabilities', function () {
      audience: w3,
      with: account.did(),
      proofs: [await any],
+      nb: {},


Types require empty object when all nb properties are optional. need to look

this should not happen theres a lot of infer types in ucanto to mitigate this. Did you open an issue in ucanto ?

packages/access-client/src/capabilities/upload.js

packages/access-client/src/capabilities/store.js

packages/access-client/src/capabilities/upload.js

alanshaw · 2022-11-21T09:27:20Z

packages/access-client/src/capabilities/store.js

@@ -130,6 +130,18 @@ export const list = base.derive({
     * be stored.
     */
    with: URI.match({ protocol: 'did:' }),
+    nb: {
+      /**
+       * Item where a previous list operation stopped, inclusive of the previous


This sounds specific to the particular implementation backing this. Maybe this could be more generic?

BREAKING CHANGE: store/list and upload/list types now require nb object with optional properties

Co-authored-by: Alan Shaw <[email protected]>

* 'main' of https://github.com/web3-storage/w3-protocol: feat: add support for list pagination in list capability invocations (#184)

🤖 I have created a release *beep* *boop* --- ## [6.0.0](access-v5.0.2...access-v6.0.0) (2022-11-22) ### ⚠ BREAKING CHANGES * **access-client:** bump to major * store/list and upload/list types now require nb object with optional properties ### Features * [#153](#153) ([#177](#177)) ([d6d448c](d6d448c)) * **access-client:** bump to major ([4d5899f](4d5899f)) * account recover with email ([#149](#149)) ([6c659ba](6c659ba)) * add support for list pagination in list capability invocations ([#184](#184)) ([ced23db](ced23db)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

…184) Adds support for list pagination in `store/list` and `upload/list`. Relevant notes on decisions made: - unfortunately, Databases like DynamoDB does not support pagination by just giving `pageNumber` and `size`. Aiming to have something generic (that works in DynamoDB, but also other DBs), chosen approach is to rely on `cursor` and `size` . - at first, attempted to make `cursor` a Link property of the capability. And it worked well for `store/list`, but then I got to use cases (`upload/list` for instance) where it cannot be a link. This is because Upload Table primary key contains `${dataCid}#${carCid}` to be unique. So, in this case that is the cursor we need to have. There might be other kind of use cases, so having this as string is really the most flexible approach to support pagination. BREAKING CHANGE: store/list and upload/list types now require nb object with optional properties

* 'main' of https://github.com/web3-storage/w3-protocol: feat: add support for list pagination in list capability invocations (#184)

🤖 I have created a release *beep* *boop* --- ## [6.0.0](access-v5.0.2...access-v6.0.0) (2022-11-22) ### ⚠ BREAKING CHANGES * **access-client:** bump to major * store/list and upload/list types now require nb object with optional properties ### Features * [#153](#153) ([#177](#177)) ([7fbf2a1](7fbf2a1)) * **access-client:** bump to major ([7be598d](7be598d)) * account recover with email ([#149](#149)) ([91ad47d](91ad47d)) * add support for list pagination in list capability invocations ([#184](#184)) ([23206ad](23206ad)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

🤖 I have created a release *beep* *boop* --- ## [2.0.1](storacha/w3ui@vue-uploads-list-v2.0.0...vue-uploads-list-v2.0.1) (2022-12-15) ### Bug Fixes * update dependencies ([996871f](storacha/w3ui@996871f)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

🤖 I have created a release *beep* *boop* --- ## [2.0.1](storacha/w3ui@vue-uploads-list-v2.0.0...vue-uploads-list-v2.0.1) (2022-12-15) ### Bug Fixes * update dependencies ([c4752b0](storacha/w3ui@c4752b0)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

vasco-santos force-pushed the feat/add-support-for-list-pagination-in-list-capability-invocation branch from 8291f4c to 6fa28b8 Compare November 18, 2022 10:21

vasco-santos commented Nov 18, 2022

View reviewed changes

vasco-santos force-pushed the feat/add-support-for-list-pagination-in-list-capability-invocation branch 2 times, most recently from 55d7949 to 57d5cc5 Compare November 18, 2022 15:00

This was referenced Nov 18, 2022

feat: store and upload list with pagination support storacha/w3infra#28

Merged

TheCapabilityParser Interface should infer nb is optional if all its properties are optional storacha/ucanto#140

Open

alanshaw reviewed Nov 21, 2022

View reviewed changes

vasco-santos marked this pull request as ready for review November 21, 2022 10:15

vasco-santos force-pushed the feat/add-support-for-list-pagination-in-list-capability-invocation branch 2 times, most recently from 3b953fe to df2c7a9 Compare November 21, 2022 10:53

vasco-santos requested a review from alanshaw November 21, 2022 11:07

vasco-santos and others added 2 commits November 22, 2022 15:22

feat!: add support for list pagination in list capability invocations

4110583

BREAKING CHANGE: store/list and upload/list types now require nb object with optional properties

chore: apply suggestions from code review

d8f479e

Co-authored-by: Alan Shaw <[email protected]>

vasco-santos force-pushed the feat/add-support-for-list-pagination-in-list-capability-invocation branch from 9615cde to 28e6761 Compare November 22, 2022 14:22

fix: make cursor docs more generic

2b20085

vasco-santos force-pushed the feat/add-support-for-list-pagination-in-list-capability-invocation branch from 28e6761 to 2b20085 Compare November 22, 2022 14:25

alanshaw approved these changes Nov 22, 2022

View reviewed changes

vasco-santos merged commit ced23db into main Nov 22, 2022

vasco-santos deleted the feat/add-support-for-list-pagination-in-list-capability-invocation branch November 22, 2022 14:47

hugomrdias mentioned this pull request Nov 22, 2022

chore(main): release access 6.0.0 #175

Merged

gobengo mentioned this pull request Dec 1, 2022

chore(main): release 1.0.0 #231

Closed

hugomrdias mentioned this pull request Dec 7, 2022

chore(main): release access 9.0.1 #269

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add support for list pagination in list capability invocations #184

feat: add support for list pagination in list capability invocations #184

vasco-santos commented Nov 18, 2022 •

edited

Loading

cloudflare-workers-and-pages bot commented Nov 18, 2022 •

edited

Loading

vasco-santos Nov 18, 2022

hugomrdias Nov 22, 2022

alanshaw Nov 21, 2022

feat: add support for list pagination in list capability invocations #184

feat: add support for list pagination in list capability invocations #184

Conversation

vasco-santos commented Nov 18, 2022 • edited Loading

cloudflare-workers-and-pages bot commented Nov 18, 2022 • edited Loading

Deploying with Cloudflare Pages

vasco-santos Nov 18, 2022

Choose a reason for hiding this comment

hugomrdias Nov 22, 2022

Choose a reason for hiding this comment

alanshaw Nov 21, 2022

Choose a reason for hiding this comment

vasco-santos commented Nov 18, 2022 •

edited

Loading

cloudflare-workers-and-pages bot commented Nov 18, 2022 •

edited

Loading