Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Client by default gives gateway URLs with CAR CID query param hints #652

Open
heyjay44 opened this issue Mar 27, 2023 · 2 comments
Open

Client by default gives gateway URLs with CAR CID query param hints #652

heyjay44 opened this issue Mar 27, 2023 · 2 comments
Milestone
w3up phase 4

Comments

@heyjay44
Copy link
Contributor

No description provided.

@heyjay44 heyjay44 mentioned this issue Mar 27, 2023
Closed
23 tasks
@heyjay44 heyjay44 added this to the w3up phase 4 milestone Mar 27, 2023
@gobengo
Copy link
Contributor

gobengo commented Apr 10, 2023

@vasco-santos can you link to some context for this? I might be able to implement from that

@vasco-santos
Copy link
Contributor

There are two bits of context here:

  1. Freeway currently works with the following R2 buckets:

CARPARK - CAR file storage area. Key format <CAR_CID>/<CAR_CID>.car
SATNAV - Indexes of block offsets within CARs. Key format <CAR_CID>/<CAR_CID>.car.idx, index format MultihashIndexSorted.
DUDEWHERE - Mapping of root data CIDs to CAR CID(s). Key format <DATA_CID>/<CAR_CID>.

  1. CF has limits on sub-requests that we can perform from one worker

Per the above, and taking into account that client will know CAR CIDs that were sent in a multiple chunk upload, we can already hint to the gateway to avoid the gateway to look into DUDEWHERE, making it faster and saving sub-requests.

It is a origins query param as you can see https://github.com/web3-storage/freeway/blob/main/src/middleware.js#L48 . All CAR CIDs are provided delimited by ,

Peeja pushed a commit to storacha/upload-service that referenced this issue Jan 17, 2025
@github-actions
chore: release main (storacha#652)
53ba26d 
🤖 I have created a release *beep* *boop*
---


<details><summary>core: 2.4.0</summary>

##
[2.4.0](storacha/w3ui@core-v2.3.2...core-v2.4.0)
(2024-12-04)


### Features

* Use latest client (`@storacha/client`)
([storacha#650](storacha/w3ui#650))
([9f4e0d9](storacha/w3ui@9f4e0d9))
</details>

<details><summary>react: 2.5.2</summary>

##
[2.5.2](storacha/w3ui@react-v2.5.1...react-v2.5.2)
(2024-12-04)


### Bug Fixes

* Force a `@w3ui/react` release
([storacha#654](storacha/w3ui#654))
([1013069](storacha/w3ui@1013069))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Peeja added a commit to storacha/upload-service that referenced this issue Jan 17, 2025
@Peeja
Import w3ui to monorepo
6a90e8c 
* import-w3ui-incoming: (525 commits)
  Prepare to import `w3ui` to monorepo
  chore: release main (storacha#661)
  fix(core): revert to web3-storage/w3up-client (storacha#660)
  chore: release main (storacha#659)
  fix(core): update storacha/client version (storacha#658)
  chore: release main (storacha#657)
  fix: force a release
  chore: release main (storacha#656)
  fix: using the latest client to create spaces (storacha#655)
  chore: release main (storacha#652)
  fix: Force a `@w3ui/react` release (storacha#654)
  chore: Force a `@w3ui/react` release (storacha#653)
  chore: move to a single-pull-request release process (storacha#544)
  feat: Use latest client (`@storacha/client`) (storacha#650)
  chore(main): release react 2.5.1 (storacha#649)
  chore(main): release core 2.3.2 (storacha#648)
  fix: add missing receiptsEndpoint to client instantiation (storacha#647)
  chore(main): release react 2.5.0 (storacha#646)
  feat: Re-`claim` delegations during client setup (storacha#644)
  chore: Add packageManager to package.json (storacha#640)
  ...
Peeja pushed a commit to storacha/upload-service that referenced this issue Jan 29, 2025
@github-actions
chore: release main (storacha#652)
0b53bde 
🤖 I have created a release *beep* *boop*
---


<details><summary>core: 2.4.0</summary>

##
[2.4.0](storacha/w3ui@core-v2.3.2...core-v2.4.0)
(2024-12-04)


### Features

* Use latest client (`@storacha/client`)
([storacha#650](storacha/w3ui#650))
([ff8b8b6](storacha/w3ui@ff8b8b6))
</details>

<details><summary>react: 2.5.2</summary>

##
[2.5.2](storacha/w3ui@react-v2.5.1...react-v2.5.2)
(2024-12-04)


### Bug Fixes

* Force a `@w3ui/react` release
([storacha#654](storacha/w3ui#654))
([d53104b](storacha/w3ui@d53104b))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Peeja added a commit to storacha/upload-service that referenced this issue Jan 29, 2025
@Peeja
feat(repo): merge main from https://github.com/storacha/w3ui
01ea1b2 
* __tmp_nx_import__/__nx_tmp_import__/main: (524 commits)
  chore: release main (storacha#661)
  fix(core): revert to web3-storage/w3up-client (storacha#660)
  chore: release main (storacha#659)
  fix(core): update storacha/client version (storacha#658)
  chore: release main (storacha#657)
  fix: force a release
  chore: release main (storacha#656)
  fix: using the latest client to create spaces (storacha#655)
  chore: release main (storacha#652)
  fix: Force a `@w3ui/react` release (storacha#654)
  chore: Force a `@w3ui/react` release (storacha#653)
  chore: move to a single-pull-request release process (storacha#544)
  feat: Use latest client (`@storacha/client`) (storacha#650)
  chore(main): release react 2.5.1 (storacha#649)
  chore(main): release core 2.3.2 (storacha#648)
  fix: add missing receiptsEndpoint to client instantiation (storacha#647)
  chore(main): release react 2.5.0 (storacha#646)
  feat: Re-`claim` delegations during client setup (storacha#644)
  chore: Add packageManager to package.json (storacha#640)
  chore: Add `dev` script (storacha#645)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
w3up phase 4
Development

No branches or pull requests
3 participants
@gobengo @heyjay44 @vasco-santos