refactor: filecoin client to use new capabilities

docs: better docs fix: client tests
storacha · Oct 23, 2023 · d2816c8 · d2816c8
1 parent d8fd799
commit d2816c8
Show file tree

Hide file tree

Showing 16 changed files with 655 additions and 466 deletions.
diff --git a/packages/filecoin-client/README.md b/packages/filecoin-client/README.md
@@ -3,7 +3,7 @@
 
 ## About
 
-The `@web3-storage/filecoin-client` package provides the "low level" client API to make data uploaded with the w3up platform available in Filecoin Storage providers. It is based on [web3-storage/specs/w3-filecoin.md])https://github.com/web3-storage/specs/blob/feat/filecoin-spec/w3-filecoin.md) and is not intended for web3.storage end users.
+The `@web3-storage/filecoin-client` package provides the "low level" client API to make data uploaded with the w3up platform available in Filecoin Storage providers. It is based on [web3-storage/specs/w3-filecoin.md](https://github.com/web3-storage/specs/blob/feat/filecoin-spec/w3-filecoin.md) and is not intended for web3.storage end users.
 
 ## Install
 
@@ -15,46 +15,110 @@ npm install @web3-storage/filecoin-client
 
 ## Usage
 
-### `Storefront.filecoinAdd`
+### `Storefront.filecoinOffer`
 
-Request a Storefront service to add computed filecoin piece into Filecoin Storage Providers.
+The [`filecoin/offer`](https://github.com/web3-storage/specs/blob/main/w3-filecoin.md#filecoinoffer) task can be executed to request storing a content piece in Filecoin. It issues a signed receipt of the execution result.
+
+A receipt for successful execution will contain an effect, linking to a `filecoin/submit` task that will complete asynchronously.
+
+Otherwise the task is failed and the receipt will contain details of the reason behind the failure.
 
 ```js
 import { Storefront } from '@web3-storage/filecoin-client'
 
-const add = await Storefront.filecoinQueue(
+const res = await Storefront.filecoinOffer(
   invocationConfig,
-  piece,
-  content
+  content,
+  piece
+)
+```
+
+```typescript
+function filecoinOffer(
+  conf: InvocationConfig,
+  content: Link, // Content CID
+  piece: Piece, // Filecoin piece
+): Promise<Receipt>
+```
+
+More information: [`InvocationConfig`](#invocationconfig)
+
+### `Storefront.filecoinSubmit`
+
+The [`filecoin/submit`](https://github.com/web3-storage/specs/blob/main/w3-filecoin.md#filecoinsubmit) task is an _effect_ linked from successful execution of a `filecoin/offer` task, it is executed to issue a receipt for the success or failure of the task.
+
+A receipt for successful execution indicates that the offered piece has been submitted to the pipeline. In this case the receipt will contain an effect, linking to a `piece/offer` task that will complete asynchronously.
+
+Otherwise the task is failed and the receipt will contain details of the reason behind the failure.
+
+```js
+import { Storefront } from '@web3-storage/filecoin-client'
+
+const res = await Storefront.filecoinSubmit(
+  invocationConfig,
+  content,
+  piece
 )
 ```
 
 ```typescript
-function filecoinQueue(
+function filecoinSubmit(
   conf: InvocationConfig,
+  content: Link, // Content CID
   piece: Piece, // Filecoin piece
+): Promise<Receipt>
+```
+
+More information: [`InvocationConfig`](#invocationconfig)
+
+### `Storefront.filecoinAccept`
+
+The [`filecoin/accept`](https://github.com/web3-storage/specs/blob/main/w3-filecoin.md#filecoinsubmit) task is an _effect_ linked from successful execution of a `filecoin/offer` task, it is executed to issue a receipt for the success or failure of the task.
+
+A receipt for successful execution indicates that the offered piece has been accepted in a Filecoin deal. In this case the receipt will contain proofs that the piece was included in an aggregate and deal.
+
+Otherwise the task is failed and the receipt will contain details of the reason behind the failure.
+
+```js
+import { Storefront } from '@web3-storage/filecoin-client'
+
+const res = await Storefront.filecoinAccept(
+  invocationConfig,
+  content,
+  piece
+)
+```
+
+```typescript
+function filecoinAccept(
+  conf: InvocationConfig,
   content: Link, // Content CID
+  piece: Piece, // Filecoin piece
 ): Promise<Receipt>
 ```
 
 More information: [`InvocationConfig`](#invocationconfig)
 
-### `Aggregator.pieceAdd`
+### `Aggregator.pieceOffer`
+
+The [`piece/offer`](https://github.com/web3-storage/specs/blob/main/w3-filecoin.md#pieceoffer) task can be executed to request that a piece be aggregated for inclusion in an upcoming an Filecoin deal. It issues a signed receipt of the execution result. It is _also_ an effect linked from successful execution of a `filecoin/submit` task.
+
+A receipt for successful execution will contain an effect, linking to a `piece/accept` task that will complete asynchronously.
 
-Request an Aggregator service to add a filecoin piece into an aggregate to be offered to Filecoin Storage Providers.
+Otherwise the task is failed and the receipt will contain details of the reason behind the failure.
 
 ```js
 import { Aggregator } from '@web3-storage/filecoin-client'
 
-const add = await Aggregator.aggregateQueue(
+const res = await Aggregator.pieceOffer(
   invocationConfig,
   piece,
   group
 )
 ```
 
 ```typescript
-function aggregateQueue(
+function pieceOffer(
   conf: InvocationConfig,
   piece: Piece, // Filecoin piece
   group: string, // Aggregate grouping with different criterium like storefront
@@ -63,48 +127,109 @@ function aggregateQueue(
 
 More information: [`InvocationConfig`](#invocationconfig)
 
-### `Dealer.aggregateAdd`
+### `Aggregator.pieceAccept`
 
-Request a Dealer service to offer a filecoin piece (larger aggregate of pieces) to Filecoin Storage Providers.
+The [`piece/accept`](https://github.com/web3-storage/specs/blob/main/w3-filecoin.md#pieceaccept) task is an _effect_ linked from successful execution of a `piece/offer` task, it is executed to issue a receipt for the success or failure of the task.
+
+A receipt for successful execution indicates that the offered piece was included in an aggregate. In this case the receipt will contain the aggregate piece CID and a proof that the piece was included in the aggregate. It also includes an effect, linking to an `aggregate/offer` task that will complete asynchronously.
+
+Otherwise the task is failed and the receipt will contain details of the reason behind the failure.
+
+```js
+import { Aggregator } from '@web3-storage/filecoin-client'
+
+const res = await Aggregator.pieceAccept(
+  invocationConfig,
+  piece,
+  group
+)
+```
+
+```typescript
+function pieceAccept(
+  conf: InvocationConfig,
+  piece: Piece, // Filecoin piece
+  group: string, // Aggregate grouping with different criterium like storefront
+): Promise<Receipt>
+```
+
+More information: [`InvocationConfig`](#invocationconfig)
+
+### `Dealer.aggregateOffer`
+
+The [`aggregate/offer`](https://github.com/web3-storage/specs/blob/main/w3-filecoin.md#aggregateoffer) task can be executed to request an aggregate be added to a deal with a Storage Provider. It issues a signed receipt of the execution result. It is _also_ an effect linked from successful execution of a `piece/accept` task.
+
+A receipt for successful execution will contain an effect, linking to an `aggregate/accept` task that will complete asynchronously.
+
+Otherwise the task is failed and the receipt will contain details of the reason behind the failure.
 
 ```js
 import { Dealer } from '@web3-storage/filecoin-client'
 
-const add = await Dealer.dealQueue(
+const res = await Dealer.aggregateOffer(
   invocationConfig,
   aggregate,
-  pieces,
-  storefront,
-  label
+  pieces
 )
 ```
 
 ```typescript
-function dealQueue(
+function aggregateOffer(
   conf: InvocationConfig,
   aggregate: Piece, // Filecoin piece representing aggregate
   pieces: Piece[],  // Filecoin pieces part of the aggregate (sorted)
-  label: string     // optional label for deal
 ): Promise<Receipt>
 ```
 
 More information: [`InvocationConfig`](#invocationconfig)
 
-### `Chain.chainInfo`
+### `Dealer.aggregateAccept`
+
+The [`aggregate/accept`](https://github.com/web3-storage/specs/blob/main/w3-filecoin.md#aggregateaccept) task is an _effect_ linked from successful execution of a `aggregate/offer` task, it is executed to issue a receipt for the success or failure of the task.
+
+A receipt for successful execution indicates that an aggregate has been accepted for inclusion in a Filecoin deal. In this case the receipt will contain proofs that the piece was included in an aggregate and deal.
+
+Otherwise the task is failed and the receipt will contain details of the reason behind the failure, as well as multiple effects, linking to `piece/offer` tasks that will retry _valid_ pieces and complete asynchronously.
+
+```js
+import { Dealer } from '@web3-storage/filecoin-client'
+
+const res = await Dealer.aggregateAccept(
+  invocationConfig,
+  aggregate,
+  pieces
+)
+```
+
+```typescript
+function aggregateAccept(
+  conf: InvocationConfig,
+  aggregate: Piece, // Filecoin piece representing aggregate
+  pieces: Piece[],  // Filecoin pieces part of the aggregate (sorted)
+): Promise<Receipt>
+```
+
+More information: [`InvocationConfig`](#invocationconfig)
+
+### `DealTracker.dealInfo`
+
+The [`deal/info`](https://github.com/web3-storage/specs/blob/main/w3-filecoin.md#dealinfo) task can be executed to request deal information for a given piece. It issues a signed receipt of the execution result.
+
+A receipt for successful execution will contain details of deals the provided piece CID is currently active in.
 
-Request a Chain service to find chain information of a given piece. It will return deals where given piece is present in Receipt.
+Otherwise the task is failed and the receipt will contain details of the reason behind the failure.
 
 ```js
-import { Chain } from '@web3-storage/filecoin-client'
+import { DealTracker } from '@web3-storage/filecoin-client'
 
-const add = await Chain.chainInfo(
+const add = await DealTracker.dealInfo(
   invocationConfig,
   piece
 )
 ```
 
 ```typescript
-function chainInfo(
+function dealInfo(
   conf: InvocationConfig,
   piece: Piece, // Filecoin piece to check
 ): Promise<Receipt>

diff --git a/packages/filecoin-client/package.json b/packages/filecoin-client/package.json
@@ -65,6 +65,7 @@
   },
   "devDependencies": {
     "@ipld/car": "^5.1.1",
+    "@ipld/dag-json": "^10.1.4",
     "@types/assert": "^1.5.6",
     "@types/mocha": "^10.0.1",
     "@ucanto/principal": "^8.0.0",

diff --git a/packages/filecoin-client/src/aggregator.js b/packages/filecoin-client/src/aggregator.js
@@ -1,8 +1,6 @@
 import { connect } from '@ucanto/client'
 import { CAR, HTTP } from '@ucanto/transport'
-
-import { Filecoin as FilecoinCapabilities } from '@web3-storage/capabilities'
-
+import * as Aggregator from '@web3-storage/capabilities/filecoin/aggregator'
 import { services } from './service.js'
 
 /**
@@ -21,14 +19,25 @@ export const connection = connect({
 })
 
 /**
- * Queues a piece to the aggregator system of the filecoin pipeline.
+ * The `piece/offer` task can be executed to request that a piece be aggregated
+ * for inclusion in an upcoming an Filecoin deal. It issues a signed receipt
+ * of the execution result. It is _also_ an effect linked from successful
+ * execution of a `filecoin/submit` task.
+ *
+ * A receipt for successful execution will contain an effect, linking to a
+ * `piece/accept` task that will complete asynchronously.
+ *
+ * Otherwise the task is failed and the receipt will contain details of the
+ * reason behind the failure.
+ *
+ * @see https://github.com/web3-storage/specs/blob/main/w3-filecoin.md#pieceoffer
  *
  * @param {import('./types.js').InvocationConfig} conf - Configuration
  * @param {import('@web3-storage/data-segment').PieceLink} piece
  * @param {string} group
  * @param {import('./types.js').RequestOptions<AggregatorService>} [options]
  */
-export async function aggregateQueue(
+export async function pieceOffer(
   { issuer, with: resource, proofs, audience },
   piece,
   group,
@@ -37,7 +46,7 @@ export async function aggregateQueue(
   /* c8 ignore next */
   const conn = options.connection ?? connection
 
-  const invocation = FilecoinCapabilities.aggregateQueue.invoke({
+  const invocation = Aggregator.pieceOffer.invoke({
     issuer,
     /* c8 ignore next */
     audience: audience ?? services.AGGREGATOR.principal,
@@ -53,32 +62,42 @@ export async function aggregateQueue(
 }
 
 /**
- * Add a piece to the aggregator system of the filecoin pipeline.
+ * The `piece/accept` task is an _effect_ linked from successful execution of a
+ * `piece/offer` task, it is executed to issue a receipt for the success or
+ * failure of the task.
+ *
+ * A receipt for successful execution indicates that the offered piece was
+ * included in an aggregate. In this case the receipt will contain the
+ * aggregate piece CID and a proof that the piece was included in the
+ * aggregate. It also includes an effect, linking to an `aggregate/offer` task
+ * that will complete asynchronously.
+ *
+ * Otherwise the task is failed and the receipt will contain details of the
+ * reason behind the failure.
+ *
+ * @see https://github.com/web3-storage/specs/blob/main/w3-filecoin.md#pieceaccept
  *
  * @param {import('./types.js').InvocationConfig} conf - Configuration
  * @param {import('@web3-storage/data-segment').PieceLink} piece
- * @param {string} storefront
  * @param {string} group
  * @param {import('./types.js').RequestOptions<AggregatorService>} [options]
  */
-export async function aggregateAdd(
+export async function pieceAccept(
   { issuer, with: resource, proofs, audience },
   piece,
-  storefront,
   group,
   options = {}
 ) {
   /* c8 ignore next */
   const conn = options.connection ?? connection
 
-  const invocation = FilecoinCapabilities.aggregateAdd.invoke({
+  const invocation = Aggregator.pieceAccept.invoke({
     issuer,
     /* c8 ignore next */
     audience: audience ?? services.AGGREGATOR.principal,
     with: resource,
     nb: {
       piece,
-      storefront,
       group,
     },
     proofs,