From 128decd6ab758b07c0bcfd8793e1b17b674b4174 Mon Sep 17 00:00:00 2001 From: Volker Mische Date: Thu, 22 Nov 2018 16:49:19 +0100 Subject: [PATCH 1/8] Major API changes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The whole IPLD APIs get a review to make them more consistent and easier to use. The biggest change is that there's not `resolve` API anymore. From now on you access the properties of the JavaScript objects directly. Issues that were taken into account: - https://github.com/ipld/interface-ipld-format/issues/31 - [x] Remove `isLink()` method from formats: `isLink()` is no longer needed as all links will be CID objects you can easily check for - https://github.com/ipld/interface-ipld-format/issues/44 - [x] Proposal: Move resolver to use CID instances for links: Not applicable anymore as `resolve.resolve()` is removed - https://github.com/ipld/interface-ipld-format/issues/46 - [x] properties: align spec with implementation: Covered by spec - https://github.com/ipld/interface-ipld-format/issues/49 - [x] Define how `toJSON()` should look like: Binary and CIDs are defined with example - https://github.com/ipld/interface-ipld-format/issues/34 - [x] Implementation of nested objects: Won’t fix as `tree()` is not part of the API anymore Closes #48. --- README.md | 100 ++++++++++++++++++++++++++++++++---------------------- 1 file changed, 60 insertions(+), 40 deletions(-) diff --git a/README.md b/README.md index d86133e..716c905 100644 --- a/README.md +++ b/README.md @@ -58,81 +58,101 @@ Include this badge in your readme if you make a new module that implements inter A valid (read: that follows this interface) IPLD format implementation the following API. -### IPLD format utils +IPLD Format APIs are restricted to a single IPLD Node, they never access any linked IPLD Nodes. -#### `util.serialize(dagNode, callback)` -> serializes a dagNode of an IPLD format into its binary format +### serialize(IpldNode) -`callback` must have the signature `function (err, binaryBlob)`, where `err` is an Error is the function fails and `binaryBlob` is a Buffer containing the serialized version. +> Serializes the internal representation of an IPLD Node into a binary blob. -#### `util.deserialize(binaryBlob, callback)` +`IpldNode` is a previously deserialized binary Block. -> deserializes a binary blob into the instance +Returns a Promise containing a `Buffer` with the serialized version of the given IPLD Node. -`callback` must have the signature `function (err, dagNode)`, where `err` is an Error if the function fails and `dagNode` is the dagNode that got deserialized in the process. -#### `util.cid(binaryBlob[, options], callback)` +### deserialize(binaryBlob) -> get the CID of a binary blob +> Deserialize into internal representation. -Options include: - - version - the CID version to be used (defaults to 1) - - hashAlg - the hash algorithm to be used (default to the one set by the format) +The result is a JavaScript object, which may also be a Proxy object in case the data shouldn’t be deserialized as a whole. Its fields are the public API that can be resolved through. It’s up to the format to add convenient methods for manipulating the data. The returned object may also be a Proxy object in case the data shouldn’t be deserialized as a whole. -`callback` must have the signature `function (err, cid)`, where `err` is an Error if the function fails and `cid` is a CID instance of the binary blob. +Returns a Promise containing the Javascript object. This object must be able to be serialized with a `serialize()` call. -### Local resolver methods -#### `resolver.resolve(binaryBlob, path, callback)` +### cid(binaryBlob, [options]) -> resolves a path in block, returns the value and or a link and the partial missing path. This way the IPLD Resolver can fetch the link and continue to resolve. +> Return the CID of the binary blob. -`callback` must have the signature `function (err, result)`, where `err` is an Error if the function fails and `result` is an object with the following keys: +Possible `options` are: + - `version` (`number`, default: 1): the CID version to be used + - `hashAlg` (`Multicodec`, default: the one the format specifies): the hash algorithm to be used -- value: <> - The value resolved or an IPLD link if it was unable to resolve it through. -- remainderPath: <> - The remaining path that was not resolved under block scope. +This can be used to verify that some data actually has a certain CID. -If `path` is the root `/`, the result is a nested object that contains all paths that `tree()` returns. The values are the same as accessing them directly with the full path. Example: -`tree()` returns: +### toJSON(IpldNode) -```JSON -["author/name", "author/email"] +> Converts an IPLD Node into a JavaScript object that contains only basic types. + +Returns a JavaScript object that can be used as input for `JSON.stringify()`. It is *not* a goal to have a JSON representation that is roundtripable back into an IPLD Node. It is meant as a representation that can be processed by third party consumers. + +The [IPLD Data Model](https://github.com/ipld/specs/blob/master/IPLD-Data-Model-v1.md) defines two special types that need special attention when converting to JSON: + - Binary: the binary data is transformed into a Base64 encoded string. + - Links: the links are CID objects. Consumers of this JSON shouldn’t need to have their own CID parsing implementation, hence the CID is provided in its original base encoded format as well as the human readable one. + +Example with [dag-cbor](https://github.com/ipld/js-ipld-dag-cbor): + +```JavaScript +'use strict' + +const CID = require('cids') +const ipldDagCbor = require('ipld-dag-cbor') + +const input = { + binary: Buffer.from('1155fa3c', 'hex'), + link: new CID('zdpuAxdeot12gCeKJxANaDAL2juLQDB2QK4PFKnnxdAJLpAZf') +} + +const serialized = await ipldDagCbor.serialize(input) +const ipldNode = await ipldDagCbor.deserialize(serialized) +const json = ipldDagCbor.toJSON(ipldNode) +console.log(JSON.stringify(json, null, 2)) ``` -`resolve(binaryblob, "/", callback)` would then have as a result: +The output is: ```JSON { - "author": { - "name": "vmx", - "email": "vmx@example.com" + "binary": "EVX6PA==", + "link": { + "cid": "bafyreifvnutjz6sgkym5cw3fw5e2opfew2gy5dw4wui4tzpphylbmmjsci", + "multibase": "base32", + "version": 1, + "multicodec": "dag-cbor", + "multihash": { + "name": "sha2-256", + "bits": 256, + "digest": "b56d269cfa465619d15b65b749a73ca4b68d8e8edcb511c9e5ef3e1616313212" + } } } ``` -Numbers within a path are interpreted as an array. - -#### `resolver.tree(binaryBlob, callback)` - -> returns all the paths available in this block. - -`callback` must have the signature `function (err, result)`, where `err` is an Error if the function fails and `result` is a list of path such as `["/foo", "/bar", "/author/name", ...]`. ### Properties -#### `defaultHashAlg` +#### `defaultHashCode` + +> Default hash algorithm of the format, -> Default hash algorithm of the format +Most formats have one specific hash algorithm, e.g. Bitcoin’s is `dbl-sha2-256`. CBOR can be used with any hash algorithm, though the default in the IPFS world is `sha256`. `defaultHashAlg` is used in the `cid()` call if no hash algorithm is given. The value of `defaultHashAlg` must be one code defined in the [Multihash Table](https://github.com/multiformats/multihash#table-for-multihash). -Most formats have one specific hash algorithm, e.g. Bitcoin’s is `dbl-sha2-256`. CBOR can be used with any hash algorithm, though the default in the IPFS world is `sha256`. `defaultHashAlg` is used in the `util.cid()` call if no hash algorithm was given. The value of `defaultHashAlg` must be one defined in the [Multihash Table](https://github.com/multiformats/multihash#table-for-multihash-v100-rc-semver). +#### `codec` -#### `multicodec` +> Identifier for the format implementation. -> Identifier for the format implementation +The `codec` property is used to register a format implementation in IPLD. It needs to be one of the codes specified in the [Multicodec Table](https://github.com/multiformats/multicodec#multicodec-table). -The `multicodec` property is used to register a format implementation in IPLD. It needs to be one specified in the [Multicodec Table](https://github.com/multiformats/multicodec#multicodec-table). ## Maintainers From 190de47c07137c890063007fa230a1acddc8574b Mon Sep 17 00:00:00 2001 From: Volker Mische Date: Wed, 28 Nov 2018 11:52:59 +0100 Subject: [PATCH 2/8] Remove `toJSON()` `toJSON()` is out of scope for this spec and can be implemented outside of it. --- README.md | 49 ------------------------------------------------- 1 file changed, 49 deletions(-) diff --git a/README.md b/README.md index 716c905..8212932 100644 --- a/README.md +++ b/README.md @@ -90,55 +90,6 @@ Possible `options` are: This can be used to verify that some data actually has a certain CID. -### toJSON(IpldNode) - -> Converts an IPLD Node into a JavaScript object that contains only basic types. - -Returns a JavaScript object that can be used as input for `JSON.stringify()`. It is *not* a goal to have a JSON representation that is roundtripable back into an IPLD Node. It is meant as a representation that can be processed by third party consumers. - -The [IPLD Data Model](https://github.com/ipld/specs/blob/master/IPLD-Data-Model-v1.md) defines two special types that need special attention when converting to JSON: - - Binary: the binary data is transformed into a Base64 encoded string. - - Links: the links are CID objects. Consumers of this JSON shouldn’t need to have their own CID parsing implementation, hence the CID is provided in its original base encoded format as well as the human readable one. - -Example with [dag-cbor](https://github.com/ipld/js-ipld-dag-cbor): - -```JavaScript -'use strict' - -const CID = require('cids') -const ipldDagCbor = require('ipld-dag-cbor') - -const input = { - binary: Buffer.from('1155fa3c', 'hex'), - link: new CID('zdpuAxdeot12gCeKJxANaDAL2juLQDB2QK4PFKnnxdAJLpAZf') -} - -const serialized = await ipldDagCbor.serialize(input) -const ipldNode = await ipldDagCbor.deserialize(serialized) -const json = ipldDagCbor.toJSON(ipldNode) -console.log(JSON.stringify(json, null, 2)) -``` - -The output is: - -```JSON -{ - "binary": "EVX6PA==", - "link": { - "cid": "bafyreifvnutjz6sgkym5cw3fw5e2opfew2gy5dw4wui4tzpphylbmmjsci", - "multibase": "base32", - "version": 1, - "multicodec": "dag-cbor", - "multihash": { - "name": "sha2-256", - "bits": 256, - "digest": "b56d269cfa465619d15b65b749a73ca4b68d8e8edcb511c9e5ef3e1616313212" - } - } -} -``` - - ### Properties #### `defaultHashCode` From 930398b74d783d72afaf7aadf75967dc6cbc88e6 Mon Sep 17 00:00:00 2001 From: Volker Mische Date: Wed, 28 Nov 2018 14:36:56 +0100 Subject: [PATCH 3/8] Rename properties Renaming properties back to their original names. defaultHashCode -> defaultHashAlg codec - format The intention was to make clear that the type is a multicodec. But this shouldn't be done by the name of the property, but by a type description. Generally we should always use the codes and not the string identifiers of the multicodecs. --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 8212932..bc59087 100644 --- a/README.md +++ b/README.md @@ -92,17 +92,17 @@ This can be used to verify that some data actually has a certain CID. ### Properties -#### `defaultHashCode` +#### `defaultHashAlg` > Default hash algorithm of the format, -Most formats have one specific hash algorithm, e.g. Bitcoin’s is `dbl-sha2-256`. CBOR can be used with any hash algorithm, though the default in the IPFS world is `sha256`. `defaultHashAlg` is used in the `cid()` call if no hash algorithm is given. The value of `defaultHashAlg` must be one code defined in the [Multihash Table](https://github.com/multiformats/multihash#table-for-multihash). +Most formats have one specific hash algorithm, e.g. Bitcoin’s is `dbl-sha2-256`. CBOR can be used with any hash algorithm, though the default in the IPFS world is `sha256`. `defaultHashAlg` is used in the `cid()` call if no hash algorithm is given. The value of `defaultHashAlg` is of type `Multicodec` must be one code defined in the [Multihash Table](https://github.com/multiformats/multihash#table-for-multihash). -#### `codec` +#### `format` > Identifier for the format implementation. -The `codec` property is used to register a format implementation in IPLD. It needs to be one of the codes specified in the [Multicodec Table](https://github.com/multiformats/multicodec#multicodec-table). +The `format` property of type `Multicodec` is used to register a format implementation in IPLD. It needs to be one of the codes specified in the [Multicodec Table](https://github.com/multiformats/multicodec#multicodec-table). ## Maintainers From 888cd27c5912fb4bb56362c7b2d01ca74895ecca Mon Sep 17 00:00:00 2001 From: Volker Mische Date: Wed, 28 Nov 2018 14:43:48 +0100 Subject: [PATCH 4/8] Update TOC --- README.md | 18 +++++++----------- 1 file changed, 7 insertions(+), 11 deletions(-) diff --git a/README.md b/README.md index bc59087..a34037f 100644 --- a/README.md +++ b/README.md @@ -14,16 +14,12 @@ - [Badge](#badge) - [Definitions](#definitions) - [API](#api) - - [IPLD format utils](#ipld-format-utils) - - [`util.serialize(dagNode, callback)`](#utilserializedagnode-callback) - - [`util.deserialize(binaryBlob, callback)`](#utildeserializebinaryblob-callback) - - [`util.cid(binaryBlob[, options], callback)`](#utilcidbinaryblob-options-callback) - - [Local resolver methods](#local-resolver-methods) - - [`resolver.resolve(binaryBlob, path, callback)`](#resolverresolvebinaryblob-path-callback) - - [`resolver.tree(binaryBlob, callback)`](#resolvertreebinaryblob-callback) + - [`serialize(IpldNode)`](#serializeipldnode) + - [`deserialize(binaryBlob)`](#deserializebinaryblob) + - [`cid(binaryBlob, [options])`](#cidbinaryblob-options) - [Properties](#properties) - [`defaultHashAlg`](#defaulthashalg) - - [`multicodec`](#multicodec) + - [`format`](#format) - [Maintainers](#maintainers) - [Contribute](#contribute) - [License](#license) @@ -61,7 +57,7 @@ A valid (read: that follows this interface) IPLD format implementation the follo IPLD Format APIs are restricted to a single IPLD Node, they never access any linked IPLD Nodes. -### serialize(IpldNode) +### `serialize(IpldNode)` > Serializes the internal representation of an IPLD Node into a binary blob. @@ -70,7 +66,7 @@ IPLD Format APIs are restricted to a single IPLD Node, they never access any lin Returns a Promise containing a `Buffer` with the serialized version of the given IPLD Node. -### deserialize(binaryBlob) +### `deserialize(binaryBlob)` > Deserialize into internal representation. @@ -79,7 +75,7 @@ The result is a JavaScript object, which may also be a Proxy object in case the Returns a Promise containing the Javascript object. This object must be able to be serialized with a `serialize()` call. -### cid(binaryBlob, [options]) +### `cid(binaryBlob, [options])` > Return the CID of the binary blob. From 3e79d97bceea51f9712168b92f58a23797ce0f9a Mon Sep 17 00:00:00 2001 From: Volker Mische Date: Thu, 29 Nov 2018 00:16:24 +0100 Subject: [PATCH 5/8] Multicodecs should be in the Multicodecs table Instead of requiring them to be in the table, make it recommendation. So it's still possible to have your custom ones. --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index a34037f..f7639ec 100644 --- a/README.md +++ b/README.md @@ -92,13 +92,13 @@ This can be used to verify that some data actually has a certain CID. > Default hash algorithm of the format, -Most formats have one specific hash algorithm, e.g. Bitcoin’s is `dbl-sha2-256`. CBOR can be used with any hash algorithm, though the default in the IPFS world is `sha256`. `defaultHashAlg` is used in the `cid()` call if no hash algorithm is given. The value of `defaultHashAlg` is of type `Multicodec` must be one code defined in the [Multihash Table](https://github.com/multiformats/multihash#table-for-multihash). +Most formats have one specific hash algorithm, e.g. Bitcoin’s is `dbl-sha2-256`. CBOR can be used with any hash algorithm, though the default in the IPFS world is `sha256`. `defaultHashAlg` is used in the `cid()` call if no hash algorithm is given. The value of `defaultHashAlg` is of type `Multicodec` should be one code defined in the [Multihash Table](https://github.com/multiformats/multihash#table-for-multihash). #### `format` > Identifier for the format implementation. -The `format` property of type `Multicodec` is used to register a format implementation in IPLD. It needs to be one of the codes specified in the [Multicodec Table](https://github.com/multiformats/multicodec#multicodec-table). +The `format` property of type `Multicodec` is used to register a format implementation in IPLD. It should be one of the codes specified in the [Multicodec Table](https://github.com/multiformats/multicodec#multicodec-table). ## Maintainers From 55585d4ecd15a228bf877f098bdd0fcb739b252b Mon Sep 17 00:00:00 2001 From: Volker Mische Date: Thu, 6 Dec 2018 15:46:07 +0100 Subject: [PATCH 6/8] Improve `cid()` documentation a bit --- README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index f7639ec..34003d3 100644 --- a/README.md +++ b/README.md @@ -77,7 +77,7 @@ Returns a Promise containing the Javascript object. This object must be able to ### `cid(binaryBlob, [options])` -> Return the CID of the binary blob. +> Calculate the CID of the binary blob. Possible `options` are: - `version` (`number`, default: 1): the CID version to be used @@ -85,6 +85,8 @@ Possible `options` are: This can be used to verify that some data actually has a certain CID. +Returns a Promise containing the calculated CID of the given binary blob. + ### Properties From f04424d86531746450c6178df399e65f907d95e4 Mon Sep 17 00:00:00 2001 From: Volker Mische Date: Tue, 19 Mar 2019 16:59:38 +0100 Subject: [PATCH 7/8] Replace "block" with "blob" --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 34003d3..c89aa26 100644 --- a/README.md +++ b/README.md @@ -48,7 +48,7 @@ Include this badge in your readme if you make a new module that implements inter ## Definitions -- **dagNode**: The implementation specific representation of a deserialized block. +- **dagNode**: The implementation specific representation of a deserialized blob. ## API @@ -61,7 +61,7 @@ IPLD Format APIs are restricted to a single IPLD Node, they never access any lin > Serializes the internal representation of an IPLD Node into a binary blob. -`IpldNode` is a previously deserialized binary Block. +`IpldNode` is a previously deserialized binary blob. Returns a Promise containing a `Buffer` with the serialized version of the given IPLD Node. From bf2184d4f07abd52091eaae9e832e676e666f3ac Mon Sep 17 00:00:00 2001 From: Volker Mische Date: Tue, 19 Mar 2019 17:00:22 +0100 Subject: [PATCH 8/8] Clarify details about the deserialized object --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index c89aa26..d77f878 100644 --- a/README.md +++ b/README.md @@ -72,6 +72,8 @@ Returns a Promise containing a `Buffer` with the serialized version of the given The result is a JavaScript object, which may also be a Proxy object in case the data shouldn’t be deserialized as a whole. Its fields are the public API that can be resolved through. It’s up to the format to add convenient methods for manipulating the data. The returned object may also be a Proxy object in case the data shouldn’t be deserialized as a whole. +All enumerable properties (the ones that are returned by a `Object.keys()` call) of the deserialized object are considered for resolving IPLD Paths. They must only return values whose type is one of the [IPLD Data Model](https://github.com/ipld/specs/blob/master/IPLD-Data-Model-v1.md). + Returns a Promise containing the Javascript object. This object must be able to be serialized with a `serialize()` call.