archive: Use subscriptions for storageDiff
#161
Conversation
Signed-off-by: Alexandru Vasile <[email protected]>
Signed-off-by: Alexandru Vasile <[email protected]>
Signed-off-by: Alexandru Vasile <[email protected]>
Signed-off-by: Alexandru Vasile <[email protected]>
|
|
||
| Where `subscription` is the value returned by this function, and `result` can be one of: | ||
|
|
||
| ### storageDiff |
There was a problem hiding this comment.
Perhaps just remove this line, since this object is no longer returned by this function?
There was a problem hiding this comment.
I've removed the The JSON object returned by this function has the following format. Dq: is that the line mentioned? 🤔
There was a problem hiding this comment.
LGTM, just a small wording tweak suggestion.
I would still like to see whether @tomaka has any further thoughts, but my general feeling is that this change is worth doing.
I also wonder how consistent this should be with archive_storage. That call contains a paginationStartKey param which I don't fully understand (given it appears to return everything at once). Should it also be a subscription so that it's able to stream the possibly large set of values found from a users request (again, rather than building up such a set in memory and then sending the whole thing)?
Signed-off-by: Alexandru Vasile <[email protected]>
Signed-off-by: Alexandru Vasile <[email protected]>
Co-authored-by: Niklas Adolfsson <[email protected]>
Co-authored-by: Niklas Adolfsson <[email protected]>
|
|
||
| - `items` (optional): Array of objects. The structure of these objects is found below. | ||
| - `previousHash` (optional): String containing a hexadecimal-encoded hash of the header of the block from which the storage difference will be calculated. The `previousHash` must be an ancestor of the provided `hash`. When this parameter is provided, the storage difference is calculated between the `hash` block and the `previousHash` block. If this parameter is not provided, the storage difference is calculated between the `hash` block and the parent of the `hash` block. | ||
|
|
||
| ```json |
There was a problem hiding this comment.
This description of items is now indented and below the previousHash bullet point. Should it be below the items bullet point instead?
(Perhaps move previousHash above items to achieve this, though it makes a bit less sense to me having a "less important" field first!)
|
|
||
| `error` is a human-readable error message indicating why the call has failed. This string isn't meant to be shown to end users, but is for developers to understand the problem. | ||
|
|
||
| No more events will be generated after a `storageDiffError` event. |
There was a problem hiding this comment.
I had a look at chainHead_v1_storage, and this mentions that operationError notificaitons can be generated, but doesn't actually say anything else about them (nor does it say anything about operationStorageDone: it doesn't appear to promise anything about when it will finish ie will it always end with operationStorageDone or will it also end when operationError is encountered).
We might want to allow multiple storageDiffError events to be emitted (in case some keys are inaccessible or something.) and always end with a storageDiffDone event. One way to work out whether to do this would be opening an issue re what the semantics are supposed to be on chainhead_v1_storage since they are unclear (to me at least): maybe we can reach a consensus that multiple errors can be emitted or maybe this isn't necessary / can't happen!
There was a problem hiding this comment.
Aahh, I was wrong: operationError and operationDone are explained in the chainHead_v1_follow docs actually! And indeed, in those, operationError signals completion and nothing else will be generated after this.
For now we could be consistent with this then as we are here. We might want to revisit this and allow multiple errors to be emitted htough if errors accessing storage values can happen in practise! (Alternately we could skip any such values in the implementation if they are inaccessible..)
There was a problem hiding this comment.
This LGTM modulo moving the itesm description back under the items bulelt point (#161 (comment)) :)
Signed-off-by: Alexandru Vasile <[email protected]>
This PR implements the `archive_unstable_storageDiff`. The implementation follows the rpc-v2 spec from: - paritytech/json-rpc-interface-spec#159. - builds on top of paritytech/json-rpc-interface-spec#161 cc @paritytech/subxt-team --------- Signed-off-by: Alexandru Vasile <[email protected]> Co-authored-by: James Wilson <[email protected]>
This PR implements the `archive_unstable_storageDiff`. The implementation follows the rpc-v2 spec from: - paritytech/json-rpc-interface-spec#159. - builds on top of paritytech/json-rpc-interface-spec#161 cc @paritytech/subxt-team --------- Signed-off-by: Alexandru Vasile <[email protected]> Co-authored-by: James Wilson <[email protected]> Signed-off-by: Alexandru Vasile <[email protected]>
This PR modifies the
archive_unstable_storageDiffas a subscription instead of a method.The change mainly leverages subscriptions' backpressure without implementing a low-level API by the RPC crates.
This is a followup from:
cc @paritytech/subxt-team