Starknet WebSocket specification proposal #211

amanusk · 2024-06-30T07:59:36Z

How to read this spec

Each subscription type is described by its own method, with relevant parameters.
The result of the subscription is a subscription ID, used to determine which stream a notification was received from, as well as an option to unsubscribe.
For each subscription type, there is a notification method describing the results received in each streamed event
A Reorg error is possible on all subscriptions, notifying that a reorg has occurred, and it is up to the client to treat this case according to their business logic

This change is

api/starknet_ws_api.json

ittaysw · 2024-06-30T09:47:17Z

api/starknet_ws_api.json

+        "name": "subscription ID",
+        "description": "An identifier for this subscription stream used to associate events with this subscription.",
+        "schema": {
+          "type": "integer"


if integer, it can't be "0x..." form.

ittaysw · 2024-06-30T09:50:10Z

api/starknet_ws_api.json

+          "$ref": "#/components/errors/WEBSOCKET_SUBSCRIPTION_CLOSED"
+        },
+        {
+          "$ref": "#/components/errors/STARKNET_REORG_DETECTED"


Technically an error is something that the callee responds with. In this case the callee is the dapp. How can it detect a reorg and why should it report it to the node (what will the node do with it?)

Indeed the intention was for the node to send the error, not the Dapp. The Dapp needs to act upon this.
The error could be received async, and not in direct response to a request

Open for suggestions for how to best describe this behaviour

I think another type of subscription.

api/starknet_ws_api.json

ittaysw · 2024-06-30T09:56:00Z

api/starknet_ws_api.json

+      "FELT": {
+        "$ref": "./api/starknet_api_openrpc.json#/components/schemas/FELT"
+      },
+      "BLOCK_SYNCING_STATUS": {


This is not compatible in properties name with SYNC_STATUS from the "pull" RPC

You are right, didn't realize it was already defined. Fixing

ittaysw · 2024-06-30T09:57:10Z

api/starknet_ws_api.json

+      }
+    },
+    "errors": {
+      "WEBSOCKET_SUBSCRIPTION_CLOSED": {


Seems like this is returned by the dapp when an event is sent. if so, the node knows the subscription (it is sending it), so why report it back?

As I wrote above, I think this should be removed.

ittaysw · 2024-06-30T09:57:27Z

api/starknet_ws_api.json

+            },
+            "reason": {
+              "title": "Closing reason",
+              "description": "The reason why the subscription was closed",


What will the node do with this?

@kkovaacs I took this error from the Pathfinder WS spec :),
Can you please elaborate on how the "reason" field is used?
Or is this some standard?
Thanks

No, this is an oversight / error in the specification. This is not really an error but a notification (or at least it should be). I think we should just remove this altogether.

ittaysw · 2024-06-30T09:58:57Z

api/starknet_ws_api.json

+          "type": "object",
+          "description": "Data about reorganized blocks",
+          "properties": {
+            "starting_block_hash": {


This is returned by a dapp in the above. It wouldn't know all these details.

Maybe you meant this to be some metadata that a node can send to a dapp? If so it's another subscription type

and seems like this is very common with SYNC_STATUS, so maybe reuse.

Indeed very similar, but there is no use for current_block

@ArielElp WDYT, makes sense to split SYNC_STAUTS, or duplicate?

ittaysw · 2024-06-30T09:59:58Z

api/starknet_ws_api.json

+      ],
+      "result": {
+        "name": "Unsubscription result",
+        "description": "True if the unsubscription was successful",


What should the dapp do if it is not successful?

It's probably an internal error in the dapp I think. The only case Pathfinder returns false here is when you're specifying a non-existent subscription id. We could also just convert this into an error instead.

api/starknet_ws_api.json

ittaysw · 2024-07-02T05:24:51Z

What is the behavior of these subscriptions when a node is syncing? E.g. I subscribed to events, will i get all events as it is going through blocks? A node may be syncing back as well, what is the meaning of 'latest' then? And if two nodes are syncing on the same range, one forward, one backward, will they send different events on the stream?

Seems to me like we should just not allow subscribing when a node is syncing, only when it is fully synced. Then we can also remove the syncing status call.

ittaysw · 2024-07-02T05:17:50Z

api/starknet_ws_api.json

+      ]
+    },
+    {
+      "name": "starknet_subscribePendingTransactions",


maybe instead have a 'block' parameter to starknet_subscribeTransactions which will also be more uniform with starknet_subscribeEvents

This should be an interim solution until PendingTransactions reflects transactions in the mempool.
I don't think it makes sense to stream all transactions if they are part of an already complete block

ittaysw · 2024-07-02T05:19:02Z

api/starknet_ws_api.json

+      ]
+    },
+    {
+      "name": "starknet_subscribeSyncing",


What is the use case here? Do we expect nodes to start syncing, then stop, then start again?

This can be also the stream for reorg notification

Worth a discussion in the nodes-collab meeting. It's currently here since Ethereum has such endpoint.
The usecase I can think of is indeed to only start some process after sync is complete

ArielElp

Reviewable status: 0 of 7 files reviewed, 16 unresolved discussions (waiting on @amanusk and @ittaysw)

api/starknet_ws_api.json line 325 at r1 (raw file):

Previously, ittaysw (Ittay Dror) wrote…

Yes

Why not have the response of tx status subscription be allOf tx_hash and TXN_STAUTS? NEW_TXN_STATUS is weird.

api/starknet_ws_api.json line 380 at r1 (raw file):

Previously, amanusk (amanusk) wrote…

Indeed very similar, but there is no use for current_block

@ArielElp WDYT, makes sense to split SYNC_STAUTS, or duplicate?

I think we can keep as-is, but up to you

api/starknet_ws_api.json line 59 at r3 (raw file):

          "required": false,
          "schema": {
            "$ref": "#/components/schemas/FELT"

ADDRESS

api/starknet_ws_api.json line 63 at r3 (raw file):

        },
        {
          "name": "keys",

keys is a nested array, e.g. [[],[1, 2], [3]], means (k1=1 or k1=2) and k3=3, with no constraints on k0

api/starknet_ws_api.json line 105 at r3 (raw file):

          "name": "result",
          "schema": {
            "$ref": "./api/starknet_api_openrpc.json#/components/schemas/EMITTED_EVENT"

am I necessarily getting events one at a time? does an array of EMITTED_EVENT make sense here?

api/starknet_ws_api.json line 200 at r3 (raw file):

          "name": "result",
          "description": "Either a tranasaction hash or full transaction details, based on subscription",
          "schema": {

same as my events question, are we insisting on one item or can we can a chunk of results?

api/starknet_ws_api.json

kirugan · 2024-07-04T10:02:54Z

api/starknet_ws_api.json

+        }
+      },
+      "STARKNET_REORG_DETECTED": {
+        "code": 202,


if we following json-rpc spec (which might be the case because WEBSOCKET_SUBSCRIPTION_CLOSED code is in valid range) then we should follow it here as well:

-32000 to -32099 Server error Reserved for implementation-defined server-errors.

ittaysw · 2024-07-04T13:05:53Z

api/starknet_ws_api.json

+      ]
+    },
+    {
+      "name": "starknet_subscribeTransactionStatus",


do we want a TTL here? E.g. maybe the tx is not known (was dropped from the mempool, deemed invalid, in some old block, typo in hash), and the subscription will stay forever?

That's a great point, can we express this in spec somehow? Or just in the description?

ittaysw · 2024-07-04T13:08:26Z

api/starknet_ws_api.json

+  },
+  "methods": [
+    {
+      "name": "starknet_subscribeNewHeads",


For all subscriptions: Say the server wants to drop a subscription (to conserve space, because maybe it suspects they are not listened to), without closing the websocket, should there be a message to state this, so the client can re-register?

ittaysw · 2024-07-04T13:09:46Z

api/starknet_ws_api.json

+      "name": "starknet_subscribeNewHeads",
+      "summary": "New block headers subscription",
+      "description": "Creates a WebSocket stream which will fire events for new block headers",
+      "params": [],


perhaps allow a block number? E.g. say a client got disconnected and now connects the websocket again and wants to receive all interim blocks? Maybe an 'n' for 'n latest'?

ittaysw · 2024-07-04T13:10:23Z

api/starknet_ws_api.json

+          "required": false,
+          "schema": {
+            "type": "string",
+            "enum": ["pending", "latest"]


similar to new heads, maybe there should be an 'n' here to close gaps?

api/starknet_ws_api.json

amanusk · 2024-08-12T08:32:06Z

Latest changes based on discussions above

Add sender_address filter to pending transactions subscriptions
Add subscriptionReorg that can be received by Headers, Events, Transaction Status
Remove sync subscription and events
Nodes only allow WS sucscriptions after they are synced
Updated keys as in event filter

Open question:

Allowing to start a subscription from a previous block and "catch up" to the latest (applies to headers, events, transaction Status)
Returning subscription notifications one by one or as an array (applies to Headers, Events, Pending Transactions)

These are also questions for Full nodes and how much does this complicate implementation

api/starknet_ws_api.json

kkovaacs · 2024-08-14T15:12:19Z

api/starknet_ws_api.json

+      ],
+      "result": {
+        "name": "Unsubscription result",
+        "description": "True if the unsubscription was successful",


It's probably an internal error in the dapp I think. The only case Pathfinder returns false here is when you're specifying a non-existent subscription id. We could also just convert this into an error instead.

kkovaacs · 2024-08-14T15:12:34Z

api/starknet_ws_api.json

+      }
+    },
+    "errors": {
+      "WEBSOCKET_SUBSCRIPTION_CLOSED": {


As I wrote above, I think this should be removed.

kkovaacs · 2024-08-14T15:13:26Z

api/starknet_ws_api.json

+            },
+            "reason": {
+              "title": "Closing reason",
+              "description": "The reason why the subscription was closed",


No, this is an oversight / error in the specification. This is not really an error but a notification (or at least it should be). I think we should just remove this altogether.

amanusk · 2024-08-25T15:02:12Z

Update:
Added the option to subscribe to up to 1024 blocks back, and the accompanying error

api/starknet_ws_api.json

sistemd · 2024-09-04T13:23:09Z

api/starknet_ws_api.json

+        },
+        {
+          "name": "block",
+          "summary": "The block to get notifications from, default is latest, limited to 1024 blocks back",


Why is there a limit in the spec? We can let nodes determine their own limits if they need to, though from pathfinder's point of view it's probably fine to do this without a limit - this can simplify some of the client code.

I see there are similar limits in other endpoints - maybe we should leave these up to nodes, if they want to limit users at all?

I tend to agree that we can leave it up to nodes, I guess the original motivation was to avoid confusion on different rpc providers supporting different things. For now I'm leaving this in so that it can be part of 0.8.0-rc0, we can continue discussing this towards the final release.

api/starknet_ws_api.json

ArielElp

Reviewed 1 of 6 files at r1, 1 of 2 files at r2, 3 of 4 files at r4, 1 of 2 files at r5, 1 of 1 files at r6, 1 of 1 files at r7, 4 of 4 files at r8, all commit messages.
Reviewable status: all files reviewed, 28 unresolved discussions (waiting on @amanusk, @ittaysw, @kirugan, @kkovaacs, and @sistemd)

ArielElp

Reviewable status: all files reviewed, 24 unresolved discussions (waiting on @amanusk, @ittaysw, @kirugan, @kkovaacs, and @sistemd)

ArielElp

Reviewable status: all files reviewed, 23 unresolved discussions (waiting on @amanusk, @ittaysw, @kirugan, @kkovaacs, and @sistemd)

ArielElp

Dismissed @sistemd, and @sistemd from 23 discussions.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on @amanusk)

amanusk requested review from ArielElp and ittaysw June 30, 2024 08:00

ittaysw requested changes Jun 30, 2024

View reviewed changes

amanusk marked this pull request as draft July 1, 2024 15:13

ittaysw reviewed Jul 2, 2024

View reviewed changes

amanusk changed the title ~~Starknet WS specification proposal~~ Starknet WebSocket specification proposal Jul 2, 2024

ArielElp reviewed Jul 4, 2024

View reviewed changes

kirugan reviewed Jul 4, 2024

View reviewed changes

api/starknet_ws_api.json Show resolved Hide resolved

kirugan reviewed Jul 4, 2024

View reviewed changes

ittaysw requested changes Jul 4, 2024

View reviewed changes

kkovaacs reviewed Jul 4, 2024

View reviewed changes

api/starknet_ws_api.json Outdated Show resolved Hide resolved

ArielElp changed the base branch from master to v0.8.0 July 25, 2024 10:36

kkovaacs reviewed Aug 14, 2024

View reviewed changes

amanusk marked this pull request as ready for review August 25, 2024 15:02

sistemd reviewed Sep 4, 2024

View reviewed changes

api/starknet_ws_api.json Outdated Show resolved Hide resolved

sistemd reviewed Sep 4, 2024

View reviewed changes

sistemd reviewed Sep 16, 2024

View reviewed changes

api/starknet_ws_api.json Outdated Show resolved Hide resolved

sistemd mentioned this pull request Sep 18, 2024

feat(rpc): reorgs in starknet_subscribeNewHeads eqlabs/pathfinder#2249

Merged

drafting ws api

997b7f4

ArielElp force-pushed the starknet-ws branch from bec0b52 to 997b7f4 Compare September 19, 2024 09:48

ArielElp approved these changes Sep 19, 2024

View reviewed changes

ArielElp merged commit 497a7be into starkware-libs:v0.8.0 Sep 19, 2024

franciszekjob mentioned this pull request Oct 22, 2024

Support RPC 0.8.0 software-mansion/starknet-jvm#530

Draft

1 task

Starknet WebSocket specification proposal #211

Starknet WebSocket specification proposal #211

Conversation

amanusk commented Jun 30, 2024 • edited by ArielElp Loading

How to read this spec

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ittaysw commented Jul 2, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ArielElp left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

amanusk commented Aug 12, 2024

Latest changes based on discussions above

Open question:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

amanusk commented Aug 25, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ArielElp left a comment

Choose a reason for hiding this comment

ArielElp left a comment

Choose a reason for hiding this comment

ArielElp left a comment

Choose a reason for hiding this comment

ArielElp left a comment

Choose a reason for hiding this comment

amanusk commented Jun 30, 2024 •

edited by ArielElp

Loading