Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added the first draft of Route Broadcasting Protocol #455

Closed
Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
9d97a53
Added first draft of Route Broadcasting Protocol
dora-gt Aug 2, 2018
c993f9e
added a link for uuid
dora-gt Aug 13, 2018
877cec2
Merge remote-tracking branch 'upstream/master' into feature/dora-gt/r…
dora-gt Aug 21, 2018
da61857
fix to make the roles clear
dora-gt Aug 21, 2018
ecedc96
added explanation of link-relation and routing
dora-gt Aug 22, 2018
b779122
fix
dora-gt Aug 22, 2018
1cf2355
fixed comments on asn.1
dora-gt Aug 22, 2018
fba9c24
fixed, the same modification as ILDCP
dora-gt Aug 22, 2018
2039c23
fixed ASN.1
dora-gt Aug 22, 2018
0ad41d3
added statements of meta
dora-gt Aug 23, 2018
b65f3b4
added `exclusive`
dora-gt Aug 23, 2018
c757046
fixed, and -> to
dora-gt Aug 23, 2018
b451590
added prepositional phrase
dora-gt Aug 23, 2018
d4a5b5f
Merge branch 'master' into feature/dora-gt/route-broadcasting-protocol
adrianhopebailie Aug 27, 2018
cf5b912
added explanation of routing
dora-gt Aug 30, 2018
eb244ad
Merge remote-tracking branch 'upstream/master' into feature/dora-gt/r…
dora-gt Nov 17, 2018
674df80
addressed Stefan's input.
dora-gt Nov 17, 2018
a65cc9f
addressed Stefan's input.
dora-gt Nov 17, 2018
f28619b
addressed Stefan's input.
dora-gt Nov 17, 2018
1afa81d
counter -> counterpart
dora-gt Nov 17, 2018
12544a6
added 3 routing table types
dora-gt Nov 24, 2018
f67c440
added an explanation for the sequence of a path
dora-gt Nov 24, 2018
5421b39
added an explanation of the security
dora-gt Nov 24, 2018
25e17ea
removed wrong explanation
dora-gt Feb 13, 2019
e02a1b7
removed wrong explanation
dora-gt Feb 13, 2019
33b4586
Merge branch 'master' into feature/dora-gt/route-broadcasting-protocol
adrianhopebailie Feb 15, 2019
55da909
Merge branch 'master' into feature/dora-gt/route-broadcasting-protocol
adrianhopebailie May 9, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
226 changes: 226 additions & 0 deletions 0000-route-broadcasting-protocol/0000-route-broadcasting-protocol.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,226 @@
---
title: Route Broadcasting Protocol
draft: 1
---

# Route Broadcasting Protocol (RBP)

## Prerequisites
This specification assumes the reader is familiar with the following documents:

- [Interledger Architecture](../0001-interledger-architecture/0001-interledger-architecture.md)
- [Interledger Protocol V4 (ILPv4)](../0027-interledger-protocol-4/0027-interledger-protocol-4.md)
- [Bilateral Transfer Protocol](../0023-bilateral-transfer-protocol/0023-bilateral-transfer-protocol.md)
- RBP information is sent in ILP packets, and ILP packets are transferred in BTP packets.
- [A Border Gateway Protocol 4 (BGP-4)](https://tools.ietf.org/html/rfc4271)
- The functionality of Route Broadcasting Protocol is similar to that of BGP.

## Terminology
- A **node** is a participant in an Interledger network. It may be a [connector](../0001-interledger-architecture/0001-interledger-architecture.md#connectors), a sender or a receiver.
- A **hop** is going over a boundary to another space, or is the destination space itself. For instance, sending a packet from a node to another node is one hop.
- An **ILP Address** is an identifier of a node or an account held by a node, that is different from an [IP address](https://tools.ietf.org/html/rfc791) or a [domain name](https://tools.ietf.org/html/rfc1035). Refer to [ILP Addresses - v2.0.0](../0015-ilp-addresses/0015-ilp-addresses.md) for more.
- A **route** is a path: a series of nodes that a packet goes through.

## Scope
In this document, the routing logic, how to determine the next hop of ILP packets is out of scope while the focus is placed on how routing information, that is used to decide how ILP packets are routed, is transferred from a node to the other nodes.

## Overview
Interledger Protocol is a protocol suite that consists of several protocols including [Bilateral Transfer Protocol](../0023-bilateral-transfer-protocol/0023-bilateral-transfer-protocol.md), [Interledger Dynamic Configuration Protocol](../0031-dynamic-configuration-protocol/0031-dynamic-configuration-protocol.md), and the other protocols. Route Broadcasting Protocol is one of them.

In short, **Route Broadcasting Protocol is a protocol for transferring routing information from a node to the other nodes in ILP packets**. The routing information roughly includes the following elements (refer to [Protocol Detail](#protocol-detail) for more).

- Prefix (string)
- e.g. `g.node-c`
- The prefix that this route can finally reach.
- Path (an array of string)
- e.g. `g.node-a` `g.node-b` `g.node-c`
- The hops that the packet will go through.
- A packet will be sent from left node to right node in order.

The node receives and utilizes these to determine which node ILP packets should be routed through, building a **routing table** that is explained later in the [Routing Table](#routing-table) section.

## Routing
### Routing Table
Interledger network works like the Internet; ILP nodes route ILP packet among the nodes while routers of the Internet route its packets among ASs ([Autonomous System](https://tools.ietf.org/html/rfc1930)). A node can be connected to multiple nodes like a web.

![ILP Network](images/nodes.svg)

Because a node has to route ILP packets to a certain next node appropriately depending on the destination address, so that the packets finally reach the specified address, the node needs to know which node can be the most efficient first hop for the destination ILP address. Therefore a node MUST know the mapping of:

- Key
- `Prefix`: The ILP address prefix that this route can handle.
- Value
- Route information
- `Next hop`: An identifier of the next node that is connected to the node directly.
- `Path`: How many hops the ILP packets will go through if sent. Generally, the number of elements of the path is expected to be less for efficiency because hops possibly cost network and server burden, or relaying fee.

This mapping is called a **routing table**. In order to build a routing table, a node requests routing information to connected nodes respectively, or a node could possibly load static route configuration from a file.

*A routing table may be a mere list that contains `Prefix`, `Next hop`, and `Path`, it doesn't need to be a map necessarily. That said, a map is more suitable for this case than a list because of the efficiency of looking up.*

A node sends an ILP packet to the next node that is determined by referencing the routing table **of the sending node** and the further hops are NOT determined at that time. The next node determines, referencing **its own routing table**, which node could be the best when received the ILP packet. This chain of discrete decisions of routes is the basis of the routing of the Interledger network.

### Routing Table Lifecycle
The lifecycle of a routing table is like the below:

- Created
- The table is empty at this point
- Updated repeatedly
- By static configuration, though this is usually done only once when the node starts up
- By updates from the other nodes
- Disposed
- When the node is shutdown
- When the node is restarted

### Routing Table ID
A routing table has its identifier, that is called **routing table ID**. A routing table ID is used for identifying its instance, and the ID is given when the table is created as empty. A routing table ID is a [UUID v4](https://tools.ietf.org/html/rfc4122#section-4.1.3).

### Epoch
Every time the routing table is modified, the update is logged and the revision number of the table is increased. The revision number of the table is called an **epoch**.

A node maintains the epoch that is already known to the counter node, and considers it when requesting route updates so that the node can send only the difference (a stack of adding or withdrawing routes) from the last update with the last known epoch as the starting point. So the node doesn't need to send the whole routing table.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we use the term "counterpart node", let's use it here as well, instead of "counter node".

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the terminology should be unified, thank you.


Because the logs of difference could be a large number in some cases, the node may send updates separately multiple times. In that case, "from" and "to" epoch will be specified to express the range of the logs.

![Epoch Logs and Its Update Request](./images/epoch.svg)

The left image shows how epochs are stored. The right image shows an example of a route update request, how "from" and "to" epoch index are given, and how epochs are extracted from the logs.

If the routing table ID that an epoch is based on is changed, the epoch MUST be reset to `0`, so that the counter node can track all the updates from the node.

### Link Relations and Routing
There are two types of link relations:
- `Peer` to `Peer`
- `Parent` to `Child`

In a `Parent` to `Child` relation, the ILP address of the child node is configured automatically using [IL-DCP](../0031-dynamic-configuration-protocol/0031-dynamic-configuration-protocol.md), and generally it is a sub-address of the parent node though it is not a requirement. For instance:
- `Parent` node
- `g.parent-node`
- `Child` node
- `g.parent-node.child-node`
- this is a sub-address of `g.parent-node`

In this case, the parent node already knows the address of the child node, so the routing is a truism. The parent node knows when to transfer ILP packets to the child node, and the child node transfers ILP packets basically to the parent node because it is the gateway to the other addresses for the child node.

On the other hand, in a `Peer` to `Peer` relation, the two nodes have different addresses respectively, and the further connected addresses are unclear. So in this case, it is generally helpful to configure to exchange routing information each other.

Conclusively, the following is the general configuration of nodes depending on the relations.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/general/default/

- `Peer` to `Peer`
- Configured to exchange routing information.
- Does send route control requests and route update requests that are explained later in the [Requests](#requests) section.
- `Parent` to `Child`
- Configured **NOT** to exchange routing information.
- Does **NOT** send route control requests and route update requests.
- That said, the nodes MAY be configured to exchange routing information for some specific situations.

### Node Finite State Machine (FSM)
A node is kind of an FSM that has a status of either:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be clearer that the IDLE/SYNC status is per counterpart.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would avoid the wording "kind of" which is imprecise. Would say something like: "Each node consists of a state machine which tracks for each routing counterpart node:"

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, my bad. Thank you for the input.


- IDLE
- Does **NOT** send routing information to the counterpart node.
- SYNC
- Does send route routing information to the counterpart node.

The status can be changed by a **route control request** that is described below. The routing information is provided by a **route update request** that is described below as well.

## Protocol Detail
### Operation Model
The entire operation model is shown below.

#### Requests
- `Route Control Request`
- A request that ask the receiver node to be a specified state ([IDLE or SYNC](#node-finite-state-machine-fsm)).
- `Route Update Request`
- A request that conveys routing information and ask the receiver node to update its routing table based on the information.

#### Nodes
- `Node A`
- Connected to `Node B`.
- A node that wants routing information from `Node B` to build its routing table.
- `Node A` wants `Node B` to send route update requests that convey routing information.
- `Node B`
- Connected to `Node A`.
- A node whose status is IDLE at this point that means this node doesn't send any route update request to `Node A`.

#### Procedure
This is a normal procedure for instance.

- `Node A` sends a **route control request** of `SYNC` to `Node B`
- `Node B` receives the route control request and updates its status to `SYNC`
- This means that from hence, `Node B` sends route update requests to provide routing information for `Node A`
- `Node B` responds that the state is correctly updated to `Node A`
- (some time after the procedure above)
- `Node B` sends a **route update request** that has routing information (some logs) to `Node A`
- `Node A` receives the route update request, and updates its routing table based on the provided information.
- `Node A` responds that the routing table is correctly updated to `Node B`

#### Packet
The request and the response above are transferred in [ILP packets](../0027-interledger-protocol-4/0027-interledger-protocol-4.md#specification).

The `fulfillment` of the response packet is always a zero-filled 32 byte octet string, therefor the condition is always the SHA-256 hash digest of that, i.e. the Base64 decoded value of `Zmh6rfhivXdsj8GLjp+OIAiXFIVu4jOzkCpZHQ1fKSU=`.

As with other bilateral protocols the packets are addressed directly to the peer using the `peer.` address prefix. RBP packets are specifically identified using the address `peer.route`.

All current implementations of RBP default to an amount of `0` in the ILP packets used to request route control or update.

### Route Control
#### Procedure
Route control is done in the following procedure:

- A node requests IDLE or SYNC mode to the corresponded node.
- The corresponded node changes its status if needed, and respond that the request is done.
- If the receiving node is not configured to send route updates, the receiving node responds with an error.
- If the request cannot be deserialized or interpreted as appropriate, the corresponded node responds with an error.

#### Packet
- Request
- The `type` of the ILP packet is `ILP Prepare` (type id: 12)
- The `amount` of the ILP packet is `0`
- The `expiresAt` of the ILP packet is arbitrary
- The `executionCondition` of the ILP packet is `Zmh6rfhivXdsj8GLjp+OIAiXFIVu4jOzkCpZHQ1fKSU=` in Base64 format
- The `destination` address of the ILP packet is `peer.route.control`
- The `data` of the ILP packet is described at `RouteControlRequestData` in [ASN.1 definition](#asn1-definition)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For forward compatibility, nodes should ignore any extra data that they do not understand.

- Response
- The `type` of the ILP packet is `ILP Fulfill` (type id: 13)
- The `fulfillment` of the ILP packet is 32byte octet string all filled with zeros
- The `data` of the ILP packet is empty (size: 0)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For forward compatibility, nodes should ignore if the packet data is non-empty.

- Error
- The `type` of the ILP packet is `ILP Reject` (type id: 14)
- The `code` of the ILP packet is arbitrary, depending on the situation
- The `message` of the ILP packet is arbitrary, depending on the situation
- The `triggeredBy` of the ILP packet is the ILP address of the node where the error occuered
- The `data` of the ILP packet is empty (size: 0)

### Route Update
#### Procedure
Route Update is done in the following procedure:

- A node sends a route update request to the corresponded node with information of route update logs.
- The receiver node updates its routing table, and responds that the request is done.
- If the receiving node is not configured to receive route updates, the receiving node responds with an error.
- If the request cannot be deserialized or interpreted as appropriate, the corresponded node responds with an error.

#### Packet
- Request
- The `type` of the ILP packet is `ILP Prepare` (type id: 12)
- The `amount` of the ILP packet is `0`
- The `expiresAt` of the ILP packet is arbitrary
- The `executionCondition` of the ILP packet is `Zmh6rfhivXdsj8GLjp+OIAiXFIVu4jOzkCpZHQ1fKSU=` in Base64 format
- The `destination` address of the ILP packet is `peer.route.update`
- The `data` of the ILP packet is described at `RouteUpdateRequestData` in [ASN.1 definition](#asn1-definition)
- Response
- The `type` of the ILP packet is `ILP Fulfill` (type id: 13)
- The `fulfillment` of the ILP packet is 32byte octet string all filled with zeros
- The `data` of the ILP packet is empty (size: 0)
- Error
- The `type` of the ILP packet is `ILP Reject` (type id: 14)
- The `code` of the ILP packet is arbitrary, depending on the situation
- The `message` of the ILP packet is arbitrary, depending on the situation
- The `triggeredBy` of the ILP packet is the ILP address of the node where the error occuered
- The `data` of the ILP packet is empty (size: 0)

### ASN.1 Definition
The ASN.1 definition of ILP packets is described in [InterledgerProtocol.asn](../asn1/InterledgerProtocol.asn) and Route Broadcasting Protocol data is in [RouteBroadcastingProtocol.asn](./RouteBroadcastingProtocol.asn).

### Encoding Rule
All ASN.1 types are encoded using [Octet Encoding Rules](../0030-notes-on-oer-encoding/0030-notes-on-oer-encoding.md) as is the norm with all Interledger protocols.
68 changes: 68 additions & 0 deletions 0000-route-broadcasting-protocol/RouteBroadcastingProtocol.asn
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
RouteBroadcastingProtocol
DEFINITIONS
AUTOMATIC TAGS ::=
BEGIN

IMPORTS
UInt8,
UInt16,
UInt32
FROM GenericTypes
Address
FROM InterledgerTypes
;

-- Route Control Request

SyncMode ::= INTEGER {
idle (0),
sync (1)
} (0..255)

RoutingTableId ::= OCTET STRING (SIZE (16))

Feature ::= OCTET STRING

RouteControlRequestData ::= SEQUENCE {
syncMode SyncMode, -- the state that the node wants the counterpart node to be
routingTableId RoutingTableId, -- the routing table ID that the requesting node knows at that point
knownEpochIndex UInt32, -- the epoch that the requesting node knows at that point
features SEQUENCE OF Feature -- reserved for the future, currently not used
}

-- Route Update Request

Auth ::= OCTET STRING (SIZE (32))

-- meta (UInt8) is used as flags, and is reserved for the future.
-- - 0x80 (1000 0000) : isOptional
-- - 0x40 (0100 0000) : isTransitive
-- - 0x20 (0010 0000) : isPartial
-- - 0x10 (0001 0000) : isUtf8
Property ::= SEQUENCE {
meta UInt8,
Copy link
Contributor

@sappenin sappenin Aug 20, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be worth defining the meaning of meta. See ilp-protocol-ccp for some examples of the data encoded into this byte field.

From what I can tell, the following fields exist: optional, transitive, partial, isUtf8. It's unclear, however, from the source code in ilp-protocol-ccp what each of these mean.

@adrianhopebailie, @justmoon for more details?

Copy link
Collaborator Author

@dora-gt dora-gt Aug 22, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually knew the options but I hesitated writing it because it seemed that the options were not used. Should I write it nevertheless?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be fine if these definitions were written later (e.g., after merging this PR), although the terms are used in the source-code (though undocumented), and so it would be good to outline them in the event some implementation wanted to start using them.

Copy link
Collaborator Author

@dora-gt dora-gt Aug 23, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added some statements about meta.
I can imagine what these flags mean, but those are not certain, so I didn't write the exact meaning of the flags at this point.

id UInt16,
value OCTET STRING
}

NewRoute ::= SEQUENCE {
addedRoutePrefix Address, -- the prefix of the added route
path SEQUENCE OF Address, -- the hops destinated to the prefix node
auth Auth, -- reserved for the future, currently not used
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not just reserved for the future! Routes are not yet filtered based on this but correct implementations will generate and correctly update the auth field.

Basically, the node originating a route will use a random value. Each node forwarding a route will SHA-256 hash this value.

It's not as good as BGPsec or other heavier schemes for routing security but it is better than nothing. An attacker can choose not to hash the auth value and pass it on unchanged, to make themselves look one hop closer than they are. But they cannot reverse the hash to make themselves look two or more hops closer. They can use an unrelated hash chain but connectors could be set such that this triggers a manual review.

In the future, we may add a routing PKI system to establish the correct base hash so that every knows which is the correct hash chain. Beyond that we may add full BGPsec style signing.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kruisdraad was discussing authenticated route updates in interledger/network#4 (comment)

My proposal to him was to look at using Distributed IDentifiers (DIDs) as a way to identify nodes and to root these in the distributed ledgers that we already use for settlement.

I have an XRP-biased proposal that could be adapted to be more generic: https://github.com/interledger/network/blob/master/docs/xrpl-pki.md

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could not understand the whole security model of auth.
How does the receiver verify the auth value?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think what makes ILP beautiful is its independency. So I think it needs careful discussions to use some ledgers for the base of identities, that makes ILP dependent.

props SEQUENCE OF Property -- reserved for the future, currently not used
}

-- NOTICE: The log of "toEpochIndex" will NOT be included.
-- e.g. with this log: [ a, b, c, d ], update request from 0 to 2 means [ a, b ]
RouteUpdateRequestData ::= SEQUENCE {
routingTableId RoutingTableId, -- the routing table ID that the requesting node knows at that point
currentEpochIndex UInt32, -- current epoch index of the node that requests the update for the other nodes
fromEpochIndex UInt32, -- the epoch index that the log of this update request starts from
toEpochIndex UInt32, -- the epoch index that the log of this update request ends at, exclusive
holdDownTime UInt32, -- reserved for the future, currently not used
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Despite not being used, can we document what this means?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to specify, but I could not find any reference to determine the function.
@justmoon Could you give me some reference?

speaker Address, -- the ILP address of the requesting node
addedRoutes SEQUENCE OF NewRoute, -- new routes that the speaker is advertising to the counterpart node
withdrawnRoutes SEQUENCE OF Address -- withdrawn routes that the speaker is advertising as being no longer valid or useful
}

END
Loading