Add Construction API (formerly Wallet API)

[patrick-ogrady]

Motivation

To support universal writes to any blockchain, we need to introduce a new collection of endpoints to complement our existing Data API ones.

Solution

Endpoints

Add support for generic transaction construction using the new endpoints (which must work entirely offline):

• /construction/derive
• /construction/preprocess
• /construction/payloads
• /construction/combine
• /construction/parse
• /construction/hash

In addition to the existing endpoints (which can be online):

• /construction/metadata
• /construction/submit

Data API Flow

                                  Caller (i.e. Coinbase)                              + Data API Implementation
                                 +------------------------------------------------------------------------------------+
                               X                                                      |
                               X      Get Supported Networks +---------------------------------> /network/list
                               X                                                      |                +
                               X  +-----------+--------------------------------------------------------+
      Get supported networks,  X  |           v                                       |
      their supported options, X  |   Get Network Options +------------------------------------> /network/options
      and their status         X  |                                                   |
                               X  +-----------+                                       |
                               X              v                                       |
                               X      Get Network Status +-------------------------------------> /network/status
                               X                                                      |
                                                                                      |
                                   X                                                  |
                                   X  Get Block +----------------------------------------------> /block
                                   X                                                  |             +
                        +---------+X                +-----------------------------------------------+
                        |          X                v                                 |
Ensure balance computed |          X  [Optional] Get Additional Block Transactions +-----------> /block/transaction
from block operations   |          X                                                  |
equals balance on node  |                                                             |
                        |                                                             |
                        +-----------> Get account balance for each +---------------------------> /account/balance
                                      account seen in a block                         |
                                                                                      |
                                                                                      |
                                   X                                                  |
                                   X  Get Mempool Transactions +-------------------------------> /mempool
      Monitor the mempool for      X                                                  |              +
      broadcast transaction status X                 +-----------------------------------------------+
      and incoming deposits        X                 v                                |
                                   X  Get a Specific Mempool Transaction +---------------------> /mempool/transaction
                                   X                                                  |
                                                                                      +

Construction API Flows

Offline Mode (construction metadata fetched by caller)

                               Caller (i.e. Coinbase)                + Construction API Implementation
                              +-------------------------------------------------------------------------------------------+
                                                                     |
                               Derive Address   +----------------------------> /construction/derive
                               from Public Key                       |
                                                                     |
                             X                                       |
                             X Create Metadata Request +---------------------> /construction/preprocess
                             X (array of operations)                 |                    +
    Get metadata needed      X                                       |                    |
    to construct transaction X            +-----------------------------------------------+
                             X            v                          |
                             X Fetch Online Metadata +-----------------------> /construction/metadata (online)
                             X                                       |
                                                                     |
                             X                                       |
                             X Construct Payloads to Sign +------------------> /construction/payloads
                             X (array of operations)                 |                   +
                             X                                       |                   |
 Create unsigned transaction X          +------------------------------------------------+
                             X          v                            |
                             X Parse Unsigned Transaction +------------------> /construction/parse
                             X to Confirm Correctness                |
                             X                                       |
                                                                     |
                             X                                       |
                             X Sign Payload(s) +-----------------------------> /construction/combine
                             X (using caller's own detached signer)  |                 +
                             X                                       |                 |
   Create signed transaction X         +-----------------------------------------------+
                             X         v                             |
                             X Parse Signed Transaction +--------------------> /construction/parse
                             X to Confirm Correctness                |
                             X                                       |
                                                                     |
                             X                                       |
                             X Get hash of signed transaction +--------------> /construction/hash
Broadcast Signed Transaction X to monitor status                     |
                             X                                       |
                             X Submit Transaction +--------------------------> /construction/submit
                             X                                       |
                                                                     +

Online Mode (construction metadata fetched by Construction API implementation)

                               Caller (i.e. Mobile Wallet)           + Construction API Implementation
                              +-------------------------------------------------------------------------------------------+
                                                                     |
                               Derive Address   +----------------------------> /construction/derive
                               from Public Key                       |
                                                                     |                                  X
                                                                     |                                  X   Fetch metadata needed for
                                                                     |                                  X   construction automatically
                             X                                       |                                  X
                             X Construct Payloads to Sign +------------------> /construction/payloads   X   /construction/preprocess
                             X (array of operations)                 |                   +              X             +
                             X                                       |                   |              X             v
 Create unsigned transaction X          +------------------------------------------------+              X   /construction/metadata
                             X          v                            |                                  X
                             X Parse Unsigned Transaction +------------------> /construction/parse
                             X to Confirm Correctness                |
                             X                                       |
                                                                     |
                             X                                       |
                             X Sign Payload(s) +-----------------------------> /construction/combine
                             X (using caller's own detached signer)  |                 +
                             X                                       |                 |
   Create signed transaction X         +-----------------------------------------------+
                             X         v                             |
                             X Parse Signed Transaction +--------------------> /construction/parse
                             X to Confirm Correctness                |
                             X                                       |
                                                                     |
                             X                                       |
                             X Get hash of signed transaction +--------------> /construction/hash
Broadcast Signed Transaction X to monitor status                     |
                             X                                       |
                             X Submit Transaction +--------------------------> /construction/submit
                             X                                       |
                                                                     +

Future Work

• Resolve other outstanding issues prior to releasing v1.4.0 + cleanup old docs to ensure consistent with new endpoints.

[niallo]

+1000000 🔥

[jakecraige]

I think the way this is documented is a little bit confusing which is my bad for stating it like this in the other doc.

This reads like the public key is 255-bits + 32-bytes which isn't true. It's 255-bits + 1-bit for a total of 32-bytes. It may be worth clarifying that here.

It also could be beneficial to link to documentation on where these encodings are described elsewhere.

For Ed25519, it's page 6 of the paper here: https://ed25519.cr.yp.to/ed25519-20110926.pdf
For secp256k1, the compressed encoding is here: https://secg.org/sec1-v2.pdf#subsubsection.2.3.3

[jakecraige]

Looks to be an extra 1 here before "SEC"

[omarish]

Nicely done! 💪

[patrick-ogrady]

Thanks for these callouts @jakecraige, I'm working on another PR prior to cutting the release and can work these fixes in there!