lnd v0.3-alpha

This release marks the 3rd major release for lnd! With this release, lnd is now has gained a considerable feature set, a new automatic channel management operating mode, RPC authentication, additional persistence logic, and further major strides towards complete spec compliance.

This release lands on the eve of the activation of segwit on Bitcoin's mainnet 🎉! As the developers of lnd, we're extremely excited to see Lightning integrated into the ecosystem, and to see all the novel applications that developers will buildout! However, we recommend that users of lnd do not yet attempt to use the software on the live network with real funds. We're getting very close to enabling a mainnet mode usage, but at this point necessary safety and fault tolerance measures aren't yet in place. With our next release: lnd v0.4-beta, we'll be targeting full spec compliance, cross-implementation interoperability, and the additional assurance and safety measures we deem necessary before adding a mainnet mode to the software.

NOTE: It is important to note that this release of lnd contains several breaking changes. As a result, users will either need to upgrade using a completely fresh installation, or remove their existing channel.db database file before upgrading. As a courtesy, we recommend that users close out their prior channels (either cooperatively if the peer is online, or unilaterally (force close) otherwise) before upgrading.

⚡️⚡️⚡️ OK, now to the rest of the release notes! ⚡️⚡️⚡️

Notable Changes

TLS Integration and Macaroon Based RPC Authentication

In this release of lnd, we now enforce the usage of TLS by default when communicating with the RPC server either via gRPC or the HTTP interface. In addition the the server authentication, we now also expose per-call RPC authentication. This mode is on by default, and can be disabled with the --no-macaroonsflag on the command line, or the corresponding line within ones lnd.conf configuration file.

For per-call RPC authentication, lnd currently uses macaroons. Macaroons are bearer credentials that are based on a construction of nested HMAC calls with a single root IV key. The root key is kept secret by the server and allows it to mint (or bake) new macaroons and also authenticate the validity of presented macaroons. Macaroons are bearer credentials meaning presenting or holding the macaroon entitles the holder to a specific set of capabilities. Macaroons are very flexible and allow the holder of a particular macaroon to add additional caveats which contextually confine the set of capabilities unlocked by the macaroon. Currently we only expose two flavors of macaroons: an admin macaroon which allows unrestricted access to all RPC calls, and a read-only macaroon which is created by adding caveats to the admin macaroon dictating which calls the holder of the macaroon can access.

In the future, we'll be adding additional granularity of the usage of macaroons within lnd. We believe tha macaroons are very exciting as they enable a very flexible mechanism of authentication which can be used to build applications upon lnd which have the most minimal set of capabilities possible. For example, imagine an application built on top of lnd which only needs access to a particular channel, and will only be sending payments below N satoshis over said channel. The flexibility of the macaroon scheme makes it easy for lnd to bake such a macaroon and present it to the application in question.

Litecoin Operating Mode

In this release, we've added support for a toggle-able Litecoin chain and wallet backend. This means that lnd is now capable for easily switching back and forth between the Bitcoin and Litecoin chains. This is very exciting as the code scaffolding put in place to make this switch seamless also lays some important ground work for the multi-chain lnd operating mode that is under development. With multi-chain, lnd will be able to manage channels on both Bitcoin and Litecoin simultaneously. In the near future, this means that lnd nodes with channels on both chains will be able to acts as a border node between the two networks, facilitating instant swaps and transfers between the two chains.

The daemon backend configuration options have been split, into two mirroring option classes for both Bitcoin and Litecoin:

Litecoin:
      --litecoin.active        If the chain should be active or not.
      --litecoin.chaindir=     The directory to store the chains's data within.
      --litecoin.rpchost=      The daemon's rpc listening address. If a port is omitted, then the default port for the selected chain parameters will be used. (localhost)
      --litecoin.rpcuser=      Username for RPC connections
      --litecoin.rpcpass=      Password for RPC connections
      --litecoin.rpccert=      File containing the daemon's certificate file (/Users/roasbeef/Library/Application Support/Ltcd/rpc.cert)
      --litecoin.rawrpccert=   The raw bytes of the daemon's PEM-encoded certificate chain which will be used to authenticate the RPC connection.
      --litecoin.testnet       Use the test network
      --litecoin.simnet        Use the simulation test network
      --litecoin.regtest       Use the regression test network

Bitcoin:
      --bitcoin.active         If the chain should be active or not.
      --bitcoin.chaindir=      The directory to store the chains's data within.
      --bitcoin.rpchost=       The daemon's rpc listening address. If a port is omitted, then the default port for the selected chain parameters will be used. (localhost)
      --bitcoin.rpcuser=       Username for RPC connections
      --bitcoin.rpcpass=       Password for RPC connections
      --bitcoin.rpccert=       File containing the daemon's certificate file (/Users/roasbeef/Library/Application Support/Btcd/rpc.cert)
      --bitcoin.rawrpccert=    The raw bytes of the daemon's PEM-encoded certificate chain which will be used to authenticate the RPC connection.
      --bitcoin.testnet        Use the test network
      --bitcoin.simnet         Use the simulation test network
      --bitcoin.regtest        Use the regression test network

Our docker configuration has been updated to allow users to spin up a cluster of lnd nodes on simnet or testnet of either Bitcoin or Litecoin.

autopilot: Self-Driving Lightning

This release also includes the first of many planned flavors of automatic channel management. We call this new operating node autopilot as if set, will automatically manage the opening of channels within the network. This operating mode is attractive is it enables a new level of plug-and-play interaction with the daemon, and can also be used in aggregate to tend the ultimate topology of the channel graph to one of a set of heuristically determined characteristics.

The autopilot operating mode is essentially a closed-loop control system: it takes in outside input such as the number of channels opened, when channels are closed, changes in the wallet balance and applies that to its internal state. Once outside signals are received, it then consults it's registered heuristic to decide: if it needs more channels, and if so to whom those channels should be opened to. The Agent then carries out the recommendations by the registered heuristic, ultimately going back to the top of it's loop to await further outside signals.

The autopilot package has been signed to be as abstract as possible in order to allow users, developers, and researchers to plug in various heuristics in order to experiment with the possibilities, or attempt to optimize the channel sub-graph for their targeted nodes. The current default heuristic is a mode called
ConstrainedPrefAttachment. This heuristic takes a set of inputs parameters, namely: the smallest allows channel size, the max channel size, the max number of active channels, and a target balance of funds within channels vs regular on-chain utxo's. Given this set of constraints (eg: 4 channels, 40% of available funds), given the known channel graph, the heuristic will employ an channel attachment recommendation driven by the Barabási–Albert model in order to attempt to drive the global graph towards a scale free topology. This is only one example of the possible heuristics which could be hooked into an active autopilot.Agent instance. We look forward to the additional heuristics that developers/researchers will implement! The package has been crafted such that, it facilitates easily generating a simulated graph with several agents each following the same heuristic, or a heterogenous set.

The current default heuristic can be activated, and configured with the following new command line flags:

autopilot:
      --autopilot.active       If the autopilot agent should be active or not.
      --autopilot.maxchannels= The maximum number of channels that should be created (5)
      --autopilot.allocation=  The percentage of total funds that should be committed to automatic channel establishment (0.6)

neutrino: a new Bitcoin Light Client Operating Mode

In prior release of lnd, the only configurable chain-backend was communicating directly with btcd over its websockets RPC interface. In this release, we've added a new configurable chain-backend: neutrino. With this option, it's now possible to run lnd standalone, as neutrino is a Bitcoin light client, which will connect directly to the p2p network in order to receive new information concerning Bitcoin blocks. We're very excited about this new operating mode, as it will allow a wider array of users to use Lightning as it doesn't require one to necessarily have an active full node locally. Additionally, it has allowed us to start experimenting with lnd on mobile operating systems.

In order to run lnd in its light client mode, you'll need to locate a full-node which is capable of serving this new light client mode. A BIP draft exists, and will be finalized in the near future, but for now you'll need to be running roasbeef's fork of btcd. A public instance of such a node can be found at faucet.lightning.community.

The new command line options allowing users to configure neutrino are:

neutrino:
      --neutrino.active        If SPV mode should be active or not.
  -a, --neutrino.addpeer=      Add a peer to connect with at startup
      --neutrino.connect=      Connect only to the specified peers at startup
      --neutrino.maxpeers=     Max number of inbound and outbound peers
      --neutrino.banduration=  How long to ban misbehaving peers.  Valid time units are {s, m, h}.  Minimum 1 second
      --neutrino.banthreshold= Maximum allowed ban score before disconnecting and banning misbehaving peers.

The next release of lnd will also feature an additional chain-backend option.

Milli-Satoshi Accounting and Configurable Fees

In this release of lnd, we've switched over all internal accounting to use mill-satoshis, rather then regular satoshis as dictated within the specification.

Milli-Satoshi are the native unit of the Lightning Network. A milli-satoshi is simply 1/1000th of a satoshi. There are 1000 milli-satoshis in a single satoshi. Within the network, all HTLC payments are denominated in milli-satoshis. As milli-satoshis aren't deliverable on the native blockchain, before settling to broadcasting, the values are rounded down to the nearest satoshi.

Additionally, we've also introduced two new RPC's that let users examine and modify the advertised fee schedules of their channels within the network. Fee accounting within Lightning is currently very simple, each node advertises two values: their base fee in milli-satoshis, and their fee rate per million mSAT transferred. The feereport RPC calls allows users to query for their currently advertised fee scheduled for each of their active channels:

⛰ lncli feereport
{
    "channel_fees": [
        {
            "channel_point": "18ae89fab0f7fca3365c45651aaa9d2a17380bb02d194d217c6a0bb24e76e4ce:0",
            "base_fee_msat": "1000",
            "fee_per_mil": "100",
            "fee_rate": 0.0001
        },
        {
            "channel_point": "1559a538d4270c1fe4a3e5690eeac7620ece3ed2a36cd581528d57d09b7cbd53:0",
            "base_fee_msat": "1000",
            "fee_per_mil": "100",
            "fee_rate": 0.0001
        }
    ]
}

The second newly introduced RPC, updatefees allows users to update the fee schedule of all their active channels:

⛰ lncli updatefees --base_fee_msat=1000 --fee_rate=0.005

Or even to a single particular channel:

⛰ lncli updatefees --fee_rate=0.01 --base_fee_msat=100 --chan_point=b3d6386eed491e15fa17be5b0a68155cadd15b1944f91a4b1b43000da468f6b3:0

Spec Compliant Commitment Design, Key Derivation, and Scripts

In this release of lnd, we've modified our commitment design to utilize the version outlined within the specification. This includes the new key derivation, scripts, and two-layer HTLC design. All sub-systems have been modified to be aware of the new possible on-chain workflows in the occasion of a direct commitment channel broadcast.

With this change, we can now begin to test opening channels and forwarding HTLC's with the various other Lightning Network node implementations.

verifymessage and signmessage

This release adds to new RPC methods: verifymessage and signmessage. The latter allows users to sign an arbitrary message with the advertised identity public key of their node. The signature generated includes the recovery byte of the signature, meaning that users can then use verifymessage to verify the validity of the signature and also return the public key of the node that signed the message.

These RPC calls are interesting, as they now allow a new type of extra-network authentication. For example: a forum website may require a signature of a nonce by a node on the network that has at least 200,000,000 SAT aggregate within active channels. This sign up workflow allows users to join the website without creating an account, or utilizing a password, and also achieves a degree of sybil resistance as the authorization logic of the website can verify the authenticity of the registered channels of the target node.

Additions to the pendingchannels RPC

In this release, we've added additional detail to the pendingchannels RPC call. Now, the command will display all pending channels in the various possible states. For example it now shows which channels have been forced closed, the number of time-locked funds, and the ultimate maturity height of each of the time-locked outputs:

lncli pendingchannels
{
    "total_limbo_balance": "16768528",
    "pending_open_channels": [
        {
            "channel": {
                "remote_node_pub": "02ae91a4c8c25bd480d31811e962cb4f2ad5db728d9a114728ffe3c14a07510f21",
                "channel_point": "c13f806327ee8d19a109967d1a76dbdcd3ad1d968781ab1c87e0a3036df9bba8:1",
                "capacity": "16777216",
                "local_balance": "16768528",
                "remote_balance": "0"
            },
            "commit_fee": "8688",
            "commit_weight": "600",
            "fee_per_kw": "12000"
        },
        {
            "channel": {
                "remote_node_pub": "033a2aff9acf9158d3f7b7df9248e3863a4703cbd80d929e69c0a4765e71c37949",
                "channel_point": "cf0d1a1de534896d3f7a3df44e79637732fcaa0a9af667d1e266b837fc927f7c:0",
                "capacity": "16777216",
                "local_balance": "16768528",
                "remote_balance": "0"
            },
            "commit_fee": "8688",
            "commit_weight": "600",
            "fee_per_kw": "12000"
        }
    ],
    "pending_force_closing_channels": [
        {
            "channel": {
                "remote_node_pub": "03c0b01e019067ddbd79bd268add5def5181122ec862bf7ebb866d06f442d3cc92",
                "channel_point": "11e8a02c098e48250abdd60b763fb513cae0e710b76ec450927ae4b45a9adbf0:0",
                "capacity": "16777216",
                "local_balance": "16768528",
                "remote_balance": "0"
            },
            "closing_txid": "b251c1d162390bafd3d400cf209c9b18921502f1728bc4ba6410f37de2222314",
            "limbo_balance": "16768528",
            "maturity_height": 1179875,
            "blocks_til_maturity": 3
        }
    ]
}

Max Payment and Channel Limits

In this release, we've start to enforce limits on the maximum payment size and also max channel size as specified within BOLT-0002. These limits are an initial precautionary measure for the initial mainnet usage of Lightning. The largest channel size is 2^24 satoshis, and the largest payment size is 2^32 mSAT.

Proper Dynamic Fee Accounting

In prior versions of lnd, we utilized hard coded fees through the daemon. With this new release, we've added a FeeEstimator interface which is now consulted when crafting funding transactions, commitment transactions, and regular payments sent within lnd's internal wallet. The current default implementation of this interface returns a static fee, but in future release we'll be hooking it up into a proper dynamic fee estimator.

Onion Encapsulated HTLC Error Messages

In prior release, we used a custom error format to signal HTLC related errors within the network. With this release, we've switch to using the onion encrypted HTLC error messages as defined within BOLT-0004. The usage of this particular format of errors aims to ensure that participants within a route (ignoring timing), don't know at exactly which position in the route an error occurred.

Full Onion HTLC Payload Verification

Within the HtlcSwitch, we now properly validate included per-hop payloads for each now. With this validation place, we now ensure that the prior hop didn't tamper with the advertised HTLC parameters as we validate the HTLC parameters as dictated by the source of the payment. Additionally, we now enforce that the sending node adheres to our advertised fee and time lock policies.

Spec Message and Port Compliance

All messages within the lnwire package are now full complaint with the BOLT specification messages. As a result, we've now switched over to using port 9735 as the default listening port in order to signal our level of compliance with the specification.

update_fee and Channel Closing Fee Negotiation

With this release, lnd is now able to properly parse and apply the update_fee message to target channels. This means that lnd can now dynamically update the prescribed fee on the latest commitment transaction. Additionally, we've added a new state machine during channel closure that is able to negotiate a final fee rate for the cooperative closure transaction.

Change Log

For the specific change log, see the commits referenced within this tag since the prior release.

Contributors (Alphabetical Order):

• afederigo
• Alex Akselrod
• Alex Bosworth
• Andrey Samokhvalov
• Bryan Vu
• Christian Decker
• Conner Fromknecht
• Johan T. Halseth
• Max Fang
• Olaoluwa Osuntokun
• Mykola Sakhno
• Philip Hayes
• Slava Zhigulin
• Steven Roose
• thesamesam