diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index e63f57962d..022e695e71 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -72,6 +72,11 @@ When adding a new feature, thought must be given to the long term technical debt that feature may require after inclusion. Before proposing a new feature that will require maintenance, please consider if you are willing to maintain it (including bug fixing). +We keep release notes directly [inside the repository](./docs/release-notes) and continuously +update them as new features are added. If your change deserves to be included in the release notes +(e.g. changes to the API, breaking changes, new feature), your pull request must also update the +`eclair-vnext` release notes. + When addressing pull request comments, we recommend using [fixup commits](https://robots.thoughtbot.com/autosquashing-git-commits). The reason for this is two fold: it makes it easier for the reviewer to see what changes have been made between versions (since Github doesn't easily show prior versions) and it makes it easier on diff --git a/docs/release-notes/eclair-v0.6.1.md b/docs/release-notes/eclair-v0.6.1.md new file mode 100644 index 0000000000..c1a8116dfe --- /dev/null +++ b/docs/release-notes/eclair-v0.6.1.md @@ -0,0 +1,155 @@ +# Eclair v0.6.1 + +This release makes major performance improvements, includes a few bug fixes and several new features. +It is fully compatible with 0.6.0 (and all previous versions of eclair). + +## Major changes + +### Performance improvements + +#### Sqlite + +Eclair now uses [write-ahead logging](https://sqlite.org/wal.html) in Sqlite (#1871). WAL is better suited to our DB access patterns, and is both much more performant and safer than the default [rollback journal](https://sqlite.org/lockingv3.html#rollback) that we were using previously. + +> 1. WAL is significantly faster in most scenarios. +> 2. WAL provides more concurrency as readers do not block writers and a writer does not block readers. +> 3. Disk I/O operations tends to be more sequential using WAL. +> 4. WAL uses many fewer fsync() operations. + +This small change improves performance by more than 5x. + +#### Payment Handling + +Invoice generation (#1878) and handling of incoming payments (#1880) are now processed in parallel, resulting in a higher throughput under load. + +### Improved Postgres support + +This is the continuation of an effort to make PosgreSQL production-ready. The database schema has been reworked (#1866) and is now better organized, with appropriate types for timestamps (#1862). There have been several concurrency-related bug fixes. + +We have also added `JSONB` columns for local channels and for network announcements (#1865). All individual data fields can now be accessed from SQL and indexed, which is very convenient for advanced analysis and tuning of a routing node. + +### Upfront shutdown script + +This release adds support for `option_upfront_shutdown_script` (feature bits 4/5). +This feature lets you specify a closing address when you open a channel: your peer will ensure that when you close, your funds can only go to that address. + +It can be useful to protect against future hacks of your node, because the attacker won't be able to close your channels and send the funds to an address that he controls. However, it doesn't prevent the attacker from exfiltrating funds by paying lightning invoices, so you shouldn't rely on this feature alone to make your node hack-proof. + +This option is disabled by default, but can be enabled in your `eclair.conf`. +Note that if you enable it, the closing address will be automatically generated by your `bitcoind` node. + +### Transaction publishing improvements + +This release reworks our internal transaction publishing architecture (see #1844 for details). +The new architecture is more flexible, provides better logging and makes it easy to add dynamic fee bumping in the future for anchor output channels. +It will also make it easier to automatically use CPFP to ensure funding transactions confirm before the [2016 blocks timeout](https://github.com/lightningnetwork/lightning-rfc/pull/839) is reached. + +### API changes + +This release updates a few APIs: + +- `parseinvoice` displays Bolt 11 invoices routing hints (#1833) +- Plugins can inject their own routes into the API instead of spawning a separate HTTP server (#1805 and #1819) + +### Miscellaneous improvements and bug fixes + +- Eclair now uses Bitcoin Core 0.21.1 by default (#taproot) +- Eclair now supports [warning messages](https://github.com/lightningnetwork/lightning-rfc/pull/834) +- Eclair uses additional entropy on top of the operating system RNG to mitigate random number generation failures (#1774) + +## Verifying signatures + +You will need `gpg` and our release signing key 7A73FE77DE2C4027. Note that you can get it: + +- from our website: https://acinq.co/pgp/drouinf.asc +- from github user @sstone, a committer on eclair: https://api.github.com/users/sstone/gpg_keys + +To import our signing key: + +```sh +$ gpg --import drouinf.asc +``` + +To verify the release file checksums and signatures: + +```sh +$ gpg -d SHA256SUMS.asc > SHA256SUMS.stripped +$ sha256sum -c SHA256SUMS.stripped +``` + +## Building + +Eclair builds are deterministic. To reproduce our builds, please use the following environment (*): + +- Ubuntu 20.04 +- AdoptOpenJDK 11.0.6 +- Maven 3.8.1 + +Use the following command to generate the eclair-node package: + +```sh +mvn clean install -DskipTests +``` + +That should generate `eclair-node/target/eclair-node-0.6.1-XXXXXXX-bin.zip` with sha256 checksums that match the one we provide and sign in `SHA256SUMS.asc` + +(*) You may be able to build the exact same artefacts with other operating systems or versions of JDK 11, we have not tried everything. + +## Upgrading + +This release is fully compatible with eclair v0.6.0. You don't need to close your channels, just stop eclair, upgrade and restart. + +## Changelog + +- [a658fa26f](https://github.com/ACINQ/eclair/commit/a658fa26f4ba359923f6e943547ec9a1c4980f54) Set version to 0.6.1-SNAPSHOT (#1813) +- [76894bd2e](https://github.com/ACINQ/eclair/commit/76894bd2e1cba226d618ffd28aa38f5da4228f97) Add additional PRNG (#1774) +- [9a20aade0](https://github.com/ACINQ/eclair/commit/9a20aade0a00f6ef4ad01247e34bbc4cbe4101cd) Allow plugins to inject their own routes into API (#1805) +- [d437ea1ed](https://github.com/ACINQ/eclair/commit/d437ea1ed1e5dc013dec7b609b3ac4cb160a3116) Improve API plugin support (#1819) +- [98cae455f](https://github.com/ACINQ/eclair/commit/98cae455fbe629cdce165647e19c76c8d2221157) Rename pending_relay to pending_commands (#1822) +- [e8c33baf5](https://github.com/ACINQ/eclair/commit/e8c33baf5477c0fabd380e149b79c24dc1d7b9da) Various improvements and fixes (#1817) +- [f829a2e8c](https://github.com/ACINQ/eclair/commit/f829a2e8ca15ebcdbe162420921602fe031a9f68) Add json type hints on channel data (#1824) +- [4dc2910c4](https://github.com/ACINQ/eclair/commit/4dc2910c4e865d100de3ddb0d28616c6b201f6ed) Make result set an iterable (#1823) +- [6f6c458a2](https://github.com/ACINQ/eclair/commit/6f6c458a2d688c9a7ca638c1d957c88dc6c9e008) Add metrics on channels processing time (#1826) +- [43a89f865](https://github.com/ACINQ/eclair/commit/43a89f8659dff7859a9e496344dc3b719bff5be2) Add a random delay before processing blocks (#1825) +- [af618bc44](https://github.com/ACINQ/eclair/commit/af618bc44fe6054957f65d77000a460c14b41f30) Symmetrical HTLC limits (#1828) +- [dbecb28d9](https://github.com/ACINQ/eclair/commit/dbecb28d96d780668fc006aeee2a043a1cc52d10) Include routing hints in parseinvoice API call response (#1833) +- [2b6d564d2](https://github.com/ACINQ/eclair/commit/2b6d564d2161176a851908b871d41961261b1a35) Expose eclair datadir to plugins (#1837) +- [bd6bad1bf](https://github.com/ACINQ/eclair/commit/bd6bad1bfd8185eea734a3ce29ce2e0d9f9da5a3) Fix eventually statements (#1835) +- [a7bb2c2b2](https://github.com/ACINQ/eclair/commit/a7bb2c2b24e6b2495bef98ccf9cc65219f4f40c3) Do not store `CannotAffordFees` errors (#1834) +- [d4b25d565](https://github.com/ACINQ/eclair/commit/d4b25d565d36627b87c608d474c2500f1f1ec50c) Udpate to Bitcoin Core 0.21.1 (#1841) +- [e750474c7](https://github.com/ACINQ/eclair/commit/e750474c727c679321691670150253e2165d35f2) Use bitcoin-lib 0.19 (#1839) +- [bbfbad597](https://github.com/ACINQ/eclair/commit/bbfbad5975925ed717ea423adb0a63b53f07ca65) Validate payment secret when decoding (#1840) +- [afb1b41ea](https://github.com/ACINQ/eclair/commit/afb1b41ea0cbc9aa85ca54b19e4b94bf50690622) Update bolt 3 spec test vectors (#1669) +- [d43d06f6e](https://github.com/ACINQ/eclair/commit/d43d06f6e263dc1212515c4f1b25a221adb18079) Rework TxPublisher (#1844) +- [45204e238](https://github.com/ACINQ/eclair/commit/45204e238052ca1691f9f6486eaf1863ae8836b2) Schedule backup at regular interval (#1845) +- [85ed4338a](https://github.com/ACINQ/eclair/commit/85ed4338a3e34d836897a434afa5353ba636e128) Reject 0-value trampoline payments (#1851) +- [f857368ea](https://github.com/ACINQ/eclair/commit/f857368ea052158e894568f620fff664a5f2f549) Make trampoline payments use per-channel fee and cltv (#1853) +- [f52c3dd3f](https://github.com/ACINQ/eclair/commit/f52c3dd3fca7b7f32a2740b9c866a237f4703c4f) Decode warning messages (#1854) +- [516929b1a](https://github.com/ACINQ/eclair/commit/516929b1a360bc62509b42fb5be91e1a6d378dd0) Fix default file backup config (#1857) +- [4ca5c62ab](https://github.com/ACINQ/eclair/commit/4ca5c62abbf38109a390b849c3b5a24a75e53c52) Remove println in tests (#1861) +- [291c128ca](https://github.com/ACINQ/eclair/commit/291c128cab5e0cd92aad5d48f721f92c3134c898) Reduce some log levels (#1864) +- [d9a03a52b](https://github.com/ACINQ/eclair/commit/d9a03a52b83ad342f06c4eb2f11e7db7af2bcd0a) Use warning messages for connection issues (#1863) +- [af8394a28](https://github.com/ACINQ/eclair/commit/af8394a28031117f0ebfbbdcfca1cf3d1633cdd0) Add support for dual db backend (#1746) +- [3a573e267](https://github.com/ACINQ/eclair/commit/3a573e267a71a4f1fe91b35065a8d0b96f7f6d69) Improve message for `CannotRetrieveFeerates` error (#1859) +- [bd57d41ef](https://github.com/ACINQ/eclair/commit/bd57d41ef3acb755cbb7039603071fec391e5d6a) Add a globalbalance api call (#1737) +- [08faf3b7f](https://github.com/ACINQ/eclair/commit/08faf3b7fd05876be358f5ddc2ef6b27d126fd6b) Add json columns in Postgres (#1865) +- [f8feb1959](https://github.com/ACINQ/eclair/commit/f8feb19593a269e42f637c8f9bb8cb0843c9a2cd) Use schemas in Postgres (#1866) +- [cea3fc026](https://github.com/ACINQ/eclair/commit/cea3fc026da3cc53f0f210fe57501c9ffe8e0c41) Use proper data type for timestamps in Postgres 2 (#1862) +- [95fffe348](https://github.com/ACINQ/eclair/commit/95fffe348cbd42da70731fbf76f5cae48a97b678) Reduce pg transaction isolation (#1860) +- [547d7e700](https://github.com/ACINQ/eclair/commit/547d7e700f05960c0f52a79819a94686af186d96) Create chain directory (#1872) +- [e9df4eece](https://github.com/ACINQ/eclair/commit/e9df4eece09f8757d4e03405b70d1d551bdadff1) Channels data format migration (#1849) +- [733c6e768](https://github.com/ACINQ/eclair/commit/733c6e768890ef75e0766b5986431a3f2038390c) Refactor global balance tests (#1874) +- [ca51a2d16](https://github.com/ACINQ/eclair/commit/ca51a2d16883d277da41258e527a4ac7a6894bb6) Enable WAL mode on Sqlite (#1871) +- [3ae9a4ae3](https://github.com/ACINQ/eclair/commit/3ae9a4ae37297e0527bb83f22135860bcc3e49b3) Additional reestablish test (#1875) +- [d02760d96](https://github.com/ACINQ/eclair/commit/d02760d96fb5f353d9f82ae8582db9e7b67c9654) Fail unsigned outgoing htlcs on force-close (#1832) +- [3bb7ee8a3](https://github.com/ACINQ/eclair/commit/3bb7ee8a3eeff9811ff0d88365f0d6bc212bc822) Parallel payment request generation (#1878) +- [51824028b](https://github.com/ACINQ/eclair/commit/51824028b303169be733c216c2f18c531d11887b) Fix flaky channel integration tests (#1879) +- [b4183edfa](https://github.com/ACINQ/eclair/commit/b4183edfa130e81a3c0be624f9331fc9ae4caab1) Fetch incoming payments in parallel (#1880) +- [8c49f779a](https://github.com/ACINQ/eclair/commit/8c49f779ada17ba2e3a061b14f5b54c148101b15) Fix payment handler tests (#1882) +- [01b40730f](https://github.com/ACINQ/eclair/commit/01b40730fe6d58bfba9c37fcc0d6a32c772851e3) Implement option-upfront-shutdown-script (#1846) +- [79729c78c](https://github.com/ACINQ/eclair/commit/79729c78c14fdd2ccea88619d4ed19272964c9b2) Update README (#1881) +- [c22596bb9](https://github.com/ACINQ/eclair/commit/c22596bb9243331c8195998738a89d1b0739bf60) Update dependencies and enable fatal warnings (#1885) +- [adf36de0e](https://github.com/ACINQ/eclair/commit/adf36de0e154950ef71caa5373a6e85410ab1789) Fix yet another flaky test (#1886) +- [3f1c2506a](https://github.com/ACINQ/eclair/commit/3f1c2506a6efe6563a890014b3fa3e4f1f34859b) Fix watcher flaky test (#1883) +- [c8c5e76d2](https://github.com/ACINQ/eclair/commit/c8c5e76d2428a37baa4654807275cd1db22af4ba) Update akka-http-json4s dependency (#1889) +- [d3ae32326](https://github.com/ACINQ/eclair/commit/d3ae32326e78729390706998b899ea63a91ecb79) Set version to 0.6.1 (#1887) diff --git a/docs/release-notes/eclair-vnext.md b/docs/release-notes/eclair-vnext.md new file mode 100644 index 0000000000..9b0dc2cc81 --- /dev/null +++ b/docs/release-notes/eclair-vnext.md @@ -0,0 +1,184 @@ +# Eclair vnext + + + +## Major changes + +### ZMQ changes + +Eclair previously used ZMQ to receive full blocks from Bitcoin Core. +In this release, we instead switch to receive only block hashes over ZMQ. +This will save bandwidth and improve support for deployments with a remote Bitcoin Core node. + +:warning: When updating eclair, you need to update your `bitcoin.conf` to have your Bitcoin Core node send block hashes via ZMQ. + +The previous configuration was: + +```conf +zmqpubrawblock=tcp://127.0.0.1:29000 +``` + +You must remove that line from your `bitcoin.conf` and replace it with: + +```conf +zmqpubhashblock=tcp://127.0.0.1:29000 +``` + +### Per node relay fees + +Relay fees are now set per node instead of per channel: + +- If you set the relay fees for a node with the `updaterelayfee` API, all new channels with this node will use these fees. +- Otherwise the default relay fees set in `eclair.conf` will be used: this means that changing `eclair.conf` will update the fees for all channels where the fee was not manually set. + +Note that you can use the `updaterelayfee` API *before* opening a channel to ensure that the channel doesn't use the default relay fees from `eclair.conf`. + +:warning: When updating eclair, the relay fees for your existing channels will be reset to the value from your `eclair.conf`. You should use the `updaterelayfee` API to reconfigure relay fees if you don't want to use the default fees for every node you're connected to. + +### Beta support for anchor outputs + +Anchor outputs is still disabled by default, but users willing to try it can activate it by adding the following line to `eclair.conf`: + +```conf +eclair.features.option_anchors_zero_fee_htlc_tx = optional +``` + +Once activated, eclair will keep the commitment feerate below 10 sat/byte regardless of the current on-chain feerate and will not close channels when there is a feerate mismatch between you and your peer. + +You can modify that threshold by setting `eclair.on-chain-fees.feerate-tolerance.anchor-output-max-commit-feerate` in your `eclair.conf`. +Head over to [reference.conf](https://github.com/ACINQ/eclair/blob/master/eclair-core/src/main/resources/reference.conf) for more details. + +In case the channel is unilaterally closed, eclair will use CPFP and RBF to ensure that transactions confirm in a timely manner. +You **MUST** ensure you have some utxos available in your Bitcoin Core wallet for fee bumping, otherwise there is a risk that an attacker steals some of your funds. + +Do note that anchor outputs may still be unsafe in high-fee environments until the Bitcoin network provides support for [package relay](https://bitcoinops.org/en/topics/package-relay/). + +### Path-finding improvements + +This release contains many improvements to path-finding and paves the way for future experimentation. + +A noteworthy addition is a new heuristic that can be used to penalize long paths by setting a virtual cost per additional hop in the route (#1815). This can be freely configured by node operators by setting fields in the `eclair.router.path-finding.default.hop-cost` section. + +We also added support for A/B testing to experiment with various configurations of the available heuristics. +A/B testing can be activated directly from `eclair.conf`, by configuring some `experiments`, for example: + +```conf +eclair.router.path-finding.experiments { + control = ${eclair.router.path-finding.default} { + percentage = 75 // 75% of the traffic will use the default configuration + } + + use-shorter-paths = ${eclair.router.path-finding.default} { + percentage = 25 // 25% of the traffic will use this custom configuration + ratios { + base = 1 + cltv = 0 + channel-age = 0 + channel-capacity = 0 + } + hop-cost { + // High hop cost penalizes strongly longer paths + fee-base-msat = 10000 + fee-proportional-millionths = 10000 + } + } +} +``` + +Have a look at [reference.conf](https://github.com/ACINQ/eclair/blob/master/eclair-core/src/main/resources/reference.conf) for more examples. + +You can also force a specific payment to use an experimental path-finding configuration by specifying the experiment name in the various path-finding APIs: + +```sh +eclair-cli payinvoice --invoice= --pathFindingExperimentName=use-shorter-paths +``` + +The results are stored in the `audit` database, inside the `path_finding_metrics` table. +You can then analyze the results after sending a large enough number of payments to decide what configuration yields the best results for your usage of lightning. + +### Tor support for blockchain watchdogs + +Eclair introduced blockchain watchdogs in v0.5.0, where secondary blockchain sources are regularly queried to detect whether your node is being eclipsed. + +Most of these watchdogs were previously queried over HTTPS, which exposes your IP address. +This is fixed in this release: when using Tor, the watchdogs will now also be queried through Tor, keeping your IP address private. + +You can also now choose to disable some watchdogs by removing them from the `eclair.blockchain-watchdog.sources` list in `eclair.conf`. +Head over to [reference.conf](https://github.com/ACINQ/eclair/blob/master/eclair-core/src/main/resources/reference.conf) for more details. + +### Sample GUI removed + +We previously included code for a sample GUI: `eclair-node-gui`. +This GUI was only meant to be used for demo purposes, not for mainnet node administration. + +However some users were using it on mainnet, which lead to several issues (e.g. channel closure and potentially loss of funds). +We completely removed it from this release to prevent it from happening again. + +### API changes + +This release contains many API updates: + +- `open` lets you specify the channel type through the `--channelType` parameter, which can be one of `standard`, `static_remotekey`, `anchor_outputs` or `anchor_outputs_zero_fee_htlc_tx` (#1867) +- `open` doesn't support the `--feeBaseMsat` and `--feeProportionalMillionths` parameters anymore: you should instead set these with the `updaterelayfee` API, which can now be called before opening a channel (#1890) +- `updaterelayfee` must now be called with nodeIds instead of channelIds and will update the fees for all channels with the given node(s) at once (#1890) +- `close` lets you specify a fee range when using quick close through the `--preferredFeerateSatByte`, `--minFeerateSatByte` and `--maxFeerateSatByte` (#1768) +- `createinvoice` now lets you provide a `--descriptionHash` instead of a `--description` (#1919) +- `sendtonode` doesn't support providing a `paymentHash` anymore since it uses `keysend` to send the payment (#1840) +- `payinvoice`, `sendtonode`, `findroute`, `findroutetonode` and `findroutebetweennodes` let you specify `--pathFindingExperimentName` when using path-finding A/B testing (#1930) +- the `--maxFeePct` parameter used in `payinvoice` and `sendtonode` must now be an integer between 0 and 100: it was previously a value between 0 and 1, which was misleading for a percentage (#1930) +- `findroute`, `findroutetonode` and `findroutebetweennodes` let you choose the format of the route returned with the `--routeFormat` parameter (supported values are `nodeId` and `shortChannelId`) (#1943) + +Have a look at our [API documentation](https://acinq.github.io/eclair) for more details. + +### Miscellaneous improvements and bug fixes + +- Eclair nodes may now use different relay fees for unannounced channels (#1893) +- Relay fees are now set per node and automatically apply to all channels with that node (#1890) +- Eclair now supports [explicit channel type negotiation](https://github.com/lightningnetwork/lightning-rfc/pull/880) +- Eclair now supports [quick close](https://github.com/lightningnetwork/lightning-rfc/pull/847), which provides more control over what feerate will be used when closing channels + +## Verifying signatures + +You will need `gpg` and our release signing key 7A73FE77DE2C4027. Note that you can get it: + +- from our website: https://acinq.co/pgp/drouinf.asc +- from github user @sstone, a committer on eclair: https://api.github.com/users/sstone/gpg_keys + +To import our signing key: + +```sh +$ gpg --import drouinf.asc +``` + +To verify the release file checksums and signatures: + +```sh +$ gpg -d SHA256SUMS.asc > SHA256SUMS.stripped +$ sha256sum -c SHA256SUMS.stripped +``` + +## Building + +Eclair builds are deterministic. To reproduce our builds, please use the following environment (*): + +- Ubuntu 20.04 +- AdoptOpenJDK 11.0.6 +- Maven 3.8.1 + +Use the following command to generate the eclair-node package: + +```sh +mvn clean install -DskipTests +``` + +That should generate `eclair-node/target/eclair-node--XXXXXXX-bin.zip` with sha256 checksums that match the one we provide and sign in `SHA256SUMS.asc` + +(*) You may be able to build the exact same artefacts with other operating systems or versions of JDK 11, we have not tried everything. + +## Upgrading + +This release is fully compatible with previous eclair versions. You don't need to close your channels, just stop eclair, upgrade and restart. + +## Changelog + +