From 34adddba8d584df00070e0b43c503d19042d625d Mon Sep 17 00:00:00 2001 From: Nadav Ivgi Date: Sat, 21 Nov 2020 04:03:19 +0200 Subject: [PATCH] docs: Document the C FFI interface (#64) --- CHANGELOG.md | 3 + README.md | 2 + contrib/nodejs-bwt-daemon/README.md | 57 +++--------- doc/libbwt.md | 136 ++++++++++++++++++++++++++++ 4 files changed, 155 insertions(+), 43 deletions(-) create mode 100644 doc/libbwt.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 1e66786..ae2d77e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,9 @@ are now identified by the descriptor checksum, addresses have descriptors associated with them, and a new `bip32_origins` field is available based on the descriptor origin information. +- Alpha release of [`libbwt`](https://github.com/shesek/bwt/blob/master/doc/libbwt.md) (#64), a C FFI interface for managing the bwt servers, + and of [`nodejs-bwt-daemon`](https://github.com/shesek/bwt/tree/master/contrib/nodejs-bwt-daemon) (#65), a nodejs package that wraps it. + - Support binding on ephemeral port (e.g. `--http-addr 127.0.0.1:0`) (#63) - Reduce the number of dependencies (#61) diff --git a/README.md b/README.md index a89ba7a..86680a3 100644 --- a/README.md +++ b/README.md @@ -1240,6 +1240,8 @@ Tip: services like [webhook.site](https://webhook.site/) or [requestbin](http:// ### Developer Resources +Documentation for the C FFI interface is [available here](https://github.com/shesek/bwt/blob/master/doc/libbwt.md). + Documentation for the public Rust API is [available on docs.rs](https://docs.rs/bwt). A yuml diagram showing how the big pieces interact together is [available here](https://yuml.me/edit/39229813). diff --git a/contrib/nodejs-bwt-daemon/README.md b/contrib/nodejs-bwt-daemon/README.md index e982603..f006f20 100644 --- a/contrib/nodejs-bwt-daemon/README.md +++ b/contrib/nodejs-bwt-daemon/README.md @@ -1,7 +1,10 @@ # bwt-daemon -A nodejs library for programmatically controlling the Bitcoin Wallet Tracker daemons -using the `libbwt` ffi interface. +A nodejs library for programmatically managing the Bitcoin Wallet Tracker Electrum RPC and HTTP API servers +using the [`libbwt` FFI interface](https://github.com/shesek/bwt/blob/master/doc/libbwt.md). + +> ⚠️ WARNING: This is an alpha preview, released to gather developers' feedback. It is not ready for general use. +> *Do not use with real bitcoins.* ### Install @@ -10,7 +13,7 @@ $ npm install bwt-daemon ``` This will download the `libbwt` library for your platform. -The currently supported platforms are linux-x64, macos-x64, windows-x64, linux-arm32v7 and linux-arm64v8. +The currently supported platforms are Linux, Mac, Windows and ARMv7/8. The hash of the downloaded library is verified against the [`SHA256SUMS`](https://github.com/shesek/bwt/blob/master/contrib/nodejs-bwt-daemon/SHA256SUMS) @@ -26,8 +29,8 @@ This reduces the download size by ~1.6MB. ### Use -Below is the minimally viable configuration. If bitcoind is running at the default -location, on the default ports and with cookie auth enabled, this should Just Work™ \o/ +Below is a minimally viable configuration. If bitcoind is running locally on the default port, at the default datadir location +and with cookie auth enabled (the default), this should Just Work™ \o/ ```js import BwtDaemon from 'bwt-daemon' @@ -74,7 +77,7 @@ const bwtd = await BwtDaemon({ // Get the assigned address/port for the Electrum/HTTP servers console.log('bwt electrum server ready on', bwtd.electrum_addr) -console.log('bwt http server ready on', bwtd.http_addr) +console.log('bwt http server ready on', bwtd.http_url) // Shutdown bwtd.shutdown() @@ -83,43 +86,11 @@ bwtd.shutdown() See [`example.js`](https://github.com/shesek/bwt/blob/master/contrib/nodejs-bwt-daemon/example.js) for an even more complete example, including connecting to the HTTP API. -### Options - -#### Network and Bitcoin Core RPC -- `network` -- `bitcoind_dir` -- `bitcoind_wallet` -- `bitcoind_url` -- `bitcoind_auth` -- `bitcoind_cookie` - -#### Address tracking -- `descriptors` -- `xpubs` -- `bare_xpubs` - -#### General settings -- `verbose` -- `gap_limit` -- `initial_import_size` -- `poll_interval` -- `tx_broadcast_cmd` - -#### Electrum -- `electrum` -- `electrum_addr` -- `electrum_skip_merkle` - -#### HTTP -- `http` -- `http_addr` -- `http_cors` - -#### Web Hooks -- `webhooks_urls` - -#### UNIX only -- `unix_listener_path` +The full list of options is available in the [FFI documentation](https://github.com/shesek/bwt/blob/master/doc/libbwt.md#config-options). +The nodejs wrapper also provides the following additional convenience options: + +- `electrum` - setting to `true` is an alias for `electrum_addr=127.0.0.1:0` +- `http` - setting to `true` is an alias for `http_addr=127.0.0.1:0` ### License MIT diff --git a/doc/libbwt.md b/doc/libbwt.md new file mode 100644 index 0000000..eeaa5af --- /dev/null +++ b/doc/libbwt.md @@ -0,0 +1,136 @@ +# `libbwt` + +C FFI library for programmatically managing the Bitcoin Wallet Tracker Electrum RPC and HTTP API servers. + +`libbwt` has two primary use-cases: + +1. Using the bwt Electrum server as a compatibility layer for Electrum-backed wallets + that wish to support using a self-hosted Bitcoin Core full node as their backend, + by running the server *in* the wallet. + +2. Shipping software that leverages the [bwt HTTP API](https://github.com/shesek/bwt#http-api) + as an all-in-one package, without requiring the user to separately install bwt. + +Pre-built signed & deterministic `libbwt` library files (`so`/`dylib`/`dll`) are available for download from the +[releases page](https://github.com/shesek/bwt/releases) for Linux, Mac, Windows and ARMv7/8, including an `electrum_only` variant. + +> ⚠️ WARNING: This is an alpha preview, released to gather developers' feedback. It is not ready for general use. +> *Do not use with real bitcoins.* + +## Binding libraries + +A nodejs package wrapping the native library with a more convenient higher-level API is [available here](https://github.com/shesek/bwt/tree/master/contrib/nodejs-bwt-daemon). + +(More binding libraries coming soon, [let me know](https://github.com/shesek/bwt/issues/69) which you'd like to see first!) + + +## C interface + +The interface exposes two functions, for starting and stopping the bwt servers. +Everything else happens through the Electrum/HTTP APIs. + +```c +typedef void (*bwt_callback)(const char* msg_type, float progress, + uint32_t detail_n, const char* detail_s); + +int32_t bwt_start(const char* json_config, + bwt_callback callback, + void** shutdown_out); + +int32_t bwt_shutdown(void* shutdown_ptr); +``` + +Both functions return `0` on success or `-1` on failure. + +### `bwt_start(json_config, callback, shutdown_out)` + +Start the configured server(s). + +This will block the current thread until the initial indexing is completed and the API servers +are ready, which may take awhile on the first run (depending on the `rescan_since` configuration). +If you'd like this to happen in the background, call this function in a new thread. + +`json_config` should be provided as a JSON-encoded string. The list of options is available [below](#config-options). +Example minimal configuration: + +``` +{ + "bitcoind_dir": "/home/satoshi/.bitcoin", + "descriptors": [ "wpkh(xpub66../0/*)" ], + "electrum_addr": "127.0.0.1:0", + "http_addr": "127.0.0.1:0" +} +``` + +> You can configure `electrum_addr`/`http_addr` to `127.0.0.1:0` to bind on any available port. +> The assigned port will be reported back via the `ready:X` notifications (see below). + +The `callback(msg_type, progress, detail_n, detail_s)` function will be called with progress updates and information +about the running services, with the `progress` argument indicating the current progress as a float from 0 to 1. +The meaning of the `detail_{n,s}` field varies for the different `msg_type`s, which are: + +- `booting` - Sent after the configuration is validated, right before booting up. `detail_{n,s}` are both empty. +- `progress:sync` - Progress updates for bitcoind's initial block download. `detail_n` contains the unix timestamp + that the chain is currently synced up to. +- `progress:scan` - Progress updates for historical transactions rescanning. `detail_n` contains the estimated + remaining time in seconds. +- `ready:electrum` - The Electrum server is ready. `detail_s` contains the address the server is bound on, + as an `:` string (useful for ephemeral binding on port 0). +- `ready:http` - The HTTP server is ready. `detail_s` contains the address the server is bound on. +- `ready` - Everything is ready. +- `error` - An error occurred during the initial indexing. `detail_s` contains the error message. + +> The `detail_s` argument will be deallocated after the callback is called. If you need to keep it around, make a copy of it. +> +> Note that `progress:X` notifications will be sent from a different thread. + +After the initial indexing is completed, a new thread will be spawned to sync new blocks/transactions in the background. + +A shutdown handler for stopping bwt will be written to `shutdown_out`. + +### `bwt_shutdown(shutdown_ptr)` + +Shutdown bwt's API server(s) and the background syncing thread. + +Should be called with the shutdown handler written to `shutdown_out`. + +## Config Options + +All options are optional, except for `descriptors`/`xpubs`/`bare_xpubs` (of which there must be at least one). + +If bitcoind is running locally on the default port, at the default datadir location and with cookie auth enabled (the default), connecting to it should Just Work™, no configuration needed. + +#### Network and Bitcoin Core RPC +- `network` - one of `bitcoin`, `testnet` or `regtest` (defaults to `bitcoin`) +- `bitcoind_url` - bitcoind url (defaults to `http://localhost:/`) +- `bitcoind_auth` - authentication in `:` format (defaults to reading from the cookie file) +- `bitcoind_dir` - bitcoind data directory (defaults to `/.bitcoin` on Linux, `~/Library/Application Support/Bitcoin` on Mac, or `%APPDATA%\Bitcoin` on Windows) +- `bitcoind_cookie` - path to cookie file (defaults to `.cookie` in the datadir) +- `bitcoind_wallet` - bitcoind wallet to use (for use with multi-wallet) + +#### Address tracking +- `descriptors` - an array of descriptors to track +- `xpubs` - an array of xpubs to track (SLIP32 ypubs/zpubs are supported too) +- `bare_xpubs` - an array of bare xpubs to track (no separate internal/external chain) +- `rescan_since` - the unix timestamp to begin rescanning from, or 'now' to track new transactions only (scans from genesis by default) +- `gap_limit` - the [gap limit](https://github.com/shesek/bwt#gap-limit) for address import (defaults to 20) +- `initial_import_size` - the chunk size to use during the initial import (defaults to 350) + +#### General settings +- `poll_interval` - interval for polling new blocks/transactions from bitcoind in seconds (defaults to 5) +- `tx_broadcast_cmd` - [custom command](https://github.com/shesek/bwt#scriptable-transaction-broadcast) for broadcasting transactions +- `verbose` - verbosity level for stderr log messages (0-4, defaults to 0) + +#### Electrum +- `electrum_addr` - bind address for electrum server (off by default) +- `electrum_skip_merkle` - skip generating merkle proofs (off by default) + +#### HTTP +- `http_addr` - bind address for http server (off by default) +- `http_cors` - allowed cross-origins for http server (none by default) + +#### Web Hooks +- `webhooks_urls` - array of urls to notify with index updates + +#### UNIX only +- `unix_listener_path` - path to bind the [sync notification](https://github.com/shesek/bwt#real-time-indexing) unix socket (off by default)