Merge pull request #8 from ActorForth/refactor

Refactor bch-toolkit

.gitignore

@@ -1,5 +1,7 @@
 .DS_Store
 .BTK-container-lock
+.env
+.lock
 
 Electron-Cash-SLP
 Electron-Cash-SLP/*
@@ -8,7 +10,6 @@ bch-api/*
 misc/*
 cert/*
 data/*
-
+configs/cert
 *.txt
 *.swp
-

README.md

@@ -1,6 +1,6 @@
 # About
 
-This repo intends to be a toolkit for running a local version of a Bitcoin Cash node for development and testing purposes. Currently it provides a self-hosted local node, a patched version of Electron Cash SLP, and a localized version of the rest.bitcoin.com REST APIs.
+This repo intends to be a toolkit for running a local version of a Bitcoin Unlimited for development and testing purposes. Currently it provides a self-hosted local node, indexer (Electrs), a drop-in Ninsight replacement (Opensight), a regtest version of the rest.bitcoin.com REST APIs, an instance of SLPDB for token querying, and a regtest-patched version of Electron Cash SLP.
 
 # Prerequisites
 
@@ -21,39 +21,37 @@ git clone https://github.com/ActorForth/bch-toolkit.git
 cd ./bch-toolkit
 ```
 
-Next, run the 'setup' script. This will check that the necessary software is installed, create certificates for use with SSL/TLS, build and prepare the docker containers, download the Electron-Cash-SLP repository, and patch necessary files to enable it to run on the regtest network.
+Next, run the 'setup' script. This will check that the necessary software is installed, and then it will download and prepare the docker containers. For a full list of options, run ./setup with no arguments to see its usage. 
 
-__NOTE:__ If you wish to have any custom changes applied to the Bitcoin Unlimited, Nginx Reverse Proxy, or REST API services, be sure to apply those changes within the _bitcoin.conf_, _nginx.conf_, or _restapi-config.sh_ files, respectively, before executing the setup script.
+__NOTE:__ If you wish to have any custom changes applied to the Bitcoin Unlimited, or REST API services, be sure to apply those changes within the _bitcoin.conf_, _fulcrum-config.conf_, or _restapi-config.sh_ files, respectively, before executing the setup script.
 
 ```bash
-./setup all
+./setup default
 ```
 
-Next, you should execute the 'run' script to start the node service and the Electron Cash SLP wallet.
+Next, you should execute the _services_ script to start the node, indexer, rest API, and/or SLPDB (depending which ones chose in the _setup_ script).
 
 ```bash
-./run wallet
+./services run
 ```
-_**Alternatively**, if you only want the Bitcoin Unlimited node and REST APIs without using the wallet, you may execute it this way:_
 
-```bash
-./run server
-```
+Once you decide to call it a day, you can shut down your local environment by executing:
 
-
-_**JS stack bootup example**_
 ```bash
-./run server slpdb js
+./services stop
 ```
 
+**If you selected the Electron Cash SLP wallet:**
 
-Once the Electron Cash SLP wallet gui appears, it will ask you for "automatic" or "manually select a server". Either option is fine. After that, you'll go through the prompts to create a new wallet. Finally once the wallet is created, you'll be greated with the main interface.
+Once the Electron Cash SLP wallet GUI appears, it will ask you for "automatic" or "manually select a server". Either option is fine. After that, you'll go through the prompts to create a new wallet. Finally once the wallet is created, you'll be greated with the main interface.
 __Please note:__ you may notice the status bar in the lower left corner indicates "Not connected." In order to remedy this, please open another terminal, navigate to the **bch-toolkit** directory, and execute:
 
 ```bash
-./bitcoin-cli 
+./bitcoin-cli generatetoaddress bchreg:xxxx
 ```
 
+where "xxxx" is a bitcoin cash address whose private keys you control. (It can be an address from the "Addresses" tab in Electron Cash SLP's main window.)
+
 This script gives you easy access to the bitcoin-cli that communicates with the running node. When executed with no parameters, this script will simply generate 1 block on your regtest blockchain. After generating a block or two, you should open the "Networks" dialog in the Electron Cash SLP main window, navigate to the "Servers" tab, and un-ban the 'localhost' server. For some reason, the first time blocks are generated, the localhost server gets banned. This does only happen the first time, after which you shouldn't need to repeat this step in the future (at least until you run the './clean' script, which completely removes everything and starts you back from scratch.)
 
 # Cleaning Up
@@ -72,7 +70,7 @@ There are a few issues with this setup that could use improvement in the future.
 
 * Currently, this setup relies on the built-in Electrscash indexing server included in the Bitcoin Unlimited node. This isn't an issue per se, but to test with other nodes (such as Bitcoin ABC, Bitcoin Cash Node, or bchd) it will be necessary to seperate the indexing server into a seperate container.
 
-* Currently blocks must be generated manually after transactions are made. The possibility of adding an automation to generate a block every few seconds could be a consideration to look into.
+* Currently blocks must be generated manually after transactions are made in order to mine them into blocks. The possibility of adding an automation to generate a block every few minutes could be a consideration to look into.
 
 
 

bitcoin-cli

@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 set -e
 
-if [ -f "./.BTK-container-lock" ]; then
+if [ -f "./.lock" ]; then
 	if [[ ! -z $1 ]]; then
 		# This will run the user's command, and also generate 5 blocks
 		docker-compose exec -T bitcoind-regtest bitcoin-cli $@
@@ -15,6 +15,6 @@ if [ -f "./.BTK-container-lock" ]; then
 		echo "Blocks generated."
 	fi
 else
-	echo "Use this fix script AFTER ./setup and ./run"
+	echo "Use this fix script AFTER ./setup and ./services start"
 	echo "Please note, the wallet should be actively running when you use this script."
 fi

clean

@@ -33,7 +33,7 @@ else
 	exit 0 
 fi
 
-if [[ -d "./cert" && $BTK_CERTS == 1 ]]; then
+if [[ -d "./configs/cert" && $BTK_CERTS == 1 ]]; then
 	echo "Removing certificates directory."
 	rm -rf cert
 	DID_SOMETHING=1
@@ -77,10 +77,24 @@ if [[ BTK_WALLET == 1 ]]; then
 	fi
 fi
 
-if [[ $BTK_DOCKER == 1 && -f "./.BTK-container-lock" ]]; then
+if [[ $BTK_DOCKER == 1 && -f "./.lock" ]]; then
+
 	echo "Removing previous docker containers"
 	docker-compose down
-	rm "./.BTK-container-lock"
+
+	# Remove node data files
+    echo "Deleting node data... (./data/bitcoind)"
+	rm -rf "./data/bitcoind"
+
+	# Remove SLPDB data files
+    echo "Deleting SLPDB data... (./data/mongodb)"
+	rm -rf "./data/mongodb"
+
+	# Remove indexer data files
+ #    echo "Deleting indexer data... (./data/fulcrum)"
+	# rm -rf "./data/fulcrum"
+
+	rm ".lock"
 	DID_SOMETHING=1
 fi
 

configs/bch-api-config.sh

@@ -0,0 +1,29 @@
+#!/bin/bash
+
+# Full node
+export RPC_BASEURL=http://localhost:18443/
+export RPC_USERNAME=regtest
+export RPC_PASSWORD=regtest
+export NETWORK=regtest
+
+# SLPDB
+export SLPDB_URL=http://localhost:12300/
+
+# Blockbook
+export BLOCKBOOK_URL=https://<Blockbook IP>:9131/
+# Allow node.js to make network calls to https using self-signed certificate.
+export NODE_TLS_REJECT_UNAUTHORIZED=0
+
+# Mainnet Fulcrum / ElectrumX
+export FULCRUM_URL=bitcoind-regtest
+export FULCRUM_PORT=50002
+
+export SECURITY=false
+### require if SECURITY=true ###
+export TOKENSECRET=somelongpassword
+# Redis require for rate limite middleware
+export REDIS_HOST=6379
+export REDIS_PORT=127.0.0.1
+##################################
+
+npm start

bitcoin.conf → configs/bitcoin-unlimited.conf

@@ -1,32 +1,30 @@
-# Bitcoind config
+# BitcoinUnlimited config
 
 regtest=1
 
 rpcuser=regtest
 rpcpassword=regtest
+
 rpcallowip=0.0.0.0/0
-rpcbind=0.0.0.0:18332
-rpcport=18332
+rpcbind=0.0.0.0:18443
+rpcport=18443
 
+txindex=1
 server=1
 rest=1
 
-### Pats settings
+maxconnections=0
+#disablewallet=1 #leave this enabled until Bitcash has Coinbase support
 
 rpcworkqueue=10000
 rpcthreads=8
 zmqpubhashtx=tcp://*:28332
 zmqpubrawtx=tcp://*:28332
 zmqpubhashblock=tcp://*:28332
 zmqpubrawblock=tcp://*:28332
-###
-
-reindex=1
-txindex=1
 
 electrum=1
 electrum.port=50001
 electrum.host=0.0.0.0
 
 debug=electrum
-

configs/bitcoin.conf

@@ -0,0 +1,27 @@
+# Bitcoin Cash Node config
+
+regtest=1
+
+txindex=1
+reindex=1
+server=1
+rest=1
+
+maxconnections=0
+# disablewallet=1
+
+rpcworkqueue=10000
+rpcthreads=8
+zmqpubhashtx=tcp://*:28332
+zmqpubrawtx=tcp://*:28332
+zmqpubhashblock=tcp://*:28332
+zmqpubrawblock=tcp://*:28332
+
+[regtest]
+rpcuser=regtest
+rpcpassword=regtest
+
+rpcallowip=0.0.0.0/0
+rpcbind=0.0.0.0:18443
+rpcport=18443
+

configs/fulcrum-config.conf

@@ -0,0 +1,113 @@
+# Quickstart Example Configuration File for Fulcrum
+#
+# This file is a quick sample config that has brief comments for the basic
+# options offered by Fulcrum. For a more complete description of all the
+# possible configuration variables, please see "fulcrum-example-config.conf"
+# from the doc/ directory of the Fulcrum source code.
+#-------------------------------------------------------------------------------
+
+# *REQUIRED* This is your database directory. The mountpoint needs about 35GB
+# free space if synching to mainnet and 8GB free space if synching to testnet.
+# NOTE: Use native path separators: '/' on Unix, '\' on Windows.
+datadir = /data # Windows: datadir = D:\FulcrumData\mainnet
+
+# *REQUIRED* This is the bitcoind RPC socket you configured usig rpcbind= and
+# rpcport= in your bitcoind .conf file.
+bitcoind = bitcoind-regtest:18443
+
+# *OPTIONAL* Use this option to connect to bitcoind via HTTPS rather than the
+# usual HTTP. This option typically is only used with `bchd`.
+#bitcoind-tls = true
+
+# *REQUIRED* This is the bitcoind RPC username you specified in your bitciond
+# .conf file. This corresponds to the rpcuser= from that file.
+rpcuser = regtest
+
+# *REQUIRED* This is the bitcoind RPC password you specified in your bitciond
+# .conf file. This corresponds to the rpcpassword= from that file.
+rpcpassword = regtest
+
+# *RECOMMENDED* - TCP bind - 'tcp' - DEFAULT: 0.0.0.0:50001, Specifies the IPv4
+# or IPv6 interface:port to bind to.
+tcp = 0.0.0.0:50001
+
+# *RECOMMENDED* - SSL bind - 'ssl' - DEFAULT: Noting, Specifies the IPv4 or IPv6
+# interface:port to bind to. The presence of this variable requires key= and
+# cert= both be specified.
+#ssl = 0.0.0.0:50002
+
+# *RECOMMENDED* - WS bind - 'ws' - DEFAULT: Nothing, Specifies the IPv4 or IPv6
+# interface:port to bind to for Web Socket support (ws://).
+#ws = 0.0.0.0:50003
+
+# *RECOMMENDED* - WSS bind - 'wss' - DEFAULT: Nothing, Specifies the IPv4 or
+# IPv6 interface:port to bind to for Web Socket Secure support (wss://). The
+# presence of this variable requires key= and cert= both be specified.
+#wss = 0.0.0.0:50004
+
+peering = false
+announce = false
+
+# SSL cert. - 'cert' - DEFAULT: None (required for SSL) (PEM, self-signed ok)
+#cert = /path/to/server-cert.pem
+
+# SSL private key - 'key' - DEFAULT: None (required for SSL) (PEM format)
+#key = /path/to/server-key.pem
+
+# WSS-specific cert. - 'wss-cert' - DEFAULT: None
+#wss-cert = /path/to/my-ca-signed-wss-fullchain.pem
+
+# Public SSL port - 'public_ssl_port' - DEFAULT: The first 'ssl' port configured
+#public_ssl_port = 50002
+
+# Public WS port - 'public_ws_port' - DEFAULT: The first 'ws' port configured
+#public_ws_port = 50003
+
+# Public WSS port - 'public_wss_port' - DEFAULT: The first 'wss' port configured
+#public_wss_port = 50004
+
+# Admin RPC bind - 'admin' - DEFAULT: None -- *REQUIRED* to use "FulcrumAdmin"
+# admin = 8000  # <-- 1.2.3.4:8000 notation also accepted here
+
+# Donation address - 'donation'
+# - DEFAULT: bitcoincash:qplw0d304x9fshz420lkvys2jxup38m9symky6k028
+# donation = bitcoincash:qplw0d304x9fshz420lkvys2jxup38m9symky6k028
+
+# Server banner text file - 'banner'
+# - DEFAULT: Send a static string "Connected to a Fulcruim xx.x server"
+#banner = /path/to/banner.txt
+
+# HTTP stats bind - 'stats' - DEFAULT: None
+# stats = 8080  # <-- 1.2.3.4:8080 notation also accepted here
+
+# Bitcoin daemon poll interval - 'polltime' - Default: 2.0 seconds
+#polltime = 2.0
+
+# Set this to a number in the 20-100 range if you want to limit memory
+# consumption to conservative levels at the expense of performance. Default
+# if unspecified is 20.
+#db_max_open_files = 20
+
+#-------------------------------------------------------------------------------
+# TOR Configuration (optional)
+#-------------------------------------------------------------------------------
+
+# This is optional but if tor_hostname is set and if at least one tor_*_port is
+# specified, we will also announce ourselves on Tor (if announce=true).
+#tor_hostname=aykwhy6o2o4ixushlonpjooqv73fwx7jqgoreiknnqxuqv4dwffmb7qd.onion
+
+# Tor banner is optional. If unset, will just user the regular banner=
+#tor_banner=/path/to/another/alternate/banner_tor.txt
+
+# Specify which port(s) are Tor. One of: tor_tcp_port, tor_ssl_port,
+# tor_ws_port, or tor_wss_port (or any combination thereof) must be specified
+# otherwise we won't announce ourselves via Tor.
+#tor_tcp_port = 50001
+#tor_ssl_port = 50002
+#tor_ws_port = 50003
+#tor_wss_port = 50004
+
+# The proxy server to use to discover/connect-to Tor peers.
+#tor_proxy = 9050   # e.g. localhost 9050. IP addr or hostname ok too: 10.0.0.1:9150, fooproxy.com:9050, etc.
+#tor_user =  # leave this out unless you specified this in your torrc
+#tor_pass =  # leave this out unless you specified this in your torrcn

nginx.conf → configs/nginx.conf

@@ -12,7 +12,7 @@ stream {
 
 	server {
 		listen 50002 ssl;
-		
+
 		proxy_pass electrum;
 
 		ssl_certificate /etc/nginx/cert.crt;
@@ -23,4 +23,3 @@ stream {
 		ssl_prefer_server_ciphers on;
 	}
 }
-

restapi-config.sh → configs/restapi-config.sh

@@ -3,7 +3,7 @@
 # My DO Testnet Insight Server
 export PORT=3000 
 export BITCOINCOM_BASEURL=http://opensight:3001/api/
-export RPC_BASEURL=http://bitcoind-regtest:18332/
+export RPC_BASEURL=http://bitcoind-regtest:18443/
 export RPC_PASSWORD=regtest
 export RPC_USERNAME=regtest
 export ZEROMQ_URL=0