diff --git a/README.md b/README.md index 6a6d3a654c..ea84a07d97 100755 --- a/README.md +++ b/README.md @@ -72,7 +72,7 @@ The easiest way to run Specter Desktop is by installing the Specter Desktop app, With this method, all you need to do is just download the right file for your operating system and install it like a normal desktop app (Debian buster is only [partially supported](https://github.com/cryptoadvance/specter-desktop/issues/769)) ### Installing Specter from Pip -* Specter requires Python version 3.6 to 3.8. We will support python 3.9 when HWI adds support for it. +* Specter requires Python version 3.7 to 3.9. * Bitcoin Core node should be at least v0.19+, better if it's the latest one from [bitcoincore.org](https://bitcoincore.org/en/download/). * HWI support requires `libusb` * Ubuntu/Debian: `sudo apt install libusb-1.0-0-dev libudev-dev python3-dev` diff --git a/docs/about.md b/docs/about.md new file mode 100644 index 0000000000..5a4a19908b --- /dev/null +++ b/docs/about.md @@ -0,0 +1,3 @@ + + +{{ git_site_authors }} \ No newline at end of file diff --git a/docs/architecture.md b/docs/architecture.md index 36f281f65b..1b45365466 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -1,21 +1,21 @@ -# Architectural Notes +# Architectural notes -specter-desktop and specter-cloud are flask-applications and follow typical Model-View-Control-principles. The model-part is covered by a number of business-objects and managers who are responsible for maintaining/managing those. Currently we have these managers: +Specter Desktop and Specter Cloud are flask applications and follow typical model-view-controller principles. The model part is covered by a number of business objects and managers who are responsible for maintaining/managing those. Currently we have these managers: ## Managers -* GenericDataManager is abstracting persistence-aspects and most managers which are persisting json-data are derived from this one. -* ConfigManager where the api-docs tells us: "The ConfigManager manages the configuration persisted in config.json. It's not supposed to have any side-effects. Setting and getting only with a lot of validation and computing while setting/getting." -* DeviceManager where the api-docs tells us: "A DeviceManager mainly manages the persistence of a device-json-structures compliant to helper.load_jsons.py". +* GenericDataManager is abstracting persistence aspects and most managers which are persisting json data are derived from this one. +* The ConfigManager manages the configuration persisted in config.json. It's not supposed to have any side-effects. Setting and getting only with a lot of validation and computing while setting/getting. +* A DeviceManager mainly manages the persistence of a device-json structure compliant to helper.load_jsons.py. * NodeManager manages internal and external nodes. * UserManager manages all stuff regarding Users. -* Similarly the WalletManager +* Similarly, the WalletManager manages all wallet stuff. -## Views / Blueprints -The controller is covered by "server_endpoints". Those are split up in “blueprints”. This is a way to split the urls in sub-urls. These sub-urls are loosely related to the business-objects: `/welcome`, `/auth`, `/devices`, `/nodes`, `/price`, `/services` (should be renamed "`/plugins`"), `/settings`, `/setup`, `/wallets`. +## Views and blueprints +The controller part is covered by "server_endpoints". Those are split up in “blueprints”. This is a way to split the urls in sub-urls. These sub-urls are loosely related to the business-objects: `/welcome`, `/auth`, `/devices`, `/nodes`, `/price`, `/services` (should be renamed "`/plugins`"), `/settings`, `/setup`, `/wallets`. ## Views -The views are effectively jinja2 templates which can extend from higher order ones and have placeholders which templates can then override. The `base.jinja` lays out the basic layout having a block for the `sidebar` and a `main` block (plus `head` and `scripts` ). Some blueprints (`/settings`,`/wallets`) span up a top-level-navigation. In that case, the template extend from its specific blueprint-base-template which in turn provides a block called `content` and the template decides which menu-point is active: +The views are effectively jinja2 templates which can extend from higher order ones and have placeholders which templates can then override. The `base.jinja` lays out the basic layout having a block for the `sidebar` and a `main` block (plus `head` and `scripts` ). Some blueprints (`/settings`,`/wallets`) span up a top-level navigation. In that case, the template extends from its specific blueprint-base template which in turn provides a block called `content` and the template decides which menu point is active: ``` {% extends "wallet/components/wallet_tab.jinja" %} {% set tab = 'receive' %} @@ -26,6 +26,6 @@ This is also a pattern used for plugins. # Plugins -The more specific a functionality became, the more awkward it felt to be integrated in the above core-Architecture. When we started to make exchange specific functionality, we wanted to protect the core-Architecture from deluting. Therefore we created a plugin-concept which allows to have the above concepts replicated in it there own self-contained standalone units. +The more specific a functionality became, the more awkward it felt to integrate it in the core architecture. When we started to make exchange specific functionality, we wanted to protect the core architecture. Therefore, we created a plugin concept which allows to have the above concepts replicated in their own self-contained standalone units. -We try now to implement bigger chunks of functionality in plugins. Maybe even core-functionality might be placed with "core-plugins" in the future. Those are placed in `src/specterext`. But plugins can also live in there own repos, have their own release-lifecycle and be used by Specter like any other dependency as well. For more information about plugins see [Third Party Service Integrations](./services/services.md). \ No newline at end of file +We now try to implement bigger chunks of functionality in plugins. Maybe, even core functionality might be implemented in "core plugins" in the future. Internal plugins are placed in `src/specterext`. But, plugins can also live in there own repos, have their own release lifecycle and be used by Specter like any other dependency. For more information about plugins see [Third Party Service Integrations](./extensions.md) where we also discuss the nuances between plugins and extensions. \ No newline at end of file diff --git a/docs/build-instructions.md b/docs/build-instructions.md index a9aa99fe94..2db57300d9 100644 --- a/docs/build-instructions.md +++ b/docs/build-instructions.md @@ -31,6 +31,11 @@ source .buildenv/bin/activate ### specterd Linux and MacOS +Below doesn't seem to work properly, at least on MacOS, better use the build script. For MacOS, that would be: +```bash +./utils/build-osx.sh --version 0.0.0-pre1 specterd +``` + ```bash cd pyinstaller # prerequisites diff --git a/docs/continuous-integration.md b/docs/continuous-integration.md index 6b367de8fd..bc10ef1b8c 100644 --- a/docs/continuous-integration.md +++ b/docs/continuous-integration.md @@ -11,7 +11,7 @@ Cirrus-CI: * supports the PR-model * quite easy to setup even though it's using docker -# Gitlab +## Gitlab Gitlab is a great CI/CD-platform and in the meantime it's quite easy to use it for GitHub-repositories. https://docs.gitlab.com/ee/ci/ci_cd_for_external_repos/github_integration.html @@ -32,17 +32,17 @@ start_bitcoind-function: * adding -rpcallowip= (from a docker network) to bitcoind * not use localhost but the docker-network-ip-address when talking to the bitcoind -# Travis-CI +## Travis-CI We're no longer using travis-ci due to the abuse-detection-system going wild on us. -# Cirrus-CI +## Cirrus-CI [Cirrus-CI](https://cirrus-ci.org) is used by Bitcoin-Core and HWI and is a quite good replacement for travis. We're using it only for PRs so far. The [../.cirrus.yml] file defines the build. We have two task, one for pytest and one for the [cypress-tests](./cypress-testing.md). -# Releasing +## Releasing -## What gets released +### What gets released We're mostly releasing automatically. Currently the following artifacts are released: * specterd (daemon) is a binary for kicking off the specter-desktop service on the command-line. We have binaries for windows, Linux and macOS @@ -50,13 +50,13 @@ We're mostly releasing automatically. Currently the following artifacts are rele * We release a pip-package * Usually some time after the release, the lncm is releasing [docker-images](https://hub.docker.com/r/lncm/specter-desktop). Very much appreciated, even though we can't guarantee for them, obviously. -# How we release +### How we release As we have a strict build-only-on-private-hardware build-policy, we're using GitLab private runners in order to build our releases. In order to test and develop the releasing automation, people can setup GitLab-projects which are syncing from their GitHub-forks. With such a setup it's possible to create test-releases and therefore test the whole procedure end-to-end. The automation of that kicks in if someone creates a tag which is named like "vX.Y.Z". This is specified in the gitlab-ci.yml. The release-job will only be triggered in cases of tags. One step will also check that the tag follows the convention above. The package upload will need a token. How to obtain the token is described in the packaging-tutorial. It's injected via GitLab-variables. ToDo: put the token on a trusted build-node. -## pyinstaller system-dependent binaries +### pyinstaller system-dependent binaries The [pyinstaller directory](../pyinstaller) contains scripts to create the platform-specific binaries (plus electron) to use specter-desktop as a desktop-software. Some of them are created and uploaded to [GitHub-releases](https://github.com/cryptoadvance/specter-desktop/releases) via more or less special build-agents. The [windows-build-agent](https://docs.gitlab.com/runner/install/windows.html) needs manual installation of git, python and docker. Docker is used to build the innosetup-file. @@ -65,7 +65,7 @@ log into the windows-machine to get docker started. Clearly there is an opportunity to move all of the creation of the windows-binary to wine on docker, similiar to the way the innosetup is running within docker. -# CI/CD-dev-env setup +## CI/CD-dev-env setup Here is a brief description on how to create a setup where the release-procedures can be tested: * We assume you have a fork of cryptoadvance/specter-desktop. We also assume that your GitLab-user-handle is the exact same as on GitHub. @@ -77,11 +77,11 @@ Here is a brief description on how to create a setup where the release-procedure * create a tag on your GitHub-fork * watch the test-release unfolding, ready to hack -# GitLab-runner setup (Windows) +### GitLab-runner setup (Windows) For Windows-releasing, we're using a windows GitLab-runner. Here is a short description on how to set one up. -## Prerequisites +#### Prerequisites You need at least Windows Home 10 which is up-to-date. The most complex dependency is setting up docker. Docker-Desktop needs a WSL2 which is a good idea to install on windows anyway. [Here](https://www.omgubuntu.co.uk/how-to-install-wsl2-on-windows-10) is a description on how to do that. @@ -96,7 +96,7 @@ Now open and check the "Environment-variables" and check that the following line ![](./images/continuous-integration_runner_windows_envvars.png) -## Runner +#### Runner The runner itself is easy to [setup](https://docs.gitlab.com/runner/install/windows.html). Follow the link or this very brief description: * `mkdir \Gitlab-Runner` diff --git a/docs/daemon.md b/docs/daemon.md index 1f1df0b15c..f80f08f1e4 100644 --- a/docs/daemon.md +++ b/docs/daemon.md @@ -68,7 +68,7 @@ WantedBy=multi-user.target You can check the status of specter.service by running `systemctl status specter.service` or for debugging, you can get more information by running `sudo journalctl -fu specter.service` -The commented section of the service file above refers to an optional reverse proxy setup which is covered in the [reverse_proxy.md](reverse_proxy.md) document. +The commented section of the service file above refers to an optional reverse proxy setup which is covered in the [reverse_proxy.md](reverse-proxy.md) document. ## bitcoind as a Service diff --git a/docs/development.md b/docs/development.md index 1d22d958e4..745e0d8eaa 100644 --- a/docs/development.md +++ b/docs/development.md @@ -84,7 +84,8 @@ sudo yum -y install libusb libudev-devel libffi libffi-devel openssl-devel && su ### Set up virtualenv -Note that `hwi-1.2.0` needs Python 3.6-3.8. If you have Python 3.9 installed then be sure to also install an old Python version and pass it to `virtualenv` (e.g. `virtualenv --python3.8 .env`). +Specter is using `hwi-2.1.0` which by now supports higher Python versions than Specter itself. Specter currently supports Python 3.7-3.9. +If you have Python 3.10 as your global version then be sure to also install an old Python version and pass it to `virtualenv` (e.g. `virtualenv --python=python3.8 .env`) or use pyenv. ```sh git clone https://github.com/cryptoadvance/specter-desktop.git diff --git a/docs/elements.md b/docs/elements.md index 526e60d494..4485614ea7 100644 --- a/docs/elements.md +++ b/docs/elements.md @@ -5,10 +5,10 @@ This document is a description on how to get started with elements/liquid. We'll After that, we'll explain how to connect your Specter Desktop to that node, create wallets and receive some coins (via sideshift.ai). Signing transactions is nowhere different than in any other Hotwallet. -# Elements Installation +## Elements Installation The Elements's instalaation is highly dependent to your system. Choose a fitting [artifact](https://github.com/ElementsProject/elements/releases) and install them. -# Liquid Node +## Liquid Node In order to validate Peg-Ins, you'll need RPC-access to your Bitcoin-Core node. I did this with a fullnode, it might also work with a pruned node (not tested, though). First, we need to create the `elements.conf` file which is located in the `datadir`. If you don't specify the datadir at startup, the standard-directory is `/.elements/elements.conf`. Here is an example: @@ -40,7 +40,7 @@ A successfull startup will result in validating the blocks. Currently the blockh It took about 12 hours for a full-sync in my case on a Intel Quadcore Gen 6 3.50GHz. -# Liquid Configuration in Specter +## Liquid Configuration in Specter You're probably familiar with this part. On the upper-left, click on the `two arrows -> connect a new node -> Connect existing node`. Here is a screenshots with the values fitting to the configuration above: @@ -54,7 +54,7 @@ As you can your wallets disappeared but your devices did not. The shown wallets ![](./images/elements/nodechoose.png) -# Hotwallet-Creation +## Hotwallet-Creation The process of creating a Liquid Hotwallet is very similiar to creating a Bitcoin-Core Hotwallet. First you need to create a Hotwallet-Device `Add new device -> Elements Core (hot wallet) -> Continue -> Enter Name -> Continue` (Don't forget to note the seed). After that, you can directly create a single key wallet: `Create single key wallet -> Enter Name -> Create Wallet`. You can download the usual Backup Pdf. @@ -63,7 +63,7 @@ So if you now get a receive-address, you have to choose between a Confidential a ![](./images/elements/receive.png) -# Fund the wallet +## Fund the wallet There are many ways you can fund your wallet. Let's assume you have Bitcoin and want to receive LiquidBtc (LBTC). At [https://sideswap.io/peg-in-out/](https://sideswap.io/peg-in-out/) you can Peg-In some Btc as a service. At [https://sideshift.ai/](https://sideshift.ai/) you can [https://sideshift.ai/btc/liquid](swap) BTC agains LBTC (and [vice versa](https://sideshift.ai/liquid/btc)). @@ -73,9 +73,9 @@ As an example, let'S choose sideshift.ai to swap some BTC to LBTC. ![](./images/elements/txs.png) -## Advanced stuff: Elements Compilation +### Advanced stuff: Elements Compilation -# Elements Compilation +## Elements Compilation Since `v1.7.0`, It's no longer needed to compile Elements yourself as Specter is now working with Elements [0.21.0](https://github.com/ElementsProject/elements/releases/tag/elements-0.21.0). However, maybe you want to compile it for some reason, so here is a quick guide on how to do that. diff --git a/docs/endless-pacman.md b/docs/endless-pacman.md index 38e6ce5bbd..8c2a0a5e9c 100644 --- a/docs/endless-pacman.md +++ b/docs/endless-pacman.md @@ -1,21 +1,23 @@ +# Introduction + Sometimes people have issues where they get an endless Pacman animation and the application is not coming up. If you suffer from this, here are some hints on how to deal with that. -# Check The Logs +## Check The Logs The logs are usually in the `.specter` subfolder of your homediretory. There, you might find a file called `specter.log` and/or specterApp.log. If you're running Specter as a binary application (in contrast to a pip installation) which most people do that `specterApp.log` is the relevant file for you. This file might contain content which gives a hint on what's wrong. If you can't find anything suspicious, feel free to create a [pastebin](https://pastebin.com/) and ask in the chat for help (with a link to the created pastebin). -# USB-issues +## USB-issues Sometimes some devices attached via USB are blocking the startup. We had that in the past with a Game-Controller. In one case, the USB drivers where so screwed up, that only a windows in protected mode could start Specter. -# Check Port 25441 +## Check Port 25441 Maybe there is another instance (still) running. Check that via opening your brower here: [http://localhost:25441](http://localhost:25441) If that's the case, the most easy solution is to reboot your computer. -# Check security software +## Check security software Sometimes, security software is distorting the startup. E.g. Acronis is a protective system which is known to prevent starting up of Specter on windows. Other security-software might be behaving similary. For troubleshooting purposes, switch off your protective software and try again. If that helps, you need to allow specter to be running on port 25441. Check the manual on how to achieve that manually. -# Check Whether the Binary is Existing +## Check Whether the Binary is Existing The first thing Specter is doing if you start up the app is downloading the correct specterd from the GitHub-release page and storing that executable in the `Homefolder/.specter/specterd-binaries` subfolder. You should find a file called `specterd`. If the file is there but you still get the endless Pacman, try one of the following things: * delete the file so it'll get redownloaded diff --git a/docs/extensions.md b/docs/extensions.md new file mode 100644 index 0000000000..fdbce40e36 --- /dev/null +++ b/docs/extensions.md @@ -0,0 +1,266 @@ +# Extensions + +A developer's guide for the Specter Desktop `Extension` framework. + +We currently rework the naming of extensions/plugins/services. If not otherwise stated, you can see those three terms as the same, for now. + +## TL;DR + +You can create an extension with an up to date Specter Desktop instance as simple as this: +``` +$ pip3 install cryptoadvance.specter --upgrade +$ mkdir /tmp/rubberduck && cd /tmp/rubberduck +$ python3 -m cryptoadvance.specter ext gen + + We need an id and a prefix for your extension. + The id should be a short string. + The prefix is usually your GitHub username + or GitHub organisation name. + Both will be used to to create a directory structure like this: + ./src/mycorpname/specterext/myextension + They will also be used when publishing this extension to pypi. + +Enter the id of your extension (lowercase only): rubberduck +Enter the prefix: mynym + + Note: Isolated client mode means that the extensions won't share the session cookie with + Specter Desktop and the integration only happens on the server side. + +Should the extension work in isolated client mode (y/n)?: n + --> Created requirements.txt + --> Created .gitignore + --> Created src/mynym/specterext/rubberduck/service.py + --> Created src/mynym/specterext/rubberduck/controller.py + --> Created src/mynym/specterext/rubberduck/config.py + --> Created src/mynym/specterext/rubberduck/__init__.py + --> Created src/mynym/specterext/rubberduck/__main__.py + --> Created src/mynym/specterext/rubberduck/templates/rubberduck/index.jinja + --> Created src/mynym/specterext/rubberduck/static/rubberduck/css/styles.css + --> Created src/mynym/specterext/rubberduck/static/rubberduck/img/ghost.png (via Github) + --> Created src/mynym/specterext/rubberduck/static/rubberduck/img/logo.jpeg (via Github) + --> Created src/mynym/specterext/rubberduck/templates/rubberduck/base.jinja + --> Created src/mynym/specterext/rubberduck/templates/rubberduck/transactions.jinja + --> Created src/mynym/specterext/rubberduck/templates/rubberduck/settings.jinja + --> Created src/mynym/specterext/rubberduck/templates/rubberduck/components/rubberduck_menu.jinja + --> Created src/mynym/specterext/rubberduck/templates/rubberduck/components/rubberduck_tab.jinja + --> Created pytest.ini + --> Created tests/conftest.py + --> Created tests/fix_ghost_machine.py + --> Created tests/fix_devices_and_wallets.py + --> Created tests/fix_testnet.py + --> Created tests/fix_keys_and_seeds.py + --> Created pyproject.toml + --> Created setup.py + --> Created setup.cfg + --> Created MANIFEST.in + + Congratulations, you've created a new extension! + + Here is how to get it to run in your development environment: + pip3 install -e . + python3 -m cryptoadvance.specter server --config DevelopmentConfig --debug + # Point your browser to http://localhost:25441 + # Click "Choose plugins" --> rubberduck + + If you want to package it, you can build it like this: + python3 -m pip install --upgrade build + python3 -m build + # Install it like this: + pip3 install dist/mynym_rubberduck-0.0.1-py3-none-any.whl + + If you want to bring your extension to production, please refer to + the readme in the dummy-extension repo: + https://github.com/cryptoadvance/specterext-dummy#how-to-get-this-to-production + + To publish your package + + python3 -m pip install --upgrade twine + python3 -m twine upload --repository testpypi dist/* + + You can get all this information again via: + python3 -m cryptoadvance.specter ext gen --help +$ +``` +The created file structure looks like this and you will feel right at home if you have some knowledge about how Flask works: + + +![](./images/extensions_file_layout.png) + + + + +## Concept +As much as possible, each `extension` should be entirely self-contained with little or no custom code altering core Specter functionality. There is a name for that: Extension framework. +The term `extension` will be used for all sorts of extensions whereas `plugin` will be used as a component which can be de-/activated by a user. + +All extensions are completely seperated in a specific folder structure. There are internal extensions which SHOULD be located in `cryptoadvance.specterext.id_of_extension` but at least 2 extensions are still at the deprecated location of `cryptoadvance.specter.services`. However, that does not mean that an extension needs to be located in the same repository as Specter itself. Extensions can be located in their own repository even if they are incorporated into the official Specter release. + +Independent of whether an extension is shipped with the official Specter-release binaries and whether it's an internal (which is shipped) or external extension (which might be shipped), the creation of extensions is already heavily supported and encouraged. +Whether an extension is shipped with the official binary is ultimately a choice of the Specter team. However, you can simply develop extensions and use them on production (only for technical personel) as described in `specterext-dummy` (see below). + +A description of how to create your own extension can be found above. + +All the attributes of an extension are currently (json support is planned) defined as attributes of a class which is derived from the class `Service` (should be renamed). That class has attributes which are essential. So let's discuss them briefly. + +## Extension attributes +Here is an example. This class definition MUST be stored in a file called "service.py" within a package with the name `org-id.specterext.extions-id`. +``` +class DiceService(Service): + id = "dice" + name = "Specter Dice" + icon = "dice/dice_logo.png" + logo = "dice/dice_logo.png" + desc = "Send your bet!" + has_blueprint = True + blueprint_module = "k9ert.specterext.dice.controller" + isolated_client = False + devstatus = devstatus_alpha +``` +This defines the base `Service` class (to be renamed to "Extension") that all extensions must inherit from. This also enables `extension` auto-discovery. Any feature that is common to most or all `Service` integrations should be implemented here. +With inheriting from `Service` you get some usefull methods explained later. + +The `id` needs to be unique within a specific specter-instance where this extension is part of. The `name` is the displayname as shown to the user in the plugin-area (currently there is not yet a technical difference between extensions and plugins). The `icon` will be used where labels are used to be diplayed if this extension is reserving addresses. The `logo` and the `desc`ription is also used in the plugin-area ("choose plugins"). +If the extension has a UI (currently all of them have one), `has_blueprint` is True. `The blueprint_module` is referencing the controller module where endpoints are defined. It's recommended to follow the format `org.specterext.extions-id.controller`. +`isolated_client` should not be used yet. It is determining where in the url-path tree the blueprint will be mounted. This might have an impact on whether the extension's frontend client has access to the cookie used in Specter. Check `config.py` for details. +`devstatus` is one of `devstatus_alpha`, `devstatus_beta` or `devstatus_prod` defined in `cryptoadvance.specter.services.service`. Each Specter instance will have a config variable called `SERVICES_DEVSTATUS_THRESHOLD` (prod in Production and alpha in Development) and depending on that, the plugin will be available to the user. + +## Frontend aspects + +As stated, you can have your own frontend with a blueprint. If you only have one, it needs to have a `/` route in order to be linkable from the `choose your plugin` page. +If you create your extension with a blueprint, it'll also create a controller for you which, simplified, looks like this: +``` +rubberduck_endpoint = ScratchpadService.blueprint + +def ext() -> ScratchpadService: + ''' convenience for getting the extension-object''' + return app.specter.ext["rubberduck"] + +def specter() -> Specter: + ''' convenience for getting the specter-object''' + return app.specter + + +@rubberduck.route("/") +@login_required +@user_secret_decrypted_required +def index(): + return render_template( + "rubberduck/index.jinja", + ) +[...] +``` + But you can also have more than one blueprint. Define them like this in your service class: +``` + blueprint_modules = { + "default" : "mynym.specterext.rubberduck.controller", + "ui" : "mynym.specterext.rubberduck.controller_ui" + } +``` +You have to have a default blueprint which has the above mentioned index page. +In your controller, the endpoint needs to be specified like this: +``` +ui = RubberduckService.blueprints["ui"] +``` + +You might have an extension which wants to inject e.g. javascript code into each and every page of specter-desktop. The extension needs to be activated for the user, though. You can do that via overwriting one of the `inject_in_basejinja_*` methods in your service-class: +``` + @classmethod + def inject_in_basejinja_head(cls): + ''' e.g. rendering some snippet ''' + return render_template("devhelp/html_inject_in_basejinja_head.jinja") + + @classmethod + def inject_in_basejinja_body_top(cls): + ''' or directly returning text ''' + return " + {% include "services/inject_in_basejinja_head.jinja" %} {% block head %} {% endblock %} + {% include "services/inject_in_basejinja_body_top.jinja" %}
{% if specter.config.auth.method == "none" or current_user.is_authenticated %} @@ -329,5 +331,6 @@ {% include "includes/language/language_js.jinja" %} {% endblock %} + {% include "services/inject_in_basejinja_body_bottom.jinja" %} diff --git a/src/cryptoadvance/specter/templates/includes/address-label.html b/src/cryptoadvance/specter/templates/includes/address-label.html index f0f0a1040f..fc3a8f6c08 100644 --- a/src/cryptoadvance/specter/templates/includes/address-label.html +++ b/src/cryptoadvance/specter/templates/includes/address-label.html @@ -227,6 +227,7 @@ disableExplorerLink() { this.explorerLink.classList.remove('explorer-link'); + this.explorerLink.removeAttribute('href') this.explorerLink.style['text-decoration'] = 'none'; this.explorerLink.onclick = () => {return false;} } diff --git a/src/cryptoadvance/specter/templates/services/inject_in_basejinja_body_bottom.jinja b/src/cryptoadvance/specter/templates/services/inject_in_basejinja_body_bottom.jinja new file mode 100644 index 0000000000..db108accf7 --- /dev/null +++ b/src/cryptoadvance/specter/templates/services/inject_in_basejinja_body_bottom.jinja @@ -0,0 +1,8 @@ +{% for _,service in specter.service_manager.services.items() %} + {% if service.id in current_user.services %} + {% set html_inject_in_basejinja = service.inject_in_basejinja_body_bottom() %} + {% if html_inject_in_basejinja %} + {{ html_inject_in_basejinja | safe }} + {% endif %} + {% endif %} +{% endfor %} \ No newline at end of file diff --git a/src/cryptoadvance/specter/templates/services/inject_in_basejinja_body_top.jinja b/src/cryptoadvance/specter/templates/services/inject_in_basejinja_body_top.jinja new file mode 100644 index 0000000000..06930fe788 --- /dev/null +++ b/src/cryptoadvance/specter/templates/services/inject_in_basejinja_body_top.jinja @@ -0,0 +1,8 @@ +{% for _,service in specter.service_manager.services.items() %} + {% if service.id in current_user.services %} + {% set html_inject_in_basejinja = service.inject_in_basejinja_body_top() %} + {% if html_inject_in_basejinja %} + {{ html_inject_in_basejinja | safe }} + {% endif %} + {% endif %} +{% endfor %} \ No newline at end of file diff --git a/src/cryptoadvance/specter/templates/services/inject_in_basejinja_head.jinja b/src/cryptoadvance/specter/templates/services/inject_in_basejinja_head.jinja new file mode 100644 index 0000000000..124c5ce69e --- /dev/null +++ b/src/cryptoadvance/specter/templates/services/inject_in_basejinja_head.jinja @@ -0,0 +1,8 @@ +{% for _,service in specter.service_manager.services.items() %} + {% if service.id in current_user.services %} + {% set html_inject_in_basejinja = service.inject_in_basejinja_head() %} + {% if html_inject_in_basejinja %} + {{ html_inject_in_basejinja | safe }} + {% endif %} + {% endif %} +{% endfor %} \ No newline at end of file diff --git a/src/cryptoadvance/specter/util/common.py b/src/cryptoadvance/specter/util/common.py index 65d15f45a4..8589b7fe40 100644 --- a/src/cryptoadvance/specter/util/common.py +++ b/src/cryptoadvance/specter/util/common.py @@ -1,4 +1,10 @@ +import logging import re +from datetime import datetime +import json +from flask_babel.speaklater import LazyString + +logger = logging.getLogger(__name__) def str2bool(my_str): @@ -21,3 +27,19 @@ def camelcase2snake_case(name): def snake_case2camelcase(word): return "".join(x.capitalize() or "_" for x in word.split("_")) + + +def robust_json_dumps(obj): + def default(o): + if isinstance(o, datetime): + return o.timestamp() + if isinstance(o, set): + return list(o) + if isinstance(o, LazyString): + return str(o) + logger.warning( + f"robust_json_dumps could not convert {o} of type {type(o)}. Converting to string instead." + ) + return str(o) + + return json.dumps(obj, default=default) diff --git a/src/cryptoadvance/specter/util/version.py b/src/cryptoadvance/specter/util/version.py index 1b14ec121b..3ab30e1db0 100644 --- a/src/cryptoadvance/specter/util/version.py +++ b/src/cryptoadvance/specter/util/version.py @@ -159,7 +159,6 @@ def _get_latest_version_from_github(self): requests_session = self.specter.requests_session(force_tor=False) else: requests_session = requests.Session() - print(requests_session.get().json()) releases = ( requests_session.get(f"https://pypi.org/pypi/{self.name}/json") .json()["releases"] diff --git a/src/cryptoadvance/specterext/devhelp/config.py b/src/cryptoadvance/specterext/devhelp/config.py new file mode 100644 index 0000000000..f84229b0c9 --- /dev/null +++ b/src/cryptoadvance/specterext/devhelp/config.py @@ -0,0 +1,13 @@ +""" +Here Configuration of your Extension takes place +""" + + +class BaseConfig: + """This is a extension-based Config which is used as Base""" + + DEVELOPER_JAVASCRIPT_PYTHON_CONSOLE = False + + +class DevelopmentConfig(BaseConfig): + DEVELOPER_JAVASCRIPT_PYTHON_CONSOLE = True diff --git a/src/cryptoadvance/specterext/devhelp/console.py b/src/cryptoadvance/specterext/devhelp/console.py new file mode 100644 index 0000000000..b1af97395f --- /dev/null +++ b/src/cryptoadvance/specterext/devhelp/console.py @@ -0,0 +1,58 @@ +# source: http://stackoverflow.com/questions/2758159/how-to-embed-a-python-interpreter-in-a-pyqt-widget + +import traceback +import logging + +logger = logging.getLogger(__name__) + + +class Console: + def __init__(self): + self.namespace = {} + self.updateNamespace({"run": self.run_script}) + + def run_script(self, filename): + with open(filename) as f: + script = f.read() + return self.exec_command(script) + + def updateNamespace(self, namespace): + self.namespace.update(namespace) + # adding help message with the available keys + self.namespace.update( + { + "help": f"You can try calling any of the following objects: {list(self.namespace.keys())}\n" + f"Use a trainling '.' to get available methods and propeties, e.g. try: '{list(self.namespace.keys())[-1]}.'\n" + "The command 'run' can execute python scripts, e.g., run('myscript.py')" + } + ) + + def exec_command(self, command): + if callable(self.namespace.get(command)): + return "'{}' is a function. Type '{}()' to use it in the Python console.".format( + command, command + ) + + try: + try: + # eval is generally considered bad practice. use it wisely! + logger.info(f'Executing console command "{command}"') + if command.endswith("."): + return { + "vars": eval( + f"vars({command[:-1]})", self.namespace, self.namespace + ), + "dir": eval( + f"dir({command[:-1]})", self.namespace, self.namespace + ), + } + return eval(command, self.namespace, self.namespace) + except SyntaxError: + # exec is generally considered bad practice. use it wisely! + return exec(command, self.namespace, self.namespace) + except BaseException: + traceback_lines = traceback.format_exc().split("\n") + # Remove traceback mentioning this file, and a linebreak + for i in (3, 2, 1, -1): + traceback_lines.pop(i) + return "\n".join(traceback_lines) diff --git a/src/cryptoadvance/specterext/devhelp/controller.py b/src/cryptoadvance/specterext/devhelp/controller.py index d4c72acc77..9ac49267f4 100644 --- a/src/cryptoadvance/specterext/devhelp/controller.py +++ b/src/cryptoadvance/specterext/devhelp/controller.py @@ -6,7 +6,7 @@ from cryptoadvance.specter.services.controller import user_secret_decrypted_required from .service import DevhelpService from cryptoadvance.specter.wallet import Wallet - +from cryptoadvance.specter.util.common import robust_json_dumps logger = logging.getLogger(__name__) @@ -24,17 +24,22 @@ def index(): @devhelp_endpoint.route("/html/") @login_required -@user_secret_decrypted_required def html_component(html_component): associated_wallet: Wallet = DevhelpService.get_associated_wallet() return render_template( f"devhelp/html/{html_component}", wallet=associated_wallet, services=app.specter.service_manager.services, - address=associated_wallet.get_address(3), + address=associated_wallet.get_address(3) if associated_wallet else None, ) +@devhelp_endpoint.route("/console", methods=["GET"]) +@login_required +def console(): + return render_template("devhelp/dev-console.jinja") + + @devhelp_endpoint.route("/settings", methods=["GET"]) @login_required @user_secret_decrypted_required @@ -68,3 +73,47 @@ def settings_post(): wallet = current_user.wallet_manager.get_by_alias(used_wallet_alias) DevhelpService.set_associated_wallet(wallet) return redirect(url_for(f"{DevhelpService.get_blueprint_name()}.settings_get")) + + +NEVER_CALLED_PYTHON_COMMAND = True + + +@devhelp_endpoint.route("/python_command", methods=["POST"]) +@login_required +def python_command(): + global NEVER_CALLED_PYTHON_COMMAND + if NEVER_CALLED_PYTHON_COMMAND: + NEVER_CALLED_PYTHON_COMMAND = False + return robust_json_dumps( + "!!!DANGER!!!\n" + "This command allows arbitrary access to Specter.\n" + "You can irreparabily damage your specter configuration and\n" + "all funds on !!!HOT!!! wallets can be lost!\n" + "Please use it with extreme care!!! Run your command again to execute it.\n\n" + "--> Never copy&paste anything you do not FULLY understand. <--" + ) + if current_user != "admin" or not app.specter.user_manager.user.is_admin: + return robust_json_dumps(f"Access forbidden for user '{current_user}'!") + if not app.config["DEVELOPER_JAVASCRIPT_PYTHON_CONSOLE"]: + return robust_json_dumps( + "DEVELOPER_JAVASCRIPT_PYTHON_CONSOLE disabled in Specter configuration. " + "This is an advanced option and should be used with great care!!!" + ) + if request.method != "POST": + return robust_json_dumps("Not a 'POST' command.") + + # The following commented lines are a further restriction of this endpoint, by limiting it only to regtest and testnet + # uncomment these lines to enable the restriction + # ---------------------------------------------- + # allowed_chains = ['regtest', 'testnet', 'liquidtestnet', 'liquidregtest'] + # if app.specter.chain not in allowed_chains: + # return robust_json_dumps(f"This command is only allowed for {allowed_chains}. " + # "The current chain is {app.specter.chain}") + # ---------------------------------------------- + + command = request.form["command"] + result = DevhelpService.console.exec_command(command) + try: + return robust_json_dumps(result) + except: + return robust_json_dumps(str(result)) diff --git a/src/cryptoadvance/specterext/devhelp/service.py b/src/cryptoadvance/specterext/devhelp/service.py index 2f6640c95f..c4ccdafbdd 100644 --- a/src/cryptoadvance/specterext/devhelp/service.py +++ b/src/cryptoadvance/specterext/devhelp/service.py @@ -1,8 +1,12 @@ import logging from flask import current_app as app +from flask import render_template from cryptoadvance.specter.services.service import Service, devstatus_alpha from cryptoadvance.specter.specter_error import SpecterError from cryptoadvance.specter.wallet import Wallet +from .console import Console +import flask +import flask_login logger = logging.getLogger(__name__) @@ -16,6 +20,14 @@ class DevhelpService(Service): has_blueprint = True blueprint_module = "cryptoadvance.specterext.devhelp.controller" devstatus = devstatus_alpha + console = Console() + console.updateNamespace( + { + "app": app, + "flask": flask, + "flask_login": flask_login, + } + ) sort_priority = 2 @@ -43,3 +55,7 @@ def get_associated_wallet(cls) -> Wallet: def set_associated_wallet(cls, wallet: Wallet): """Set the Specter `Wallet` that is currently associated with Swan auto-withdrawals""" cls.update_current_user_service_data({cls.SPECTER_WALLET_ALIAS: wallet.alias}) + + @classmethod + def inject_in_basejinja_body_top(cls): + return render_template("devhelp/html_inject_in_basejinja.jinja") diff --git a/src/cryptoadvance/specterext/devhelp/templates/devhelp/components/devhelp_menu.jinja b/src/cryptoadvance/specterext/devhelp/templates/devhelp/components/devhelp_menu.jinja index d2e6578118..f866472127 100644 --- a/src/cryptoadvance/specterext/devhelp/templates/devhelp/components/devhelp_menu.jinja +++ b/src/cryptoadvance/specterext/devhelp/templates/devhelp/components/devhelp_menu.jinja @@ -1,13 +1,14 @@ {% from 'components/menu_item.jinja' import menu_item %} {# - vaultoro_menu - Tabs menu to navigate between the vaultoro screens. + devhelp_menu - Tabs menu to navigate between the vaultoro screens. Parameters: - active_menuitem: Current active tab. Options: 'general', 'bitcoin_core', 'auth'. #} {% macro devhelp_menu(active_menuitem) -%}