feat(docker): allow users to use Zebra + LWD with persistent states

[gustavovalverde]

Motivation

We want to make it easy for users to use Zebra and Lightwalletd without the risk of losing their cached states, and avoiding complexity when running both tools together.

Fixes #8023

Thanks to @AArnott for putting up this guide, as it served as the foundation for building these compose files.

PR Author Checklist

Check before marking the PR as ready for review:

• Will the PR name make sense to users?
• Does the PR have a priority label?
• Have you added or updated tests?
• Is the documentation up to date?

For significant changes:

• Is there a summary in the CHANGELOG?
• Can these changes be split into multiple PRs?

If a checkbox isn't relevant to the PR, mark it as done.

Complex Code or Requirements

• A Health-check was needed for Zebra, so the Lightwalletd service doesn't start until Zebra is ready
• The default LWD has to be run unsecurely with --no-tls-very-insecure as we can't create certificates for the user
• curl is required in our production image to allow the health-check to work

Solution

• Create a docker-compose.yml to run an isolated Zebra node
• Create a docker-compose.lwd.yml to run a lightwalletd server alongside Zebra
• Create a zcash.conf file for lightwalletd
• Move unused production environment variables from test.env to .env

Testing

This can be tested running:

docker-compose -f docker/docker-compose.yml up

For a Zcash Node, and change its behavior in .env

docker-compose -f docker/docker-compose.yml -f docker/docker-compose.lwd.yml up

To run LWD + Zcash

Review

Note: This is not ready until respective user documentation is updated

Reviewer Checklist

Check before approving the PR:

• Does the PR scope match the ticket?
• Are there enough tests to make sure it works? Do the tests cover the PR motivation?
• Are all the PR blockers dealt with?
  PR blockers can be dealt with in new tickets or PRs.

And check the PR Author checklist is complete.

[AArnott]

I'm confused at how you've created two docker-compose files and how you've distributed the yaml across them. In my blog post I used just one file. I've never seen it broken out like this, so I'm probably just ignorant and my comments may be wrong. But hopefully at least one of us will come out having learned something. :)

[gustavovalverde]

I'm confused at how you've created two docker-compose files and how you've distributed the yaml across them

@AArnott that's a really good question, this is a docker compose functionality https://docs.docker.com/compose/multiple-compose-files/merge/

As our users might have different needs, we've created 2 docker-compose yaml:

1. docker-compose.yml which runs the Zebra node completely isolated, and
2. docker-compose.lwd.yml which inherits and overrides the first one

A user might decide to just run the Zebra node with docker-compose -f docker/docker-compose.yml up or running Zebra + LWD together, with docker-compose -f docker/docker-compose.yml -f docker/docker-compose.lwd.yml up using both files.

[AArnott]

a user might decide to just run the Zebra node

That sounds good. Then why are zebra-specific things only put into the lwd file? A zebra-only container should still know its own ports and healthcheck, shouldn't it?

[gustavovalverde]

A zebra-only container should still know its own ports and healthcheck, shouldn't it?

@AArnott Yes, I did this as the healtcheck depends on port 8232 (the RPC endpoint) to be opened. But we don't always want this port opened by default on a Zebra node. Though it's required if we want to have lightwalletd.