Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DV Setup: Charon Integration with Sedge #368

Merged
merged 75 commits into from
Nov 14, 2024

Conversation

sugh01
Copy link
Contributor

@sugh01 sugh01 commented Mar 26, 2024

Changes:

Charon is used by stakers to distribute the responsibility of running Ethereum Validators across a number of different instances and client implementations. Setting up and running a full ethereum node with charon, needs some learning curve and compatibility knowledge, in order for the setup to be fully compliant with the charon configuration requirements for different BN-VC combinations. We want to provide a better and guided user experience for setting up a DV with Charon.

Sedge, is a one-click ethereum node setup tool which focuses on ease of use and, as such, lowers the entry barrier for non-technical users. With Sedge, one can run a node or a validator on Ethereum with little to no technical background.
Integrating Charon with Sedge would make it easy for stakers to setup and run a DV with Charon without having to go through each individual client setup docs and their compatibility with DVT.

Please refer Design Doc here - https://docs.google.com/document/d/1nQsOmkd-od3-lej3OcdUgVZ0GYXAHK9GZqlwEoyIzSY/edit

Types of changes

What types of changes does your code introduce?
Put an x in the boxes that apply

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation Update
  • Code style update (formatting, renaming)
  • Refactoring (no functional changes, no api changes)
  • Build related changes
  • Other (please describe):

Testing

Requires testing

  • Yes
  • No

In case you checked yes, did you write tests?

  • Yes
  • No

Comments about testing , should you have some (optional)

Further comments (optional)

How to test

Checkout branch
Perform Charon DKG process, a .charon folder is created

make compile
sudo cp build/sedge /usr/local/bin/
sedge generate full-node --validator=lodestar --network=holesky --distributed
sedge import-key --from keystore --distributed -n holesky lodestar
sedge run

@sugh01
Copy link
Contributor Author

sugh01 commented Apr 5, 2024

I was having a problem importing keys.
Charon DKG process outputs a folder which has a subfolder validator_keys inside it.
I saved the folder in the root.
And then ran the commands -
sedge generate full-node --validator=lodestar --network=holesky --distributed
sudo sedge import-key --from node0/validator_keys -n holesky lodestar
But i got the below error -
2024-04-05 13:57:51 -- [INFO] [Logger Init] Log level: info 2024-04-05 13:57:51 -- [WARN] A new Version of sedge is available. Please update to the latest Version. See https://github.com/NethermindEth/sedge/releases for more information. Latest detected tag: v0.0.0 2024-04-05 13:57:52 -- [INFO] Setting up containers 2024-04-05 13:57:52 -- [INFO] Running command: docker compose -f /Users/sg/obol/nethermind/sedge/sedge-data/docker-compose.yml build validator 2024-04-05 13:57:52 -- [INFO] Running command: docker compose -f /Users/sg/obol/nethermind/sedge/sedge-data/docker-compose.yml pull validator [+] Pulling 0/0 ⠋ validator Pulling 0.1s [+] Pulling 0/17:52 must use ASL logging (which requires CGO) if running ⠙ validator Pulling 0.2s error getting credentials - err: exit status 1, out: `` 2024-04-05 13:57:52 -- [FATA] exit status 1

node0.zip

@matilote matilote requested a review from stdevMac May 1, 2024 13:33
@adriantpaez
Copy link
Contributor

adriantpaez commented May 16, 2024

Hi @sugh01. The issue is related to the use of sudo with the sedge import-key command. I recreated the scenario locally (without sudo) and attempted to import the keys from the provided node0 directory, which resulted in the following error:

$ sedge import-key --from node0/validator_keys -n holesky lodestar
2024-05-16 12:40:34 -- [INFO] [Logger Init] Log level: info
2024-05-16 12:40:34 -- [INFO] You are running the latest version of sedge. Version:  v1.3.2
2024-05-16 12:40:35 -- [INFO] Setting up containers
2024-05-16 12:40:35 -- [INFO] Running command: docker compose -f /abs/path/to/sedge-data/docker-compose.yml build validator
2024-05-16 12:40:35 -- [INFO] Running command: docker compose -f /abs/path/to/sedge-data/docker-compose.yml pull validator
[+] Pulling 1/1
 ✔ validator Pulled                                                                                           1.3s 
2024-05-16 12:40:36 -- [INFO] Running command: docker compose -f /abs/path/to/sedge-data/docker-compose.yml create validator
[+] Creating 2/0
 ✔ Container sedge-validator-blocker  Created                                                                 0.0s 
 ✔ Container sedge-validator-client   Created                                                                 0.0s 
2024-05-16 12:40:36 -- [WARN] The keys path is not the default one, copying the keys to the default path /abs/path/to/sedge-data/keystore
2024-05-16 12:40:36 -- [INFO] Importing validator keys
2024-05-16 12:40:36 -- [INFO] The keys import container is starting
2024-05-16 12:40:38 -- [FATA] validator import container exited with non-zero exit code: validator-import service container with id 85b3e4a24962a0923f93754a8855c0575d768f7a75bb4191718ba926b8cbafc8 ends with status code 1. Here are the logs for more details:
 ✖ Error: ENOENT: no such file or directory, open '/keystore/keystore_password.txt'
    at Object.readFileSync (node:fs:453:20)
    at readPassphraseFile (file:///usr/app/packages/cli/src/util/passphrase.ts:9:19)
    at readPassphraseOrPrompt (file:///usr/app/packages/cli/src/cmds/validator/signers/importExternalKeystores.ts:30:12)
    at Object.handler (file:///usr/app/packages/cli/src/cmds/validator/import.ts:61:23)
    at file:///usr/app/node_modules/yargs/build/lib/command.js:206:54
    at maybeAsyncResult (file:///usr/app/node_modules/yargs/build/lib/utils/maybe-async-result.js:9:15)
    at CommandInstance.handleValidationAndGetResult (file:///usr/app/node_modules/yargs/build/lib/command.js:205:25)
    at CommandInstance.applyMiddlewareAndGetResult (file:///usr/app/node_modules/yargs/build/lib/command.js:245:20)
    at CommandInstance.runCommand (file:///usr/app/node_modules/yargs/build/lib/command.js:128:20)
    at YargsInstance.[runYargsParserAndExecuteCommands] (file:///usr/app/node_modules/yargs/build/lib/yargs-factory.js:1386:105)

To import the keys correctly, provide the keystore password in the keystore_password.txt file.

Example of successfully import

This is a keystore directory keystore.zip, generated with sedge keys command, which will be used for the imprt-key in this example. This is the structure of the keystore directory:

$ tree keystore 
keystore
├── deposit_data.json
├── keystore_password.txt
└── validator_keys
    └── keystore-m_12381_3600_0_0_0.json

2 directories, 3 files
  1. sedge generate full-node --validator=lodestar --network=holesky --distributed. This command generates the following files in the sedge-data directory:
$ tree sedge-data
sedge-data
├── docker-compose.yml
└── jwtsecret

1 directory, 2 files
  1. sedge import-key --from keystore -n holesky lodestar This command produces these logs:
sedge import-key --from keystore -n holesky lodestar
2024-05-16 12:59:10 -- [INFO] [Logger Init] Log level: info
2024-05-16 12:59:10 -- [INFO] You are running the latest version of sedge. Version:  v1.3.2
2024-05-16 12:59:10 -- [INFO] Setting up containers
2024-05-16 12:59:10 -- [INFO] Running command: docker compose -f /abs/path/to/sedge-data/docker-compose.yml build validator
2024-05-16 12:59:10 -- [INFO] Running command: docker compose -f /abs/path/to/sedge-data/docker-compose.yml pull validator
[+] Pulling 1/1
 ✔ validator Pulled                                                                                           1.4s 
2024-05-16 12:59:12 -- [INFO] Running command: docker compose -f /abs/path/to/sedge-data/docker-compose.yml create validator
[+] Creating 2/0
 ✔ Container sedge-validator-blocker  Created                                                                 0.1s 
 ✔ Container sedge-validator-client   Created                                                                 0.0s 
2024-05-16 12:59:12 -- [WARN] The keys path is not the default one, copying the keys to the default path /abs/path/to/sedge-data/keystore
2024-05-16 12:59:12 -- [INFO] Importing validator keys
2024-05-16 12:59:12 -- [INFO] The keys import container is starting
2024-05-16 12:59:15 -- [INFO] Validator keys imported successfully

And the sedge-data directory is updated with the imported keystore.

$ tree sedge-data 
sedge-data
├── docker-compose.yml
├── jwtsecret
├── keystore
│   ├── deposit_data.json
│   ├── keystore_password.txt
│   └── validator_keys
│       └── keystore-m_12381_3600_0_0_0.json
└── validator-data
    ├── keystores
    │   └── 0xb53a59a1a5f59adc0d33686164bb90dcadfabbe3000bc95d3ac738b8269d80acb1bdb081a8f23592b6cd21e00bb86f94
    │       └── voting-keystore.json
    └── secrets
        └── 0xb53a59a1a5f59adc0d33686164bb90dcadfabbe3000bc95d3ac738b8269d80acb1bdb081a8f23592b6cd21e00bb86f94

7 directories, 7 files

@adriantpaez
Copy link
Contributor

It appears that the import process is working fine, but the provided keystore directory structure is not correct

@sugh01
Copy link
Contributor Author

sugh01 commented Jun 19, 2024

Hi @adriantpaez
While testing the key import process, i ran into an error when trying with Lighthouse.

Steps:

  1. sedge generate full-node --validator=lighthouse --network=holesky
  2. sedge keys
sedge-data
├── docker-compose.yml
├── jwtsecret
└── keystore
    ├── deposit_data.json
    ├── keystore_password.txt
    └── validator_keys
        └── keystore-m_12381_3600_0_0_0.json
  1. sedge import-key -n holesky lighthouse

This step does not complete.

2024-06-19 13:38:45 -- [INFO] [Logger Init] Log level: info
2024-06-19 13:38:45 -- [INFO] You are running the latest version of sedge. Version:  v1.3.2
2024-06-19 13:38:45 -- [INFO] Setting up containers
2024-06-19 13:38:45 -- [INFO] Running command: docker compose -f /Users/sg/sedge/sedge/sedge-data/docker-compose.yml build validator
2024-06-19 13:38:45 -- [INFO] Running command: docker compose -f /Users/sg/sedge/sedge/sedge-data/docker-compose.yml pull validator
[+] Pulling 1/1
 ✔ validator Pulled                                                                                                                                    1.3s 
2024-06-19 13:38:55 -- [INFO] Running command: docker compose -f /Users/sg/sedge/sedge/sedge-data/docker-compose.yml create validator
[+] Creating 3/0
 ✔ Network sedge-network              Created                                                                                                          0.0s 
 ✔ Container sedge-validator-blocker  Created                                                                                                          0.0s 
 ✔ Container sedge-validator-client   Created                                                                                                          0.0s 
2024-06-19 13:38:55 -- [INFO] Running command: docker build -t sedge/validator-import-lighthouse --build-arg LH_VERSION=sigp/lighthouse:v5.0.0 --build-arg NETWORK=holesky /var/folders/ng/dv8zg6ld6nvdrbqmysj1klxh0000gn/T/sedge-validator-import-context-3819293176
[+] Building 1.0s (11/11) FINISHED                                                                                                     docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                                                                   0.0s
 => => transferring dockerfile: 529B                                                                                                                   0.0s
 => [internal] load metadata for docker.io/library/debian:stable-20221004-slim                                                                         0.9s
 => [internal] load .dockerignore                                                                                                                      0.0s
 => => transferring context: 2B                                                                                                                        0.0s
 => [1/7] FROM docker.io/library/debian:stable-20221004-slim@sha256:92ed72016796475ea1f18f84cd8e2f8519ece3a9ea27fdde1157464078ea5371                   0.0s
 => [internal] load build context                                                                                                                      0.0s
 => => transferring context: 677B                                                                                                                      0.0s
 => CACHED [2/7] RUN apt-get update && apt-get install -y curl                                                                                         0.0s
 => CACHED [3/7] RUN version=$(echo sigp/lighthouse:v5.0.0 | cut -d':' -f 2) && curl -L https://github.com/sigp/lighthouse/releases/download/$version  0.0s
 => CACHED [4/7] RUN tar -xvf lh.gz                                                                                                                    0.0s
 => CACHED [5/7] COPY validator-init.sh .                                                                                                              0.0s
 => CACHED [6/7] RUN chmod +x validator-init.sh                                                                                                        0.0s
 => exporting to image                                                                                                                                 0.0s
 => => exporting layers                                                                                                                                0.0s
 => => writing image sha256:68726c147b82af43d6a88a143b1d5560d3f67fee8e14f14950824c8a965c22c8                                                           0.0s
 => => naming to docker.io/sedge/validator-import-lighthouse                                                                                           0.0s

View build details: docker-desktop://dashboard/build/desktop-linux/desktop-linux/yknf6rtsfipgqfwzfzjngzr8d

What's next:
    View a summary of image vulnerabilities and recommendations → docker scout quickview 
2024-06-19 13:38:56 -- [INFO] Importing validator keys
2024-06-19 13:38:56 -- [INFO] The keys import container is starting
2024-06-19 13:38:57 -- [FATA] validator import container exited with non-zero exit code: validator-import service container with id 67c460dfabc79a3402901c19e6dbfb08572829c58bf3156ad84e20744699df8a ends with status code 255. Here are the logs for more details:
Found validator key in /keystore/validator_keys/keystore-m_12381_3600_0_0_0.json
Importing validator...
qemu-x86_64: Could not open '/lib64/ld-linux-x86-64.so.2': No such file or directory

The same steps run successful for Teku, Lodestar and Prysm.
Do i need to preform some additional steps for Lighthouse?

internal/pkg/generate/types.go Outdated Show resolved Hide resolved
configs/client_images.yaml Outdated Show resolved Hide resolved
Copy link
Member

@AntiD2ta AntiD2ta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you so much for this feature, @sugh01, and for your patience throughout the entire process. We are very close to merging this PR. Please address the few open comments, and we will be good to go.

scripts/charon/import_lodestar_keys.sh Outdated Show resolved Hide resolved
internal/pkg/generate/generate_scripts.go Outdated Show resolved Hide resolved
Copy link
Contributor

@khalifaa55 khalifaa55 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created Issue #475 to add E2E tests for this integration later on.

@stdevMac
Copy link
Member

stdevMac commented Nov 7, 2024

We found one error while running:

make compile
sudo cp build/sedge /usr/local/bin/
sedge generate full-node --validator=lodestar --network=holesky --distributed
sedge import-key --from keystore --distributed -n holesky lodestar

and this is the error message
on Windows:

2024-11-07 00:27:19 -- [INFO] Copying the keys from D:\Desktop\nethermind\projects\sedge\sedge\keystore
2024-11-07 00:27:19 -- [INFO] Copying Charon contents to the default path D:\Desktop\nethermind\projects\sedge\sedge\sedge-data\.charon charon                                                                                                                             fied.
2024-11-07 00:27:19 -- [FATA] CreateFile D:\Desktop\nethermind\projects\sedge\sedge\keystore: The system cannot find the file specicified.

on WSL:

2024-11-07 00:30:58 -- [INFO] Copying the keys from /mnt/d/Desktop/nethermind/projects/sedge/sedge/keystore
2024-11-07 00:30:58 -- [INFO] Copying Charon contents to the default path /mnt/d/Desktop/nethermind/projects/sedge/sedge/sedge-data/.charon
2024-11-07 00:30:58 -- [FATA] lstat /mnt/d/Desktop/nethermind/projects/sedge/sedge/keystore: no such file or directory

CHANGELOG.md Outdated Show resolved Hide resolved
Copy link
Member

@AntiD2ta AntiD2ta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved!

Co-authored-by: Miguel Tenorio <[email protected]>
@stdevMac stdevMac merged commit d04b0b4 into NethermindEth:develop Nov 14, 2024
9 of 11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants