Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

automated devent deploy + skip verify on build #94

Merged
merged 69 commits into from
Apr 6, 2023
Merged

automated devent deploy + skip verify on build #94

merged 69 commits into from
Apr 6, 2023

Conversation

gyuho
Copy link
Contributor

@gyuho gyuho commented Mar 14, 2023 •
edited by patrick-ogrady
Loading

Screenshot 2023-03-15 at 12 04 18 AM

Screenshot 2023-03-15 at 12 04 33 AM

TODO

@gyuho gyuho requested a review from patrick-ogrady as a code owner March 14, 2023 15:56
@gyuho
Copy link
Contributor Author

gyuho commented Mar 14, 2023

Screenshot 2023-03-15 at 12 05 21 AM

@gyuho
Copy link
Contributor Author

gyuho commented Mar 14, 2023

Screenshot 2023-03-15 at 12 06 28 AM

@patrick-ogrady patrick-ogrady linked an issue Mar 15, 2023 that may be closed by this pull request
[tokenvm] add cluster deployer script for devnets #84
Closed
patrick-ogrady
patrick-ogrady reviewed Mar 16, 2023
View reviewed changes
examples/tokenvm/avalanche-ops.aws.md Outdated
@@ -37,9 +37,10 @@ chmod +x /tmp/avalancheup-aws

### Step 2

Now we can spin up a new network of 6 nodes with some defaults:
Now we can spin up a new network of 6 nodes with some defaults. `avalanche-ops` also supports [Graviton-based processors](https://aws.amazon.com/ec2/graviton/) (ARM64). Use `--arch-type arm64` to run nodes in ARM64 CPUs. `avalanche-ops` also supports Spot instances for cost savings. Use `--instance-mode=spot` to run instances in spot mode.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️

Base automatically changed from warp-messaging to main March 20, 2023 00:54
@gyuho gyuho added the DO NOT MERGE This PR is not meant to be merged in its current state label Mar 20, 2023
@gyuho gyuho force-pushed the testing-warp-with-avalanche-ops branch 2 times, most recently from d482040 to 8dfc98c Compare March 20, 2023 05:32
@gyuho gyuho force-pushed the testing-warp-with-avalanche-ops branch from 05b10b7 to bf6575f Compare March 20, 2023 05:46
@gyuho
Copy link
Contributor Author

gyuho commented Mar 20, 2023 •
edited
Loading

Example commands:

# EXAMPLE: write subnet config
/home/ubuntu/avalanche-ops/target/release/avalancheup-aws subnet-config \
--log-level=info \
--proposer-min-block-delay 0 \
--file-path /tmp/subnet-config.json

cat <<EOF > /tmp/allocations.json
[{"address":"token1rvzhmceq997zntgvravfagsks6w0ryud3rylh4cdvayry0dl97nsjzf3yp", "balance":1000000000000}]
EOF
rm -f /tmp/tokenvm-genesis.json
/tmp/token-cli genesis generate /tmp/allocations.json \
--genesis-file /tmp/tokenvm-genesis.json
cat /tmp/tokenvm-genesis.json

cat <<EOF > /tmp/tokenvm-chain-config.json
{
  "mempoolSize": 10000000,
  "mempoolExemptPayers":["token1rvzhmceq997zntgvravfagsks6w0ryud3rylh4cdvayry0dl97nsjzf3yp"],
  "parallelism": 5,
  "trackedPairs":["*"],
  "logLevel": "info",
  "stateSyncServerDelay": 0
}
EOF
cat /tmp/tokenvm-chain-config.json


/home/ubuntu/avalanche-ops/target/release/avalancheup-aws install-subnet-chain \
--log-level info \
--region us-west-2 \
--s3-bucket avalanche-ops-202303-3hxfve9pfh-us-west-2 \
--s3-key-prefix aops-custom-202303-uAnbAp/install-subnet-chain \
--ssm-doc aops-custom-202303-uAnbAp-ssm-install-subnet-chain \
--chain-rpc-url http://54.71.147.118:9650 \
--key 0x56289e99c94b6912bfc12adc093c9b51124f0dc54ac7a766b2bc5ccf558d8027 \
--staking-perioid-in-days 15 \
--subnet-config-local-path /tmp/subnet-config.json \
--subnet-config-remote-dir /data/avalanche-configs/subnets \
--vm-binary-local-path /tmp/tokenvm \
--vm-binary-remote-dir /data/avalanche-plugins \
--chain-name tokenvm1 \
--chain-genesis-path /tmp/tokenvm-genesis.json \
--chain-config-local-path /tmp/tokenvm-chain-config.json \
--chain-config-remote-dir /data/avalanche-configs/chains \
--avalanchego-config-remote-path /data/avalanche-configs/config.json \
--node-ids-to-instance-ids '{"NodeID-CtgUPgUxEP74QzWCGXCXbZWe7AatMZXi":"i-0e86b8a0816e2bc4f","NodeID-PJZbK8d7jTygnMAhaWw55cypP7EzE4a49":"i-0f96c61ec4e4430e2","NodeID-GJEiqPr2gZvu5v9pUniSf9mbbTtn98tsy":"i-0fde89668ae67d894"}'
# Vm Id tHBYNu8ikt25R77fH4znHYC4B5mkaEnXPFmsJnECZjq59dySw
# created a blockchain DKmZansv3VnDVAXZSzpXwSid2aFurhYbJxkdezx4F1AR1dqiz for subnet 2vF4eZUKQMWpefJ2ympzp3t82S2dder1gKcpFHBke4K3nWmtAg

/home/ubuntu/avalanche-ops/target/release/avalancheup-aws install-subnet-chain \
--log-level info \
--region us-west-2 \
--s3-bucket avalanche-ops-202303-3hxfve9pfh-us-west-2 \
--s3-key-prefix aops-custom-202303-uAnbAp/install-subnet-chain \
--ssm-doc aops-custom-202303-uAnbAp-ssm-install-subnet-chain \
--chain-rpc-url http://54.71.147.118:9650 \
--key 0x56289e99c94b6912bfc12adc093c9b51124f0dc54ac7a766b2bc5ccf558d8027 \
--staking-perioid-in-days 15 \
--subnet-config-local-path /tmp/subnet-config.json \
--subnet-config-remote-dir /data/avalanche-configs/subnets \
--vm-binary-local-path /tmp/tokenvm \
--vm-binary-remote-dir /data/avalanche-plugins \
--chain-name tokenvm2 \
--chain-genesis-path /tmp/tokenvm-genesis.json \
--chain-config-local-path /tmp/tokenvm-chain-config.json \
--chain-config-remote-dir /data/avalanche-configs/chains \
--avalanchego-config-remote-path /data/avalanche-configs/config.json \
--node-ids-to-instance-ids '{"NodeID-E2tRmg87X57SAAvy4PYt3dsL3ShBNNhYW":"i-03a7e453bdbb80064","NodeID-E9vKaRMRE7TusJgxfP4qCmbw9m6g5cRqK":"i-0f1ed3fd7fba149fc","NodeID-NpPH3V9HrrM2Ts6ruirDK7fNLz2t9eS7d":"i-094416ec533aeffbc"}'
# tHBYNu8ikt4i8cEV4nsSuj7Ldc9sXAHc8L6qKRJR4e5CR7T3t
# SUCCESS: subnet Id 22areT4JiTwZwVPEQ4hony6uiG4XtzMfDEjoYDeNsLcmP8F8hr, blockchain Id oD9AREFxtsj7P6dw3WtRWmxsCCjQty7h5NV74y8BDQNmWGoDP

{"checks":{"C":{"message":{"consensus":{"longestRunningBlock":"0s","outstandingBlocks":0},"vm":null},"timestamp":"2023-03-20T05:57:01.709856988Z","duration":13720},"DKmZansv3VnDVAXZSzpXwSid2aFurhYbJxkdezx4F1AR1dqiz":{"message":{"consensus":{"longestRunningBlock":"0s","outstandingBlocks":0},"vm":{"database":{"v1.4.5":null},"health":200}},"timestamp":"2023-03-20T05:57:01.711074082Z","duration":1206474},"P":{"message":{"consensus":{"longestRunningBlock":"0s","outstandingBlocks":0},"vm":{"2vF4eZUKQMWpefJ2ympzp3t82S2dder1gKcpFHBke4K3nWmtAg-percentConnected":1,"primary-percentConnected":1}},"timestamp":"2023-03-20T05:57:01.709913598Z","duration":67710},"X":{"message":{"consensus":{"outstandingVertices":0,"snowstorm":{"outstandingTransactions":0}},"vm":null},"timestamp":"2023-03-20T05:57:01.709834418Z","duration":21580},"bootstrapped":{"message":[],"timestamp":"2023-03-20T05:57:01.709840818Z","duration":3020},"database":{"timestamp":"2023-03-20T05:57:01.709836928Z","duration":1420},"diskspace":{"message":{"availableDiskBytes":299770765312},"timestamp":"2023-03-20T05:57:01.709845228Z","duration":3760},"network":{"message":{"connectedPeers":5,"sendFailRate":0,"timeSinceLastMsgReceived":"709.836028ms","timeSinceLastMsgSent":"2.709836028s"},"timestamp":"2023-03-20T05:57:01.709840848Z","duration":9580},"router":{"message":{"longestRunningRequest":"0s","outstandingRequests":0},"timestamp":"2023-03-20T05:57:01.709866918Z","duration":8850}},"healthy":true}

gyuho added 2 commits March 20, 2023 16:21
@gyuho
specify --subnet-validate-period-in-days
1bc9568 
Signed-off-by: Gyuho Lee <[email protected]>
@gyuho
fix typo
35bc2d2 
Signed-off-by: Gyuho Lee <[email protected]>
@gyuho gyuho removed the DO NOT MERGE This PR is not meant to be merged in its current state label Mar 20, 2023
gyuho added 3 commits March 20, 2023 16:55
@gyuho
default
88b06f5 
Signed-off-by: Gyuho Lee <[email protected]>
@gyuho
update
d17e31d 
Signed-off-by: Gyuho Lee <[email protected]>
@gyuho
just use 16 day
b612818 
Signed-off-by: Gyuho Lee <[email protected]>
@patrick-ogrady
Copy link
Contributor

patrick-ogrady commented Apr 2, 2023 •
edited
Loading

#126

  • Fix cache size config on MerkleDB
  • Import metrics on MerkleDB
  • Update Avalanche-ops to remove put gossip
  • Don't re-process state when a block was built

@patrick-ogrady patrick-ogrady changed the title avalanche-ops.aws.md: initial commit automated devent deploy + skip verify on build Apr 3, 2023
gyuho and others added 10 commits April 3, 2023 19:08
@gyuho
DEVNETS: comment "instance-types" flags
480e474 
Signed-off-by: Gyuho Lee <[email protected]>
@patrick-ogrady
Skip verify on build (#126)
52f5a9c 
* insert height into state

* add merkledb metrics

* cleanup more metrics handling

* add additional arguments for config

* add HeightKey

* log lock wait

* use NodeID as random

* fix profiles

* gofumpt

* gofmt

* add more logs

* perform proper verify

* cache built block

* reduce memory usage of tstate

* improve memory efficiency of storage

* align devnets and scripts

* re-add proposer delay

* use default peer gossip size
@patrick-ogrady
add note about custom instance types
b93f8aa
@patrick-ogrady
revert mainnet gossip changes
5e141a9
@patrick-ogrady
more doc tweaks
46393db
@patrick-ogrady
fixing incorrect filepath
70e8cd0
@patrick-ogrady
add avalanchego profiling
f4c4c8b
@patrick-ogrady
add star history to readme
b474524
@patrick-ogrady
fix duplicate image
1caf086
@patrick-ogrady
add warning about memory size
29f552e
@patrick-ogrady
Copy link
Contributor

Processed 120M transactions at a sustained throughput of 7500 TPS. Nodes used ~2.5 cores and the consensus engine was busy 180 ms/s.

@patrick-ogrady patrick-ogrady merged commit dc7eb5e into main Apr 6, 2023
@patrick-ogrady patrick-ogrady deleted the testing-warp-with-avalanche-ops branch April 6, 2023 01:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@patrick-ogrady patrick-ogrady patrick-ogrady left review comments

Assignees
No one assigned
Labels
devnet blocker
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[tokenvm] add cluster deployer script for devnets
2 participants
@gyuho @patrick-ogrady