Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tpu quic proxy as module and lite-rpc switch #164

Merged
merged 132 commits into from
Aug 11, 2023
Merged

tpu quic proxy as module and lite-rpc switch #164

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
132 commits
Select commit Hold shift + click to select a range
56743e7
integrate proxy with lite-rpc for testing
grooviegermanikus Jun 22, 2023
23b7d75
patch connection utils
grooviegermanikus Jun 22, 2023
87219da
Cargo.lock
grooviegermanikus Jun 23, 2023
90e0468
add sample client
grooviegermanikus Jun 23, 2023
4e25095
define basic proxy wire format
grooviegermanikus Jun 23, 2023
b812f81
Merge remote-tracking branch 'origin/keypair_file_location_macos_fix'…
grooviegermanikus Jun 23, 2023
47fd8b1
rountrip test with instruction
grooviegermanikus Jun 25, 2023
eb33095
introduce proxy request
grooviegermanikus Jun 26, 2023
f7ad7da
use quic uni instead of bi stream
grooviegermanikus Jun 26, 2023
1146589
proxy works
grooviegermanikus Jun 26, 2023
77c801f
proxy forwarding works
grooviegermanikus Jun 26, 2023
a786e19
add readme
grooviegermanikus Jun 27, 2023
eb25711
comment tpu_service input
grooviegermanikus Jun 27, 2023
c0d4b18
improve error handling on accept stream
grooviegermanikus Jun 27, 2023
c406f44
remove useless async
grooviegermanikus Jun 27, 2023
f5979eb
WIP: refactor active type connection
grooviegermanikus Jun 27, 2023
7c3f7be
refactor out tpu_quic_connection
grooviegermanikus Jun 29, 2023
228c58d
cut dependency between quic-proxy and lite-rpc crate
grooviegermanikus Jul 20, 2023
0490e55
merge integration test branch into proxy branch
grooviegermanikus Jul 20, 2023
9e06400
extract integration test
grooviegermanikus Jul 20, 2023
04147bf
add tpu connection path
grooviegermanikus Jul 20, 2023
9c1ce6b
wire up quic proxy connection path
grooviegermanikus Jul 20, 2023
c4093b5
test works - send tx via test quic proxy
grooviegermanikus Jul 20, 2023
4c603e4
use quic proxy in integration test
grooviegermanikus Jul 20, 2023
5d89c63
cleanup imports
grooviegermanikus Jul 21, 2023
a2a52be
integration test now supports proxy and non-proxy mode
grooviegermanikus Jul 21, 2023
dd03d33
remove misleading log
grooviegermanikus Jul 21, 2023
4c7f01d
fix env format
grooviegermanikus Jul 21, 2023
21bef36
port refactored integration test to proxy branch
grooviegermanikus Jul 21, 2023
9ab08d2
fix compile issues
grooviegermanikus Jul 21, 2023
0d1c95c
move proxy send code
grooviegermanikus Jul 24, 2023
067c1a1
experiment with tokio tracing
grooviegermanikus Jul 24, 2023
19f7922
build dumb lazy-one-connection per tpu allocation strategy
grooviegermanikus Jul 24, 2023
fd2cf5a
fix dashmap arc issue
grooviegermanikus Jul 24, 2023
a2207da
wip works
grooviegermanikus Jul 25, 2023
c9c6b7b
use stream per client-proxy connection
grooviegermanikus Jul 25, 2023
391a42e
wip
grooviegermanikus Jul 25, 2023
5a4546c
implement chunk+parallel sending
grooviegermanikus Jul 25, 2023
4f09278
works
grooviegermanikus Jul 25, 2023
7b33ec7
set quic client+server transport config
grooviegermanikus Jul 26, 2023
400048c
integrate send_transaction_batch
grooviegermanikus Jul 26, 2023
0102058
connection stats
grooviegermanikus Jul 26, 2023
f2dbdd8
add cli option
grooviegermanikus Jul 26, 2023
3bebfa2
replace DashMap
grooviegermanikus Jul 26, 2023
d378209
restore DashMap dependency
grooviegermanikus Jul 27, 2023
23ffbc5
refactor
grooviegermanikus Jul 27, 2023
d2d28bb
add tx forwarder
grooviegermanikus Jul 27, 2023
7abf135
add quinn auto reconnect
grooviegermanikus Jul 28, 2023
8ea7428
inline proxy outbound quinn handling
grooviegermanikus Jul 28, 2023
7cae76b
drain queues
grooviegermanikus Jul 28, 2023
8cacfe6
add exit signal
grooviegermanikus Jul 28, 2023
d5df190
add auto reconnect to proxy client
grooviegermanikus Jul 28, 2023
e8f2bef
reduce logging; add parallel streams on client side
grooviegermanikus Jul 28, 2023
385d94b
use fanout
grooviegermanikus Jul 28, 2023
816f978
make validator id optional from cli
grooviegermanikus Jul 28, 2023
c13f0f5
add validator identity
grooviegermanikus Jul 28, 2023
88b6935
fix wrong agent-tpu assignment
grooviegermanikus Jul 28, 2023
1d4e626
make proxy address configurable
grooviegermanikus Jul 28, 2023
87b8c07
cargo.lock
grooviegermanikus Jul 29, 2023
94686f3
fix compile ValidatorIdentity
grooviegermanikus Jul 29, 2023
a27b146
fix compile ValidatorIdentity
grooviegermanikus Jul 29, 2023
20c2710
cargo fix
grooviegermanikus Jul 29, 2023
aea5657
works: split listener+forwarder
grooviegermanikus Jul 31, 2023
0e19b85
clenaup proxy.rs
grooviegermanikus Jul 31, 2023
0b718bc
cleanup mod dependencies
grooviegermanikus Jul 31, 2023
40bdae5
cleanup unused quic code
grooviegermanikus Jul 31, 2023
ff803fd
minor rename
grooviegermanikus Jul 31, 2023
9bfc9ac
split tls client/server/self signed
grooviegermanikus Jul 31, 2023
949646c
quic util cleanup
grooviegermanikus Jul 31, 2023
86f2572
minor cleanup
grooviegermanikus Jul 31, 2023
62a2f76
add exit signal
grooviegermanikus Jul 31, 2023
4844844
add error context
grooviegermanikus Jul 31, 2023
035ce6e
fmt+clippy
grooviegermanikus Jul 31, 2023
3551c47
fix typo
grooviegermanikus Jul 31, 2023
a76c02d
clean service integration
grooviegermanikus Jul 31, 2023
d03f73d
align services/
grooviegermanikus Jul 31, 2023
0ad0563
Merge remote-tracking branch 'origin/main' into groovie/tpu-proxy-as-…
grooviegermanikus Aug 2, 2023
89ad1bc
comment on selfsigned cert logic
grooviegermanikus Aug 2, 2023
2f97cb8
cleanup
grooviegermanikus Aug 2, 2023
b41d05e
cleanup minor todos
grooviegermanikus Aug 2, 2023
92a977f
clippy
grooviegermanikus Aug 2, 2023
12fb602
fix all clippy errors
grooviegermanikus Aug 2, 2023
92e0972
cargo fmt
grooviegermanikus Aug 2, 2023
f00d174
simplified validator identity
grooviegermanikus Aug 2, 2023
a8863cd
code format lite-rpc
grooviegermanikus Aug 2, 2023
e1be1d0
fmt
grooviegermanikus Aug 2, 2023
ca7927d
fmt again
grooviegermanikus Aug 2, 2023
bd3fc48
fmt+clippy
grooviegermanikus Aug 2, 2023
9cb24e6
log forward channel buffering
grooviegermanikus Aug 3, 2023
d7e1539
fix sender getting dropped in .listen
grooviegermanikus Aug 3, 2023
2651469
remove DS_Store
grooviegermanikus Aug 7, 2023
bcd8d29
fix fanout to n agents using broadcast channel
grooviegermanikus Aug 7, 2023
6938a0a
clippy-fmt
grooviegermanikus Aug 7, 2023
75c2b31
Merge remote-tracking branch 'origin/groovie/tpu-proxy-as-module-with…
grooviegermanikus Aug 7, 2023
485bb39
add sharding to spread work across connections
grooviegermanikus Aug 8, 2023
3bbe99f
make connection failures more explicit
grooviegermanikus Aug 8, 2023
dd5ad1b
make connection failures more explicit
grooviegermanikus Aug 8, 2023
becff04
Merge remote-tracking branch 'origin/groovie/tpu-proxy-as-module-with…
grooviegermanikus Aug 8, 2023
3902dcc
improve loggin
grooviegermanikus Aug 8, 2023
905cb2f
docs on autoconnect
grooviegermanikus Aug 8, 2023
9e05d2b
refactored out configure_panic_hook
grooviegermanikus Aug 8, 2023
6a1d4c6
comment on configure_panic_hook
grooviegermanikus Aug 8, 2023
946536f
use TryFrom+TryInto for (de-)serialization if proxy_request_format
grooviegermanikus Aug 8, 2023
4158359
return transactions as slice
grooviegermanikus Aug 8, 2023
b19a7a5
Revert "use TryFrom+TryInto for (de-)serialization if proxy_request_f…
grooviegermanikus Aug 8, 2023
4762901
rename request wire format
grooviegermanikus Aug 8, 2023
8a1b74b
wrap forward packet
grooviegermanikus Aug 8, 2023
3667c61
avoid de-serialization of wire format
grooviegermanikus Aug 8, 2023
d776ed9
auto-reconnect: more context in logs
grooviegermanikus Aug 8, 2023
47c95d5
agent cleanup
grooviegermanikus Aug 9, 2023
4ca4e03
add shutdown for agent
grooviegermanikus Aug 9, 2023
5f0d6ae
fmt
grooviegermanikus Aug 9, 2023
9a6542c
tpu auto connection with infinite retries
grooviegermanikus Aug 9, 2023
677801f
pack list of target tpu nodes in the wire format
grooviegermanikus Aug 9, 2023
1e055fe
clippy+fmt
grooviegermanikus Aug 9, 2023
a27cc06
fix broken log output
grooviegermanikus Aug 9, 2023
414216e
handle 0 tpu case
grooviegermanikus Aug 9, 2023
c7a79ab
fmt
grooviegermanikus Aug 9, 2023
f4deb51
adjust broadcast channel capacity to pow2
grooviegermanikus Aug 9, 2023
b080c23
inrease test timeouts
grooviegermanikus Aug 9, 2023
e0f3538
move broadcast receiver
grooviegermanikus Aug 10, 2023
e089693
remove useless serialization to VersionedTransaction
grooviegermanikus Aug 10, 2023
5a07037
cleanups
grooviegermanikus Aug 10, 2023
e246aa9
check for missing tx in test
grooviegermanikus Aug 10, 2023
7df7f27
properly drain channel on agent shutdown
grooviegermanikus Aug 10, 2023
bd107c2
add smarter agent shutdown
grooviegermanikus Aug 10, 2023
f2aa706
refactor timeout
grooviegermanikus Aug 10, 2023
10e4ea9
clarify test naming
grooviegermanikus Aug 10, 2023
4982325
direct test - wait in loop
grooviegermanikus Aug 10, 2023
0aebd7b
MAN-59 disable flaky tests
grooviegermanikus Aug 11, 2023
75a1777
tweaked log levels on agent shutdown
grooviegermanikus Aug 11, 2023
3e4b7a0
fix compile
grooviegermanikus Aug 11, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ jobs:
sudo apt-get install libssl-dev openssl -y

- name: Early Build
run: cargo build --workspace
run: cargo build --workspace --tests

- name: Check Rust Code Format
run: cargo fmt -- --check
Expand All @@ -33,7 +33,7 @@ jobs:
run: cargo clippy --all-targets -- -D warnings

- name: Run Tests
run: cargo test
run: RUST_LOG=info cargo test

- name: Install node deps
run: yarn
Expand Down
Loading