Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: wasm support for browser #204

Open
wants to merge 76 commits into
base: develop
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
9bb7606
feat: wasm support with btreemap
driftluo Nov 12, 2024
be9f344
feat: wasm supports data persistence through indexdb
driftluo Nov 13, 2024
411deac
chore: impl new commit
driftluo Nov 22, 2024
3c9863e
Implement db-worker and db-common
Officeyutong Nov 18, 2024
b9b1a9e
misc: remove dependency num-traits and num-derive
Officeyutong Nov 25, 2024
968f78a
update: remove tmpdb
Officeyutong Nov 25, 2024
1731d23
replace serde_json with bincode
Officeyutong Nov 26, 2024
6604bb0
remove some unused debug lines
Officeyutong Nov 26, 2024
8202fa2
tidy the code of db-common and db-worker
Officeyutong Nov 27, 2024
de7605e
remove take while benchmark
Officeyutong Nov 27, 2024
37673e7
Make db worker and lightclient-wasm npm-friendly
Officeyutong Nov 27, 2024
8e2309e
remove worker-loader
Officeyutong Nov 27, 2024
1848243
make packages npm-ready
Officeyutong Nov 28, 2024
fbdcb64
remove examples
Officeyutong Nov 29, 2024
198d5a8
Update basic lightclient js wrapper
Officeyutong Nov 29, 2024
1f59662
Update lightclient.js
Officeyutong Nov 30, 2024
6d4b74f
almost done
Officeyutong Dec 1, 2024
6b18315
fix @ckb-ccc/core imports
Officeyutong Dec 2, 2024
56d3604
fix reference local spec file
Officeyutong Dec 2, 2024
fe09648
fix types related bug
Officeyutong Dec 3, 2024
9e4169c
fix type conversion
Officeyutong Dec 3, 2024
93967ba
fix get_transactions types
Officeyutong Dec 3, 2024
2b09a41
fix in bug
Officeyutong Dec 3, 2024
8a56c8f
Add unit test for light-client-js
Officeyutong Dec 3, 2024
8c9b4e6
Add lock for command invocation
Officeyutong Dec 3, 2024
c1f19de
Allow customizing config with mainnet and testnet
Officeyutong Dec 4, 2024
2a37471
replace std::sync::RwLock to tokio::sync::RwLock
Officeyutong Dec 4, 2024
15200f3
Make rust clippy happy
Officeyutong Dec 5, 2024
99b40ed
fix native build
Officeyutong Dec 5, 2024
6c74758
make some comments more reasonable
Officeyutong Dec 5, 2024
83b2bb8
add some basic tests
Officeyutong Dec 8, 2024
057de27
Add more unit tests
Officeyutong Dec 10, 2024
708b6c2
fix bug of estimateCycles
Officeyutong Dec 11, 2024
deb3ce7
fix bug for filter scripts
Officeyutong Dec 12, 2024
9b69113
fix fetch_header and fetch_transaction
Officeyutong Dec 12, 2024
7f7b7da
fix exception handle bug
Officeyutong Dec 12, 2024
579185d
Fix cursor bug of get_transaction and get_cells
Officeyutong Dec 12, 2024
d699a39
fix get_header response bug
Officeyutong Dec 13, 2024
a3ce904
fix bug of get cells
Officeyutong Dec 13, 2024
706fac3
update quanta
Officeyutong Dec 17, 2024
5d4654b
fix data isolation
Officeyutong Dec 19, 2024
a78872d
add workspace
Officeyutong Dec 24, 2024
fef82ca
add trace log callback
Officeyutong Dec 25, 2024
4cd3527
remove predefined secret key
Officeyutong Dec 25, 2024
601e575
update webpack config
Officeyutong Dec 26, 2024
fdf1a3a
make light-client-wasm compatible with webpack
Officeyutong Dec 26, 2024
5771fc3
Use UMD as bundle result
Officeyutong Dec 29, 2024
49c644a
use new ckb commit
Officeyutong Dec 30, 2024
0706e68
update types export
Officeyutong Dec 30, 2024
1097f58
update
Officeyutong Dec 30, 2024
84f6f49
fix cursor keyrange bug
Officeyutong Dec 30, 2024
20fbe9a
fix get_transactions with groupByTransaction
Officeyutong Dec 31, 2024
275e817
update tentacle
Officeyutong Dec 31, 2024
c7cdec9
add wildcard export
Officeyutong Jan 1, 2025
0d7a0fb
update export
Officeyutong Jan 1, 2025
8fa8f25
update remotenode def
Officeyutong Jan 1, 2025
5d896c6
fix get_transactions iterator bug
Officeyutong Jan 2, 2025
b4fcd3f
use new tentacle
Officeyutong Jan 3, 2025
16e5786
use esbuild to build light-client
Officeyutong Jan 6, 2025
e06168f
allow emitting .d.ts
Officeyutong Jan 6, 2025
7369ac5
fix worker target
Officeyutong Jan 6, 2025
fb2e6ef
restructure project
Officeyutong Jan 6, 2025
c8ce27f
update
Officeyutong Jan 6, 2025
4e6c15f
update native adaption
Officeyutong Jan 7, 2025
76e1117
update README
Officeyutong Jan 8, 2025
be23711
update
Officeyutong Jan 8, 2025
f2612dc
remove test files
Officeyutong Jan 9, 2025
46b6ecd
tidy files
Officeyutong Jan 9, 2025
2081581
remove test files
Officeyutong Jan 9, 2025
b204971
revert changes of default config
Officeyutong Jan 9, 2025
c7cf1df
revert some changes
Officeyutong Jan 9, 2025
e38d995
revert cargo.toml changes
Officeyutong Jan 9, 2025
45b6abb
update
Officeyutong Jan 9, 2025
7a00924
remove useless debug logs
Officeyutong Jan 9, 2025
118d953
update
Officeyutong Jan 9, 2025
aab3c9d
update rust version in CI
Officeyutong Jan 10, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
feat: wasm supports data persistence through indexdb
driftluo committed Nov 13, 2024
commit be9f3442cc2733ff5559f09b26b0e14fdad5512b
233 changes: 105 additions & 128 deletions Cargo.lock

Large diffs are not rendered by default.

18 changes: 9 additions & 9 deletions light-client-bin/Cargo.toml
Original file line number Diff line number Diff line change
@@ -9,15 +9,15 @@ homepage = "https://github.com/nervosnetwork/ckb-light-client"
repository = "https://github.com/nervosnetwork/ckb-light-client"

[dependencies]
ckb-async-runtime = { git = "https://github.com/driftluo/ckb.git", rev = "879ab46" }
ckb-chain-spec = { git = "https://github.com/driftluo/ckb.git", rev = "879ab46" }
ckb-network = { git = "https://github.com/driftluo/ckb.git", rev = "879ab46" }
ckb-stop-handler = { git = "https://github.com/driftluo/ckb.git", rev = "879ab46" }
ckb-resource = { git = "https://github.com/driftluo/ckb.git", rev = "879ab46" }
ckb-jsonrpc-types = { git = "https://github.com/driftluo/ckb.git", rev = "879ab46" }
ckb-types = { git = "https://github.com/driftluo/ckb.git", rev = "879ab46" }
ckb-traits = { git = "https://github.com/driftluo/ckb.git", rev = "879ab46" }
ckb-systemtime = { git = "https://github.com/driftluo/ckb.git", rev = "879ab46" }
ckb-async-runtime = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e" }
ckb-chain-spec = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e" }
ckb-network = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e" }
ckb-stop-handler = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e" }
ckb-resource = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e" }
ckb-jsonrpc-types = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e" }
ckb-types = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e" }
ckb-traits = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e" }
ckb-systemtime = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e" }

ckb-light-client-lib = { path = "../light-client-lib" }
clap = { version = "4", features = ["cargo"] }
40 changes: 21 additions & 19 deletions light-client-lib/Cargo.toml
Original file line number Diff line number Diff line change
@@ -9,19 +9,19 @@ homepage = "https://github.com/nervosnetwork/ckb-light-client"
repository = "https://github.com/nervosnetwork/ckb-light-client"

[dependencies]
ckb-app-config = { git = "https://github.com/driftluo/ckb.git", rev = "879ab46" }
ckb-constant = { git = "https://github.com/driftluo/ckb.git", rev = "879ab46" }
ckb-types = { git = "https://github.com/driftluo/ckb.git", rev = "879ab46" }
ckb-network = { git = "https://github.com/driftluo/ckb.git", rev = "879ab46" }
ckb-jsonrpc-types = { git = "https://github.com/driftluo/ckb.git", rev = "879ab46" }
ckb-error = { git = "https://github.com/driftluo/ckb.git", rev = "879ab46" }
ckb-script = { git = "https://github.com/driftluo/ckb.git", rev = "879ab46" }
ckb-chain-spec = { git = "https://github.com/driftluo/ckb.git", rev = "879ab46" }
ckb-traits = { git = "https://github.com/driftluo/ckb.git", rev = "879ab46" }
ckb-resource = { git = "https://github.com/driftluo/ckb.git", rev = "879ab46" }
ckb-verification = { git = "https://github.com/driftluo/ckb.git", rev = "879ab46" }
ckb-hash = { git = "https://github.com/driftluo/ckb.git", rev = "879ab46" }
ckb-systemtime = { git = "https://github.com/driftluo/ckb.git", rev = "879ab46" }
ckb-app-config = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e" }
ckb-constant = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e" }
ckb-types = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e" }
ckb-network = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e" }
ckb-jsonrpc-types = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e" }
ckb-error = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e" }
ckb-script = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e" }
ckb-chain-spec = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e" }
ckb-traits = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e" }
ckb-resource = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e" }
ckb-verification = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e" }
ckb-hash = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e" }
ckb-systemtime = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e" }

log = "0.4.14"
path-clean = "0.1.0"
@@ -41,6 +41,9 @@ numext-fixed-uint = { version = "0.1", features = [
anyhow = "1.0.56"
thiserror = "1.0.30"
toml = "0.5.8"
tokio = { version = "1.20" }
idb = "0.6"
serde-wasm-bindgen = "0.6.5"

[target.'cfg(not(target_arch = "wasm32"))'.dependencies]
rocksdb = { package = "ckb-rocksdb", version = "=0.21.1", features = [
@@ -54,16 +57,15 @@ idb = "0.6"
serde-wasm-bindgen = "0.6.5"

[dev-dependencies]
ckb-shared = { git = "https://github.com/driftluo/ckb.git", rev = "879ab46" }
ckb-chain = { git = "https://github.com/driftluo/ckb.git", rev = "879ab46" }
ckb-tx-pool = { git = "https://github.com/driftluo/ckb.git", rev = "879ab46" }
ckb-store = { git = "https://github.com/driftluo/ckb.git", rev = "879ab46" }
ckb-systemtime = { git = "https://github.com/driftluo/ckb.git", rev = "879ab46", features = [
ckb-shared = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e" }
ckb-chain = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e" }
ckb-tx-pool = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e" }
ckb-store = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e" }
ckb-systemtime = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e", features = [
"enable_faketime",
] }
tempfile = "3.0"
rand = "0.6"
serde_json = "1.0"
tokio = { version = "1.20" }
lazy_static = "1.3.0"
env_logger = "0.11"
10 changes: 8 additions & 2 deletions light-client-lib/src/protocols/relayer.rs
Original file line number Diff line number Diff line change
@@ -125,8 +125,14 @@ impl CKBProtocolHandler for RelayProtocol {

let epoch = match prove_state_epoch {
Some(proved) => {
let stored: EpochNumberWithFraction =
self.storage.get_last_state().1.raw().epoch().unpack();
let stored: EpochNumberWithFraction = self
.storage
.get_last_state_async()
.await
.1
.raw()
.epoch()
.unpack();
if stored > proved {
trace!("RelayProtocol.connected peer={} got a stale epoch, ignore and close the protocol", peer);
close_protocol(&nc, peer);
1,683 changes: 1,683 additions & 0 deletions light-client-lib/src/storage/db/browser.rs

Large diffs are not rendered by default.

1,296 changes: 0 additions & 1,296 deletions light-client-lib/src/storage/db/browser_test.rs

This file was deleted.

8 changes: 2 additions & 6 deletions light-client-lib/src/storage/db/mod.rs
Original file line number Diff line number Diff line change
@@ -4,10 +4,6 @@ mod native;
pub use native::{Batch, Storage};

#[cfg(target_arch = "wasm32")]
mod browser_test;
mod browser;
#[cfg(target_arch = "wasm32")]
pub use browser_test::{Batch, Direction, Storage};
// #[cfg(target_arch = "wasm32")]
// mod browser;
// #[cfg(target_arch = "wasm32")]
// pub use browser::{collect_iterator, Batch, CursorDirection, Storage};
pub use browser::{Batch, CursorDirection, Storage};
11 changes: 9 additions & 2 deletions light-client-lib/src/storage/mod.rs
Original file line number Diff line number Diff line change
@@ -4,10 +4,12 @@ use std::{collections::HashMap, sync::Arc};
use ckb_traits::{
CellDataProvider, ExtensionProvider, HeaderFields, HeaderFieldsProvider, HeaderProvider,
};
#[cfg(not(target_arch = "wasm32"))]
use ckb_types::core::cell::CellMeta;
use ckb_types::{
bytes::Bytes,
core::{
cell::{CellMeta, CellProvider, CellStatus},
cell::{CellProvider, CellStatus},
BlockNumber, BlockView, HeaderView,
},
packed::{self, Byte32, CellOutput, Header, OutPoint, Script, Transaction},
@@ -19,7 +21,7 @@ use ckb_types::{
mod db;

#[cfg(target_arch = "wasm32")]
pub use db::{Batch, Direction, Storage};
pub use db::{Batch, CursorDirection, Storage};

#[cfg(not(target_arch = "wasm32"))]
pub use db::{Batch, Storage};
@@ -121,6 +123,10 @@ impl StorageWithChainData {
&self.storage
}

pub fn peers(&self) -> &Arc<Peers> {
&self.peers
}

pub fn pending_txs(&self) -> &RwLock<PendingTxs> {
&self.pending_txs
}
@@ -144,6 +150,7 @@ impl StorageWithChainData {
}
}

#[derive(Clone)]
pub struct TmpDB {
pub(crate) cells: HashMap<OutPoint, CellStatus>,
pub(crate) headers: HashMap<Byte32, HeaderView>,
2 changes: 1 addition & 1 deletion light-client-lib/src/verify.rs
Original file line number Diff line number Diff line change
@@ -180,7 +180,7 @@ where
}

#[cfg(target_arch = "wasm32")]
async fn generate_temporary_db(
pub async fn generate_temporary_db(
swc: &StorageWithChainData,
transaction: TransactionView,
) -> Result<TmpDB, OutPointError> {
21 changes: 9 additions & 12 deletions light-client-wasm/Cargo.toml
Original file line number Diff line number Diff line change
@@ -21,18 +21,15 @@ wasm-bindgen-futures = "0.4"
log = "0.4"
wasm-logger = "0.2"
serde-wasm-bindgen = "0.6.5"
idb = "0.6"
indexed-db = "0.4"
futures = "0.3"
anyhow = "1"

ckb-chain-spec = { git = "https://github.com/driftluo/ckb.git", rev = "879ab46" }
ckb-resource = { git = "https://github.com/driftluo/ckb.git", rev = "879ab46" }
ckb-network = { git = "https://github.com/driftluo/ckb.git", rev = "879ab46" }
ckb-async-runtime = { git = "https://github.com/driftluo/ckb.git", rev = "879ab46" }
ckb-types = { git = "https://github.com/driftluo/ckb.git", rev = "879ab46" }
ckb-jsonrpc-types = { git = "https://github.com/driftluo/ckb.git", rev = "879ab46" }
ckb-stop-handler = { git = "https://github.com/driftluo/ckb.git", rev = "879ab46" }

ckb-chain-spec = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e" }
ckb-resource = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e" }
ckb-network = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e" }
ckb-async-runtime = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e" }
ckb-types = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e" }
ckb-jsonrpc-types = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e" }
ckb-stop-handler = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e" }
ckb-systemtime = { git = "https://github.com/driftluo/ckb.git", rev = "e75496e" }

getrandom = { version = "0.2", features = ["js"] }

15 changes: 0 additions & 15 deletions light-client-wasm/index.js
Original file line number Diff line number Diff line change
@@ -9,23 +9,8 @@ import {
get_scripts,
Order,
stop,
test_idb,
test_serde,
test_indexed_db,
} from "./pkg";
debugger;
// await test_idb();
// await test_indexed_db();
// console.log(test_serde());
// console.log("start 13");
// ckb_vm_test("13");
// console.log("end 13");

// setTimeout(() => {
// console.log("start 12");
// ckb_vm_test("12");
// console.log("end 12");
// }, 2000);

await ligth_client("dev");

1,193 changes: 816 additions & 377 deletions light-client-wasm/src/lib.rs

Large diffs are not rendered by default.