DA compression for fuel-tx types

CHANGELOG.md

@@ -6,6 +6,10 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
 
 ## [Unreleased]
 
+### Added
+
+- [#670](https://github.com/FuelLabs/fuel-vm/pull/670): Add DA compression functionality to `Transaction` and any types within
+
 ## [Version 0.45.0]
 
 ### Changed

Cargo.toml

@@ -1,6 +1,7 @@
 [workspace]
 members = [
     "fuel-asm",
+    "fuel-compression",
     "fuel-crypto",
     "fuel-merkle",
     "fuel-storage",
@@ -21,7 +22,9 @@ repository = "https://github.com/FuelLabs/fuel-vm"
 version = "0.45.0"
 
 [workspace.dependencies]
+anyhow = "1.0"
 fuel-asm = { version = "0.45.0", path = "fuel-asm", default-features = false }
+fuel-compression = { version = "0.45.0", path = "fuel-compression", default-features = false }
 fuel-crypto = { version = "0.45.0", path = "fuel-crypto", default-features = false }
 fuel-derive = { version = "0.45.0", path = "fuel-derive", default-features = false }
 fuel-merkle = { version = "0.45.0", path = "fuel-merkle", default-features = false }

README.md

@@ -13,6 +13,7 @@ and [the Sway compiler](https://github.com/FuelLabs/sway/).
 | Crate        | Version                                                                                             | Description                                                                                                                                                                                                                        | 
 |--------------|-----------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| 
 | fuel-asm     | [![crates.io](https://img.shields.io/crates/v/fuel-asm)](https://crates.io/crates/fuel-asm)         | Contains the FuelVM instruction set - opcodes used by the Sway and VM.                                                                                                                                                             | 
+| fuel-compression | [![crates.io](https://img.shields.io/crates/v/fuel-compression)](https://crates.io/crates/fuel-compression)   | DA-layer compression of Fuel transaction types | 
 | fuel-crypto  | [![crates.io](https://img.shields.io/crates/v/fuel-crypto)](https://crates.io/crates/fuel-crypto)   | Cryptographic primitives used across Fuel Rust based projects.                                                                                                                                                                     | 
 | fuel-merkle  | [![crates.io](https://img.shields.io/crates/v/fuel-merkle)](https://crates.io/crates/fuel-merkle)   | Implementations of the Merkle Tree used by the `fuel-core` to fulfill fraud proofs requirements, and `fuel-tx` to validate transaction validity.                                                                                   | 
 | fuel-storage | [![crates.io](https://img.shields.io/crates/v/fuel-storage)](https://crates.io/crates/fuel-storage) | Storage abstraction is used to connect FuelVM, `fuel-merkle`, and `fuel-core` together without direct access.                                                                                                                      | 

fuel-compression/Cargo.toml

@@ -0,0 +1,28 @@
+[package]
+name = "fuel-compression"
+version = { workspace = true }
+authors = { workspace = true }
+categories = ["cryptography::cryptocurrencies"]
+edition = { workspace = true }
+homepage = { workspace = true }
+keywords = ["blockchain", "cryptocurrencies", "fuel-compression"]
+license = { workspace = true }
+repository = { workspace = true }
+description = "Compression and decompression of Fuel blocks for DA storage."
+
+[dependencies]
+anyhow = { workspace = true }
+fuel-derive = { workspace = true }
+postcard = { version = "1.0", features = ["use-std"] }
+serde = { version = "1.0", features = ["derive"] }
+serde-big-array = "0.5"
+
+[dev-dependencies]
+bincode = "1.3"
+fuel-asm = { workspace = true }
+fuel-compression = { path = "." } # Self-dependency needed by test for macros
+fuel-tx = { workspace = true }
+fuel-types = { workspace = true, features = ["serde"] }
+
+[features]
+test-helpers = []

fuel-compression/README.md

@@ -0,0 +1,34 @@
+# Compression and decompression of fuel-types for the DA layer
+
+## Compressed block header
+
+Each compressed block begins with a single-byte version field, so that it's possible to change the format later.
+
+## Temporal registry
+
+This crate provides offchain registries for different types such as `AssetId`, `ContractId`, scripts, and predicates. Each registry is a key-value store with three-byte key. The registires are essentially compression caches. The three byte key allows cache size of 16 million values before reregistering the older values.
+
+The registries allow replacing repeated objects with their respective keys, so if an object
+is used multiple times in a short interval (couple of months, maybe), then the full value
+exists on only a single uncompressed block,
+
+### Fraud proofs
+
+Compressed block will start with 32 bytes of merkle root over all compression smts, followed by newly registered values along with their keys. Using an SMT provides flexibility around the algorithm we use to define keys without knowing how exactly values were chosen to be registered.
+
+Each registry also uses an SMT. Since the keys are three bytes long, the depth of the SMT is capped at 24 levels. 
+
+
+    - More efficient for fraud proofs instead of needing to provide entire previous blocks with proofs
+
+## Compression of `UtxoIds`
+
+Since each `UtxoId` only appears once, there's no point in registering them. Instead, they are replaced with `TxPointer`s (7 bytes worst case), which are still unique.
+
+### Fraud proofs
+
+During fraud proofs we need to use the `prev_root` to prove that the referenced block height is part of the chain.
+
+## Other techniques
+
+- These techniques should be good enough for now, but there are lots of other interesting ideas for this.