Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(tee): TEE Prover Gateway #2333

Merged
merged 36 commits into from
Jul 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
1527411
feat(tee): TEE Prover Gateway
pbeza Jun 26, 2024
0b730ea
Backoff and retry if no batches available
pbeza Jun 26, 2024
93b6506
Merge remote-tracking branch 'origin/main' into tee/tee_prover_binary…
pbeza Jun 26, 2024
4153d73
Fix: replace TEE_SIGNING_KEY with TEE_QUOTE_FILE
pbeza Jun 27, 2024
c7772e8
Respect TEE_TYPE env variable
pbeza Jun 27, 2024
401539b
Merge remote-tracking branch 'origin/main' into tee/tee_prover_binary…
pbeza Jun 27, 2024
b8368cd
Minor fix: use fully qualified path for `anyhow!`
pbeza Jun 27, 2024
effed5c
Simplify TeeProverTask struct name
pbeza Jun 27, 2024
16f11d5
Thin wrapper over reqwest::Client
pbeza Jun 27, 2024
c04c9e6
Pass API URL in the environment variable
pbeza Jun 27, 2024
b5faf26
Remove --batch-count param
pbeza Jun 27, 2024
fec37fd
Implement TeeProverConfig::from_env()
pbeza Jun 27, 2024
4a04d31
fixup! Respect TEE_TYPE env variable
pbeza Jun 27, 2024
ac8f820
Merge remote-tracking branch 'origin/main' into tee/tee_prover_binary…
pbeza Jun 27, 2024
3a3060a
Untangle dependencies to reuse TeeProofGenerationDataResponse
pbeza Jun 27, 2024
0bb6e09
fixup! Untangle dependencies to reuse TeeProofGenerationDataResponse
pbeza Jun 27, 2024
960e009
fixup! Respect TEE_TYPE env variable
pbeza Jun 27, 2024
9d92ecf
Split the implementation into several files
pbeza Jun 27, 2024
10f04c7
Merge remote-tracking branch 'origin/main' into tee/tee_prover_binary…
pbeza Jun 27, 2024
aa8cd4a
Replace the k256 crate with secp256k1
pbeza Jun 28, 2024
2972334
Merge remote-tracking branch 'origin/main' into tee/tee_prover_binary…
pbeza Jun 28, 2024
5a90d34
Merge remote-tracking branch 'origin/main' into tee/tee_prover_binary…
pbeza Jun 28, 2024
0b736cb
Update prover's Cargo.lock
pbeza Jun 28, 2024
00f7f8f
fixup! Update prover's Cargo.lock
pbeza Jun 28, 2024
ecd4009
Merge remote-tracking branch 'origin/main' into tee/tee_prover_binary…
pbeza Jun 28, 2024
b852c2b
Add #[derive(Debug)] to `struct`s + more docs
pbeza Jun 28, 2024
efe655a
Merge remote-tracking branch 'origin/main' into tee/tee_prover_binary…
pbeza Jun 28, 2024
8423881
Fixing simple issues from a code review
pbeza Jul 2, 2024
52713f5
Merge remote-tracking branch 'origin/main' into tee/tee_prover_binary…
pbeza Jul 2, 2024
4074e13
Addressing additional minor code review comments
pbeza Jul 2, 2024
eb53a14
Fix tests
pbeza Jul 3, 2024
f427357
fixup! Fix tests
pbeza Jul 3, 2024
5141a71
fixup! Fix tests
pbeza Jul 3, 2024
fb96103
Handle transient errors gracefully
pbeza Jul 3, 2024
ebc8826
Merge remote-tracking branch 'origin/main' into tee/tee_prover_binary…
pbeza Jul 3, 2024
fc21b8e
Merge branch 'main' into tee/tee_prover_binary_with_node_framework
haraldh Jul 3, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 36 additions & 12 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ members = [
"core/bin/verified_sources_fetcher",
"core/bin/zksync_server",
"core/bin/genesis_generator",
"core/bin/zksync_tee_prover",
# Node services
"core/node/node_framework",
"core/node/proof_data_handler",
Expand Down
29 changes: 29 additions & 0 deletions core/bin/zksync_tee_prover/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
[package]
name = "zksync_tee_prover"
version.workspace = true
edition.workspace = true
authors.workspace = true
homepage.workspace = true
repository.workspace = true
license.workspace = true
keywords.workspace = true
categories.workspace = true

[dependencies]
anyhow.workspace = true
async-trait.workspace = true
reqwest.workspace = true
secp256k1.workspace = true
serde = { workspace = true, features = ["derive"] }
thiserror.workspace = true
tokio = { workspace = true, features = ["full"] }
tracing.workspace = true
url.workspace = true
zksync_basic_types.workspace = true
zksync_config.workspace = true
zksync_env_config.workspace = true
zksync_node_framework.workspace = true
zksync_prover_interface.workspace = true
zksync_tee_verifier.workspace = true
zksync_types.workspace = true
zksync_vlog.workspace = true
111 changes: 111 additions & 0 deletions core/bin/zksync_tee_prover/src/api_client.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
use reqwest::Client;
use secp256k1::{ecdsa::Signature, PublicKey};
use serde::{de::DeserializeOwned, Serialize};
use url::Url;
use zksync_basic_types::H256;
use zksync_prover_interface::{
api::{
RegisterTeeAttestationRequest, RegisterTeeAttestationResponse, SubmitTeeProofRequest,
SubmitTeeProofResponse, TeeProofGenerationDataRequest, TeeProofGenerationDataResponse,
},
inputs::TeeVerifierInput,
outputs::L1BatchTeeProofForL1,
};
use zksync_types::{tee_types::TeeType, L1BatchNumber};

use crate::error::TeeProverError;

/// Implementation of the API client for the proof data handler, run by
/// [`zksync_proof_data_handler::run_server`].
#[derive(Debug)]
pub(crate) struct TeeApiClient {
api_base_url: Url,
http_client: Client,
}

impl TeeApiClient {
pub fn new(api_base_url: Url) -> Self {
TeeApiClient {
api_base_url,
http_client: Client::new(),
}
}

async fn post<Req, Resp, S>(&self, endpoint: S, request: Req) -> Result<Resp, reqwest::Error>
where
Req: Serialize + std::fmt::Debug,
Resp: DeserializeOwned,
S: AsRef<str>,
{
let url = self.api_base_url.join(endpoint.as_ref()).unwrap();

tracing::trace!("Sending POST request to {}: {:?}", url, request);

self.http_client
.post(url)
.json(&request)
.send()
.await?
.error_for_status()?
.json::<Resp>()
.await
}

/// Registers the attestation quote with the TEE prover interface API, effectively proving that
/// the private key associated with the given public key was used to sign the root hash within a
/// trusted execution environment.
pub async fn register_attestation(
&self,
attestation_quote_bytes: Vec<u8>,
public_key: &PublicKey,
) -> Result<(), TeeProverError> {
let request = RegisterTeeAttestationRequest {
attestation: attestation_quote_bytes,
pubkey: public_key.serialize().to_vec(),
};
self.post::<_, RegisterTeeAttestationResponse, _>("/tee/register_attestation", request)
.await?;
tracing::info!(
"Attestation quote was successfully registered for the public key {}",
public_key
);
Ok(())
}

/// Fetches the next job for the TEE prover to process, verifying and signing it if the
/// verification is successful.
pub async fn get_job(&self) -> Result<Option<Box<TeeVerifierInput>>, TeeProverError> {
let request = TeeProofGenerationDataRequest {};
pbeza marked this conversation as resolved.
Show resolved Hide resolved
let response = self
.post::<_, TeeProofGenerationDataResponse, _>("/tee/proof_inputs", request)
.await?;
Ok(response.0)
}

/// Submits the successfully verified proof to the TEE prover interface API.
pub async fn submit_proof(
&self,
batch_number: L1BatchNumber,
signature: Signature,
pubkey: &PublicKey,
root_hash: H256,
tee_type: TeeType,
) -> Result<(), TeeProverError> {
let request = SubmitTeeProofRequest(Box::new(L1BatchTeeProofForL1 {
signature: signature.serialize_compact().into(),
pubkey: pubkey.serialize().into(),
proof: root_hash.as_bytes().into(),
tee_type,
}));
self.post::<_, SubmitTeeProofResponse, _>(
format!("/tee/submit_proofs/{batch_number}").as_str(),
request,
)
.await?;
tracing::info!(
"Proof submitted successfully for batch number {}",
batch_number
);
Ok(())
}
}
39 changes: 39 additions & 0 deletions core/bin/zksync_tee_prover/src/config.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
use std::path::PathBuf;

use secp256k1::SecretKey;
use url::Url;
use zksync_env_config::FromEnv;
use zksync_types::tee_types::TeeType;

/// Configuration for the TEE prover.
#[derive(Debug)]
pub(crate) struct TeeProverConfig {
/// The private key used to sign the proofs.
pub signing_key: SecretKey,
/// The path to the file containing the TEE quote.
pub attestation_quote_file_path: PathBuf,
/// Attestation quote file.
pub tee_type: TeeType,
/// TEE proof data handler API.
pub api_url: Url,
}

impl FromEnv for TeeProverConfig {
/// Constructs the TEE Prover configuration from environment variables.
///
/// Example usage of environment variables for tests:
/// ```
/// export TEE_SIGNING_KEY="b50b38c8d396c88728fc032ece558ebda96907a0b1a9340289715eef7bf29deb"
/// export TEE_QUOTE_FILE="/tmp/test" # run `echo test > /tmp/test` beforehand
/// export TEE_TYPE="sgx"
/// export TEE_API_URL="http://127.0.0.1:3320"
/// ```
fn from_env() -> anyhow::Result<Self> {
Ok(Self {
signing_key: std::env::var("TEE_SIGNING_KEY")?.parse()?,
attestation_quote_file_path: std::env::var("TEE_QUOTE_FILE")?.parse()?,
tee_type: std::env::var("TEE_TYPE")?.parse()?,
api_url: std::env::var("TEE_API_URL")?.parse()?,
})
}
}
47 changes: 47 additions & 0 deletions core/bin/zksync_tee_prover/src/error.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
use std::{error::Error as StdError, io};

use reqwest::StatusCode;

#[derive(Debug, thiserror::Error)]
pub(crate) enum TeeProverError {
#[error(transparent)]
Request(#[from] reqwest::Error),
#[error(transparent)]
Verification(anyhow::Error),
}

impl TeeProverError {
pub fn is_transient(&self) -> bool {
match self {
Self::Request(err) => is_transient_http_error(err),
_ => false,
}
}
}

fn is_transient_http_error(err: &reqwest::Error) -> bool {
err.is_timeout()
|| err.is_connect()
// Not all request errors are logically transient, but a significant part of them are (e.g.,
// `hyper` protocol-level errors), and it's safer to consider an error transient.
|| err.is_request()
|| has_transient_io_source(err)
|| err.status() == Some(StatusCode::BAD_GATEWAY)
|| err.status() == Some(StatusCode::SERVICE_UNAVAILABLE)
}

fn has_transient_io_source(err: &(dyn StdError + 'static)) -> bool {
// We treat any I/O errors as transient. This isn't always true, but frequently occurring I/O errors
// (e.g., "connection reset by peer") *are* transient, and treating an error as transient is a safer option,
// even if it can lead to unnecessary retries.
get_source::<io::Error>(err).is_some()
}

fn get_source<'a, T: StdError + 'static>(mut err: &'a (dyn StdError + 'static)) -> Option<&'a T> {
loop {
if let Some(err) = err.downcast_ref::<T>() {
return Some(err);
}
err = err.source()?;
}
}
Loading
Loading