diff --git a/bindings/rust/bench/README.md b/bindings/rust/bench/README.md index 98926ebbbca..b541d5c0146 100644 --- a/bindings/rust/bench/README.md +++ b/bindings/rust/bench/README.md @@ -23,13 +23,13 @@ cargo bench --bench handshake --bench throughput -- --profile-time 5 rm -rf .cargo ``` -## Setup +## Setup -Setup is easy! Just have OpenSSL installed, generate Rust bindings for s2n-tls using `../generate.sh`, and generate certs using `scripts/generate-certs.sh`. +Setup is easy! Just have OpenSSL installed, generate Rust bindings for s2n-tls using `../generate.sh`, and generate certs using `scripts/generate-certs.sh`. -Dependencies are the same as with s2n-tls. Currently, this crate has only been tested on Ubuntu (both x86 and ARM), but we expect everything to work with other Unix environments. +Dependencies are the same as with s2n-tls. Currently, this crate has only been tested on Ubuntu (both x86 and ARM), but we expect everything to work with other Unix environments. -To bench with AWS-LC, Amazon's custom libcrypto implementation, first run `scripts/install-aws-lc.sh` to install AWS-LC for the bench crate. To then run the benchmarks with AWS-LC, use Cargo with either the flag `--config aws-lc-config/s2n.toml` or `--config aws-lc-config/rustls.toml` (or both). You can also append these configs to `.cargo/config.toml` to let Cargo automatically detect the settings without specifying the flags each time. +To bench with AWS-LC, Amazon's custom libcrypto implementation, first run `scripts/install-aws-lc.sh` to install AWS-LC for the bench crate. To then run the benchmarks with AWS-LC, use Cargo with either the flag `--config aws-lc-config/s2n.toml` or `--config aws-lc-config/rustls.toml` (or both). You can also append these configs to `.cargo/config.toml` to let Cargo automatically detect the settings without specifying the flags each time. ### Features @@ -37,7 +37,7 @@ Default features (`rustls` and `openssl`) can be disabled by running the benches ## Performance benchmarks -The handshake and throughput benchmarks can be run with the `cargo bench` command. Criterion will auto-generate an HTML report in `target/criterion/`. +The handshake and throughput benchmarks can be run with the `cargo bench` command. Criterion will auto-generate an HTML report in `target/criterion/`. Throughput benchmarks measure round-trip throughput with the client and server connections in the same thread for symmetry. In practice, a machine would either host only the client or only the server and use multiple threads, so throughput for a single connection could theoretically be up to ~4x higher than the values from the benchmarks (when run on the same machine). @@ -74,9 +74,24 @@ heaptrack target/release/memory (pair|client|server) (s2n-tls|rustls|openssl) To do historical benchmarks, run `scripts/bench-past.sh`. This will checkout old versions of s2n-tls back to v1.3.16 in `target/` and run benchmarks on those with the `historical-perf` feature, disabling Rustls and OpenSSL benches. +## PKI Structure +``` + ┌────root──────┐ + │ │ + │ │ + ▼ │ + branch │ + │ │ + │ │ + │ │ + ▼ ▼ + leaf client +``` +`generate-certs.sh` will generate 4 certificates for each key type, with the signing relationships that are indicated in the diagram above. This cert chain length was chosen because it matches the cert chain length used by public AWS services. + ### Caveats -The last version benched is v1.3.16, since before that, the s2n-tls Rust bindings have a different API and would thus require a different bench harness to test. +The last version benched is v1.3.16, since before that, the s2n-tls Rust bindings have a different API and would thus require a different bench harness to test. v1.3.30-1.3.37 are not benched because of depedency issues when generating the Rust bindings. However, versions before and after are benched, so the overall trend in performance can still be seen without the data from these versions. @@ -87,7 +102,7 @@ Because these benches take a longer time to generate (>30 min), we include the r Notes: - Two sets of parameters for the handshake couldn't be benched before 1.3.40, since security policies that negotiated those policies as their top choice did not exist before then. - There is no data from 1.3.30 to 1.3.37 because those versions have a dependency issue that cause the Rust bindings not to build. However, there is data before and after that period, so the performance for those versions can be inferred via interpolation. -- The improvement in throughput in 1.3.28 was most likely caused by the addition of LTO to the default Rust bindings build. +- The improvement in throughput in 1.3.28 was most likely caused by the addition of LTO to the default Rust bindings build. - Since the benches are run over a long time, noise on the machine can cause variability, and background processes can cause spikes. - The variability can be seen with throughput especially because it is calculated as the inverse of time taken. @@ -97,11 +112,11 @@ Notes: ## Implementation details -We use Rust bindings for s2n-tls and OpenSSL. All of our benchmarks are run in Rust on a single thread for consistency. +We use Rust bindings for s2n-tls and OpenSSL. All of our benchmarks are run in Rust on a single thread for consistency. ### IO -To remove external factors, we use custom IO with our benchmarks, bypassing the networking layer and having the client and server connections transfer data to each other via a local buffer. +To remove external factors, we use custom IO with our benchmarks, bypassing the networking layer and having the client and server connections transfer data to each other via a local buffer. ### Certificate generation diff --git a/bindings/rust/bench/benches/resumption.rs b/bindings/rust/bench/benches/resumption.rs index 7b0872660ff..e98887521e0 100644 --- a/bindings/rust/bench/benches/resumption.rs +++ b/bindings/rust/bench/benches/resumption.rs @@ -67,7 +67,7 @@ where /// s2n-tls?". pub fn bench_resumption(c: &mut Criterion) { // compare resumption savings across both client and server - for sig_type in [SigType::Rsa2048, SigType::Ecdsa384] { + for sig_type in [SigType::Rsa2048, SigType::Ecdsa256] { let mut bench_group = c.benchmark_group(format!("resumption-pair-{:?}", sig_type)); bench_handshake_pair::(&mut bench_group, sig_type); } diff --git a/bindings/rust/bench/certs/config/ca.cnf b/bindings/rust/bench/certs/config/ca.cnf deleted file mode 100644 index 66447a71611..00000000000 --- a/bindings/rust/bench/certs/config/ca.cnf +++ /dev/null @@ -1,13 +0,0 @@ -[ req ] -prompt = no -distinguished_name = ca_distinguished_name - -[ ca_distinguished_name ] -# country name -C = JP -# state or province -ST = Chiba -L = Chiba City -O = Tessier-Ashpool -CN = develop.localca -emailAddress = ca@develop.localca diff --git a/bindings/rust/bench/certs/config/client.cnf b/bindings/rust/bench/certs/config/client.cnf deleted file mode 100644 index 2018077e6c3..00000000000 --- a/bindings/rust/bench/certs/config/client.cnf +++ /dev/null @@ -1,24 +0,0 @@ -[req] -distinguished_name = client_distinguished_name -prompt = no - -# these fields are used to force x509 to v3 certificates -# rustls will complain otherwise -x509_extensions = v3_req -req_extensions = req_ext - -[client_distinguished_name] -countryName = US -stateOrProvinceName = Washington -localityName = Seattle -organizationName = client -commonName = client.localhost - -[v3_req] -subjectAltName = @alt_names - -[req_ext] -subjectAltName = @alt_names - -[alt_names] -DNS = localhost diff --git a/bindings/rust/bench/certs/config/server.cnf b/bindings/rust/bench/certs/config/server.cnf deleted file mode 100644 index 403932d8fcf..00000000000 --- a/bindings/rust/bench/certs/config/server.cnf +++ /dev/null @@ -1,24 +0,0 @@ -[req] -distinguished_name = server_distinguished_name -prompt = no - -# these fields are used to force x509 to v3 certificates -# rustls will complain otherwise -x509_extensions = v3_req -req_extensions = req_ext - -[server_distinguished_name] -countryName = US -stateOrProvinceName = Washington -localityName = Seattle -organizationName = server -commonName = server.localhost - -[v3_req] -subjectAltName = @alt_names - -[req_ext] -subjectAltName = @alt_names - -[alt_names] -DNS = localhost diff --git a/bindings/rust/bench/scripts/generate-certs.sh b/bindings/rust/bench/scripts/generate-certs.sh index 8fab212fc60..b2ac029a4f7 100755 --- a/bindings/rust/bench/scripts/generate-certs.sh +++ b/bindings/rust/bench/scripts/generate-certs.sh @@ -35,40 +35,108 @@ cert-gen () { mkdir -p $dir_name cd $dir_name + # The "basicConstraints" and "keyUsage" extensions are necessary for CA + # certificates that sign other certificates. Normally the openssl x509 tool + # will ignore the extensions requests in the .csr, but by using the + # copy_extensions=copyall flag we can pass the extensions from the .csr on + # to the final public certificate. + + # The advantage of manually specifying the extensions is that there is no + # dependency on any openssl config files + echo "generating CA private key and certificate" - openssl req -new -nodes -x509 -newkey $key_family -pkeyopt $argname$key_size -keyout ca-key.pem -out ca-cert.pem -days 65536 -config ../config/ca.cnf + openssl req -new -noenc -x509 \ + -newkey $key_family \ + -pkeyopt $argname$key_size \ + -keyout ca-key.pem \ + -out ca-cert.pem \ + -days 65536 \ + -subj "/C=US/CN=root" \ + -addext "basicConstraints = critical,CA:true" \ + -addext "keyUsage = critical,keyCertSign" + + echo "generating intermediate private key and CSR" + openssl req -new -noenc \ + -newkey $key_family \ + -pkeyopt $argname$key_size \ + -keyout intermediate-key.pem \ + -out intermediate.csr \ + -subj "/C=US/CN=branch" \ + -addext "basicConstraints = critical,CA:true" \ + -addext "keyUsage = critical,keyCertSign" echo "generating server private key and CSR" - openssl req -new -nodes -newkey $key_family -pkeyopt $argname$key_size -keyout server-key.pem -out server.csr -config ../config/server.cnf + openssl req -new -noenc \ + -newkey $key_family \ + -pkeyopt $argname$key_size \ + -keyout server-key.pem \ + -out server.csr \ + -subj "/C=US/CN=leaf" \ + -addext "subjectAltName = DNS:localhost" echo "generating client private key and CSR" - openssl req -new -nodes -newkey $key_family -pkeyopt $argname$key_size -keyout client-key.pem -out client.csr -config ../config/client.cnf + openssl req -new -noenc \ + -newkey $key_family \ + -pkeyopt $argname$key_size \ + -keyout client-key.pem \ + -out client.csr \ + -subj "/C=US/CN=client" \ + -addext "subjectAltName = DNS:localhost" + + echo "generating intermediate certificate and signing it" + openssl x509 -days 65536 \ + -req -in intermediate.csr \ + -CA ca-cert.pem \ + -CAkey ca-key.pem \ + -CAcreateserial \ + -out intermediate-cert.pem \ + -copy_extensions=copyall echo "generating server certificate and signing it" - openssl x509 -days 65536 -req -in server.csr -CA ca-cert.pem -CAkey ca-key.pem -CAcreateserial -out server-cert.pem -extensions req_ext -extfile ../config/server.cnf + openssl x509 -days 65536 \ + -req -in server.csr \ + -CA intermediate-cert.pem \ + -CAkey intermediate-key.pem \ + -CAcreateserial -out server-cert.pem \ + -copy_extensions=copyall echo "generating client certificate and signing it" - openssl x509 -days 65536 -req -in client.csr -CA ca-cert.pem -CAkey ca-key.pem -CAcreateserial -out client-cert.pem -extensions req_ext -extfile ../config/client.cnf - - echo "verifying generated certificates" - openssl verify -CAfile ca-cert.pem server-cert.pem + openssl x509 -days 65536 \ + -req -in client.csr \ + -CA ca-cert.pem \ + -CAkey ca-key.pem \ + -CAcreateserial -out client-cert.pem \ + -copy_extensions=copyall + + touch server-chain.pem + cat server-cert.pem >> server-chain.pem + cat intermediate-cert.pem >> server-chain.pem + cat ca-cert.pem >> server-chain.pem + + echo "verifying server certificates" + openssl verify -CAfile ca-cert.pem intermediate-cert.pem + openssl verify -CAfile ca-cert.pem -untrusted intermediate-cert.pem server-cert.pem + + echo "verifying client certificates" openssl verify -CAfile ca-cert.pem client-cert.pem echo "cleaning up temporary files" rm server.csr + rm intermediate.csr rm client.csr rm ca-key.pem cd .. } -if [[ $1 != "clean" ]] +if [[ $1 != "clean" ]] then + cert-gen ec 256 ecdsa256 cert-gen ec 384 ecdsa384 cert-gen rsa 2048 rsa2048 cert-gen rsa 3072 rsa3072 cert-gen rsa 4096 rsa4096 -else +else echo "cleaning certs" rm -rf ecdsa*/ rsa*/ fi diff --git a/bindings/rust/bench/src/harness.rs b/bindings/rust/bench/src/harness.rs index 5e7697cad80..237fbee534d 100644 --- a/bindings/rust/bench/src/harness.rs +++ b/bindings/rust/bench/src/harness.rs @@ -25,7 +25,7 @@ impl PemType { fn get_filename(&self) -> &str { match self { PemType::ServerKey => "server-key.pem", - PemType::ServerCertChain => "server-cert.pem", + PemType::ServerCertChain => "server-chain.pem", PemType::ClientKey => "client-key.pem", PemType::ClientCertChain => "client-cert.pem", PemType::CACert => "ca-cert.pem", @@ -40,6 +40,7 @@ pub enum SigType { Rsa4096, #[default] Ecdsa384, + Ecdsa256, } impl SigType { @@ -49,6 +50,7 @@ impl SigType { SigType::Rsa3072 => "rsa3072", SigType::Rsa4096 => "rsa4096", SigType::Ecdsa384 => "ecdsa384", + SigType::Ecdsa256 => "ecdsa256", } } }