Integrate bulletproof rewinding #25

philipr-za · 2020-11-30T05:33:26Z

This PR integrates the rewinding of the dalek bulletproofs, currently implemented in the Tari fork of the bulletproofs crate, into tari_crypto. A method is provided that construct the rewindable range proof and two methods are provided to rewind the bulletproof to two levels, the first reveals the committed value and the proof message and the second will fully rewinds the bullet proof to also reveal the blinding factor.

While the underlying bulletproof implementation can accept a 23 byte proof message we use the first 4 bytes for a canonical message that lets us determine if the rewinding was successful or produced a garbage output. The remaining 19 bytes are provided for the client to use as they will

hansieodendaal · 2020-11-30T07:36:40Z

Getting this error on Windows:

2020/11/30
09:35	Cargo project update failed:
			Execution failed (exit code 1).
			        C:/Users/pluto/.cargo/bin/cargo.exe metadata --verbose --format-version 1 --all-features
			        stdout : error: the 'cargo.exe' binary, normally provided by the 'cargo' component, is not applicable to the 'nightly-2020-09-15-x86_64-pc-windows-msvc' toolchain
			error: backtrace:
			error:    0: <unknown>
			   1: <unknown>
			   2: <unknown>
			   3: <unknown>
			   4: <unknown>
			   5: <unknown>
			   6: <unknown>
			   7: BaseThreadInitThunk
			   8: RtlUserThreadStart
			
			
			        stderr : 

09:35	Auto fetch: finished

philipr-za · 2020-11-30T07:37:54Z

Getting this error on Windows:

2020/11/30
09:35	Cargo project update failed:
			Execution failed (exit code 1).
			        C:/Users/pluto/.cargo/bin/cargo.exe metadata --verbose --format-version 1 --all-features
			        stdout : error: the 'cargo.exe' binary, normally provided by the 'cargo' component, is not applicable to the 'nightly-2020-09-15-x86_64-pc-windows-msvc' toolchain
			error: backtrace:
			error:    0: <unknown>
			   1: <unknown>
			   2: <unknown>
			   3: <unknown>
			   4: <unknown>
			   5: <unknown>
			   6: <unknown>
			   7: BaseThreadInitThunk
			   8: RtlUserThreadStart
			
			
			        stderr : 

09:35	Auto fetch: finished

that doesn't look like an issue related to the PR?

CjS77

Look good. A few maintainability and readability nits.

src/ristretto/dalek_range_proof.rs

CjS77 · 2020-11-30T07:34:31Z

src/ristretto/dalek_range_proof.rs

+    fn construct_proof_with_rewind_key(&self, key: &RistrettoSecretKey, value: u64, rewind_key: &RistrettoSecretKey, rewind_blinding_key: &RistrettoSecretKey, proof_message: &[u8; 19]) -> Result<Vec<u8>, RangeProofError> {
+        let mut pt = Transcript::new(b"tari");
+        let mut full_proof_message = [0u8; 23];
+        full_proof_message[0..4].clone_from_slice(b"tari");


Extract message as a const; And I'd highly recommend making it 2 bytes to give a precious extra 2 bytes to the message. 1 byte is a bit short 1:256 chance of getting a false positive, 2 bytes is 1:65k.

Will do, was just following the precedent set by the transcript but I agree.

The selection of how many bytes to use felt tricky. Happy to make it 2 for a 1 out of 65k chance of a false positive. Is that truly big enough? Maybe split the difference for a 3 byte check?

src/ristretto/dalek_range_proof.rs

src/range_proof.rs

src/ristretto/dalek_range_proof.rs

hansieodendaal · 2020-11-30T09:43:40Z

Getting this error on Windows:
that doesn't look like an issue related to the PR?

It works perfectly with nightly-2020-06-10, but not with nightly-2020-09-15

philipr-za · 2020-11-30T09:57:08Z

Getting this error on Windows:

that doesn't look like an issue related to the PR?

It works perfectly with nightly-2020-06-10, but not with nightly-2020-09-15

@CjS77 Shall we update the version in the repo?

hansieodendaal · 2020-11-30T09:57:20Z

Could we perhaps run cargo fmt --all on this crate? I think it would improve readability.

philipr-za · 2020-11-30T09:58:08Z

Could we perhaps run cargo fmt --all on this crate? I think it would improve readability.

I was keen to but lets do it as a separate PR else it will confuse what this PR was updating.

CjS77 · 2020-11-30T10:05:51Z

It works perfectly with nightly-2020-06-10, but not with nightly-2020-09-15

@CjS77 Shall we update the version in the repo?

Check the commit logs. IIRC there was a reason to move away from 2020-06-10. I forget the reason, but was Simd / WASM incompatibility or something.

Yup, see #23

CjS77

LGTM

philipr-za · 2020-11-30T10:41:05Z

Had to update the branch of our fork of the Bulletproofs crate which was moved from main to development to reflect the branch naming of Daleks crate.

hansieodendaal

One comment below, and I think we should find a toolchain that also compiles on Windows.

src/ristretto/dalek_range_proof.rs

philipr-za · 2020-11-30T11:20:41Z

One comment below, and I think we should find a toolchain that also compiles on Windows.

Can you suggest a toolchain that works for windows and allows for the WASM issue?

hansieodendaal

Please see suggested code change below.
Toolchain nightly-2020-09-16 looks promising for Windows builds; maybe for wasm_builder as well. (Note: nightly-2020-09-15 is broken for Windows.)

Cargo.toml

philipr-za · 2020-11-30T11:59:48Z

@CjS77 I can't actually successfully run wasm-pack build . -- --features "wasm" for nightly-2020-09-15 or nightly-2020-09-16 but I can run it for nightly-2020-06-10

philipr-za · 2020-11-30T12:31:20Z

Ok we tested nightly-2020-11-24 on ubuntu, mac os x and windows and it seems to work for the wasm build.

CjS77 · 2020-11-30T12:53:07Z

@philipr-za Were you getting this issue on the 2020-09-15 build? rust-lang/rust#76698

Not sure why it worked when I was troubleshooting this before -- maybe the LLVM dylib was cached on the system or something.

CjS77 · 2020-11-30T12:53:53Z

Confirmed: nightly-2020-11-24does not compile with-all-features`

nightly-2020-09-11 seems to work for wasm and all-features on Mac.

philipr-za · 2020-11-30T12:54:47Z

@philipr-za Were you getting this issue on the 2020-09-15 build? rust-lang/rust#76698

Not sure why it worked when I was troubleshooting this before -- maybe the LLVM dylib was cached on the system or something.

Yes that was the exact issue I was having.

hansieodendaal · 2020-11-30T14:30:02Z

Toolchain nightly-2020-09-11 can be compiled on WIndows and Ubuntu

cargo build --all-features --release
wasm-pack build --target nodejs -d tari_js . -- --features "wasm"

However, the demo only works on Windows:

node wasm-demo

All toolchains tested with Ubuntu 18.04 LTS and Node.js 8.10.0 , even the prior nightly-2020-09-15, gives this error:

/home/pluto/@test/tari-crypto/tari_js/tari_crypto.js:80
const uint64CvtShim = new BigUint64Array(u32CvtShim.buffer);
                      ^

ReferenceError: BigUint64Array is not defined
    at Object.<anonymous> (/home/pluto/@test/tari-crypto/tari_js/tari_crypto.js:80:23)
    at Module._compile (module.js:652:30)
    at Object.Module._extensions..js (module.js:663:10)
    at Module.load (module.js:565:32)
    at tryModuleLoad (module.js:505:12)
    at Function.Module._load (module.js:497:3)
    at Module.require (module.js:596:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (/home/pluto/@test/tari-crypto/wasm-demo.js:25:19)
    at Module._compile (module.js:652:30)
~

Update: Working with the latest version of Node.js (nvm install v15)

hansieodendaal · 2020-11-30T15:46:21Z

Update: Node.js requires v10+

This could probably be added to README.md

This PR integrates the rewinding of the dalek bulletproofs into tari_crypto. A method is provided that construct the rewindable range proof and two methods are provided to rewind the bullet proof to two levels, the first reveals the committed value and the proof message and the second will fully rewind the bullet proof to also reveal the blinding factor. While the underlying bulletproof implementation can accept a 23 byte proof message we use the first 4 bytes for a canonical message that lets us determine if the rewinding was successful or produced a garbage output. The remaining 19 bytes are provided for the client to use as they will

CjS77 reviewed Nov 30, 2020

View reviewed changes

philipr-za force-pushed the philip-rewind branch from 03237dd to f2fbfa9 Compare November 30, 2020 08:39

CjS77 previously approved these changes Nov 30, 2020

View reviewed changes

philipr-za dismissed CjS77’s stale review via 7b02b1b November 30, 2020 10:40

philipr-za force-pushed the philip-rewind branch from f2fbfa9 to 7b02b1b Compare November 30, 2020 10:40

hansieodendaal reviewed Nov 30, 2020

View reviewed changes

src/ristretto/dalek_range_proof.rs Outdated Show resolved Hide resolved

philipr-za force-pushed the philip-rewind branch from 7b02b1b to 9dac756 Compare November 30, 2020 11:25

hansieodendaal reviewed Nov 30, 2020

View reviewed changes

Cargo.toml Outdated Show resolved Hide resolved

philipr-za force-pushed the philip-rewind branch from 9dac756 to c232fbc Compare November 30, 2020 11:58

philipr-za force-pushed the philip-rewind branch from c232fbc to 6253992 Compare November 30, 2020 12:29

philipr-za force-pushed the philip-rewind branch from 6253992 to 19619f1 Compare November 30, 2020 13:32

philipr-za force-pushed the philip-rewind branch from 19619f1 to dded0fa Compare December 1, 2020 06:59

philipr-za force-pushed the philip-rewind branch from dded0fa to 4f87194 Compare December 1, 2020 08:08

CjS77 merged commit d09e1ec into tari-project:main Dec 1, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Integrate bulletproof rewinding #25

Integrate bulletproof rewinding #25

philipr-za commented Nov 30, 2020

hansieodendaal commented Nov 30, 2020

philipr-za commented Nov 30, 2020

CjS77 left a comment

CjS77 Nov 30, 2020

philipr-za Nov 30, 2020 •

edited

Loading

hansieodendaal commented Nov 30, 2020

philipr-za commented Nov 30, 2020

hansieodendaal commented Nov 30, 2020

philipr-za commented Nov 30, 2020 •

edited

Loading

CjS77 commented Nov 30, 2020 •

edited

Loading

CjS77 left a comment

philipr-za commented Nov 30, 2020

hansieodendaal left a comment

philipr-za commented Nov 30, 2020

hansieodendaal left a comment •

edited

Loading

philipr-za commented Nov 30, 2020 •

edited

Loading

philipr-za commented Nov 30, 2020

CjS77 commented Nov 30, 2020

CjS77 commented Nov 30, 2020 •

edited

Loading

philipr-za commented Nov 30, 2020

hansieodendaal commented Nov 30, 2020 •

edited

Loading

hansieodendaal commented Nov 30, 2020

Integrate bulletproof rewinding #25

Integrate bulletproof rewinding #25

Conversation

philipr-za commented Nov 30, 2020

hansieodendaal commented Nov 30, 2020

philipr-za commented Nov 30, 2020

CjS77 left a comment

Choose a reason for hiding this comment

CjS77 Nov 30, 2020

Choose a reason for hiding this comment

philipr-za Nov 30, 2020 • edited Loading

Choose a reason for hiding this comment

hansieodendaal commented Nov 30, 2020

philipr-za commented Nov 30, 2020

hansieodendaal commented Nov 30, 2020

philipr-za commented Nov 30, 2020 • edited Loading

CjS77 commented Nov 30, 2020 • edited Loading

CjS77 left a comment

Choose a reason for hiding this comment

philipr-za commented Nov 30, 2020

hansieodendaal left a comment

Choose a reason for hiding this comment

philipr-za commented Nov 30, 2020

hansieodendaal left a comment • edited Loading

Choose a reason for hiding this comment

philipr-za commented Nov 30, 2020 • edited Loading

philipr-za commented Nov 30, 2020

CjS77 commented Nov 30, 2020

CjS77 commented Nov 30, 2020 • edited Loading

philipr-za commented Nov 30, 2020

hansieodendaal commented Nov 30, 2020 • edited Loading

hansieodendaal commented Nov 30, 2020

philipr-za Nov 30, 2020 •

edited

Loading

philipr-za commented Nov 30, 2020 •

edited

Loading

CjS77 commented Nov 30, 2020 •

edited

Loading

hansieodendaal left a comment •

edited

Loading

philipr-za commented Nov 30, 2020 •

edited

Loading

CjS77 commented Nov 30, 2020 •

edited

Loading

hansieodendaal commented Nov 30, 2020 •

edited

Loading