GIPA, TIPP, MIPP, and example of Groth16 proof aggregation #3

nirvantyagi · 2020-08-11T05:03:20Z

This pull request provides the following additions:

inner_products: Generic trait for inner products including implementations for pairing, multiexponentiation, and scalar inner products
dh_commitments: Generic trait for doubly-homomorphic commitments including implementations for Pedersen and AFGHO16
ip_proofs/gipa: Generic implementation of generalized inner product argument with respect to a set of doubly-homomorphic commitments and an inner product.
ip_proofs/tipa: Generic implementation of trusted inner product argument for DH commitments that have commitment keys compatible with the TIPP KZG polynomial commitment trick, e.g. Pedersen and AFGHO16.
ip_proofs/tipa/structured_scalar_message: Extends TIPP with alternate proving and verification algorithms when one of the messages in the inner product is a structured scalar of the form (1, r, r^2, ...). (trusted MIPP is captured by this)
ip_proofs/examples/groth16_aggregation: Example using TIPP and MIPP for Groth16 proof aggregation.

I moved the previous code for SIPP into sipp. It might be nice to generalize the "plaintext" message option that is used in SIPP and structured_scalar_message in a future refactor. Currently they fall outside the abstraction of GIPA and DH commitments.

I tried my best to keep things modular and reuse code as much as possible. In some cases, this led to some repeated computation, so there is plenty of room for optimization. I have also not yet implemented any support for parallel computation.

…rmats

…ition

Pratyush · 2020-08-26T17:43:16Z

Hi Nirvan, Sorry for the delay on this, I'll try to leave a detailed review today!

nirvantyagi · 2020-08-26T17:51:07Z

No worries. I have some updated benchmarks for the Groth16 proof aggregation (I realized I wasn't building the optimized binary), so now the numbers are more or less what we would expect from counting the group operations.

# of proofs	generation (ms)	aggregation (ms)	verification (ms)
16	130	690	138
64	572	2890	187
256	2095	11650	232
1024	9238	48976	280

Pratyush · 2020-09-01T16:21:36Z

dh_commitments/src/identity/mod.rs

+        let mut out = Vec::new();
+        out.extend_from_slice(m);
+        Ok(IdentityOutput(out))


Suggested change

let mut out = Vec::new();

out.extend_from_slice(m);

Ok(IdentityOutput(out))

Ok(IdentityOutput(m.to_vec()))

Pratyush · 2020-09-01T16:23:35Z

dh_commitments/src/pedersen/mod.rs

+use inner_products::{InnerProduct, MultiexponentiationInnerProduct};
+
+#[derive(Clone)]
+pub struct PedersenCommitment<G: Group> {


You can change from G: Group to C: ProjectiveCurve if that helps you here (for efficiency or otherwise)

👍

I changed it to ProjectiveGroup in order to use the improved VariableMSM calculation of MultiexponentiationInnerProduct.

Pratyush · 2020-09-01T16:27:39Z

inner_products/src/lib.rs

+            left.iter()
+                .zip(right)
+                .map(|(v, a)| P::pairing(v.clone().into(), a.clone().into()))
+                .product(),


This does n final exponentations, you probably want to use something like this: https://github.com/scipr-lab/ripp/blob/e18c63cdda5710c9c65a7ec43532ac8281af3f35/src/lib.rs#L150

👍

Done. I used:

https://github.com/scipr-lab/zexe/blob/master/algebra-core/src/curves/mod.rs#L73

I think adding in parallelization support can be done in a future pass?

Pratyush · 2020-09-01T16:29:33Z

inner_products/src/lib.rs

+                right.len(),
+            )));
+        };
+        Ok(left.iter().zip(right).map(|(g, x)| g.mul(x)).sum())


For larger inputs (len > 100), you might want to use something like https://github.com/scipr-lab/zexe/blob/38b6f6f2c6b9a6a1b5e8eb90b1287c98b06520d1/algebra-core/src/msm/variable_base.rs#L100

It should speed up this inner product massively.

👍

Had to change the generic parameter away from Group to a curve in order to use VariableMSM::multi_scalar_mul.

Pratyush · 2020-09-01T16:50:35Z

ip_proofs/src/gipa.rs

+    D: Digest,
+    IP: InnerProduct<
+        LeftMessage = LMC::Message,
+        RightMessage = RMC::Message,
+        Output = IPC::Message,
+    >,
+    LMC: DoublyHomomorphicCommitment,
+    RMC: DoublyHomomorphicCommitment<Scalar = LMC::Scalar>,
+    IPC: DoublyHomomorphicCommitment<Scalar = LMC::Scalar>,
+    RMC::Message: MulAssign<LMC::Scalar>,
+    IPC::Message: MulAssign<LMC::Scalar>,
+    RMC::Key: MulAssign<LMC::Scalar>,
+    IPC::Key: MulAssign<LMC::Scalar>,
+    RMC::Output: MulAssign<LMC::Scalar>,
+    IPC::Output: MulAssign<LMC::Scalar>,


It might be easier to wrap up things into an GIPAConfig trait, maybe something like the following:

pub trait GIPAConfig { type Scalar: ...; type LMCKey: ...; type RMCKey: ...; type IPCKey: ...; type LMCMsg: ...; type RMCMsg: ...; type IPCMsg: ...; type LMCOutput: ...; type RMCOutput: ...; type IPCOutput: ...; type IP: InnerProduct<LeftMessage = Self::LMCMsg, RightMessage = Self::RMCMsg, Output = Self::IPCMsg>; type LMC: ...; type RMC: ...; type IPC: ...; }

Agreed with your other comment that having a GIPAConfig that implements Clone would allow other structs that parameterize with GIPAConfig to simply derive Clone, which would be easier.

Initializing a GIPAConfig would be slightly more annoying due to the 9 extra types (Key, Msg, Output) that need to be specified.

I'm going to keep as is for now, just because of inertia to refactor, but if you have a strong preference, I'm happy to make the change.

Separately: In zexe, is there a reason Group doesn't implement Mul<Self> or even better Mul<&Self>? Currently am using MulAssign as a workaround.

Do you mean Mul<Scalar>? It's because we didn't like the idea of right multiplication by the scalar =P

Pratyush · 2020-09-01T16:56:05Z

ip_proofs/src/gipa.rs

+        if !(LMC::verify(ck.0, values.0, com.0)?
+            && RMC::verify(ck.1, values.1, com.1)?
+            && IPC::verify(&vec![ck.2.clone()], &vec![values.2.clone()], com.2)?)


should we parallelize these? Or do you think it would be unnecessary (verification is already fast enough)?

Yes, verification will be a significant cost. I think instead of parallelizing the calls to each of these three verifies, we will get what we want when we implement parallelism for the inner products in inner_products, e.g. pedersen uses MultiexponentiationInnerProduct and afgho16 uses PairingInnerProduct.

Pratyush · 2020-09-01T17:02:06Z

ip_proofs/src/gipa.rs

+    > {
+        let (m_a, m_b) = values;
+        let (ck_a, ck_b, ck_t) = ck;
+        match m_a.len() {


maybe this is can be replaced with a

if !m_a.len().is_power_of_two() { return Err(...); } if m_a.len() == 1 { something } else { something else }

👍

Fixed when unrolling recursion.

Pratyush · 2020-09-01T17:12:35Z

ip_proofs/src/gipa.rs

+                let (mut r_steps, r_base, mut r_transcript, ck_base) = Self::recursive_prove(
+                    (&m_a_recurse, &m_b_recurse),
+                    (&ck_a_recurse, &ck_b_recurse, ck_t),
+                    &c,
+                )?;


Is there a way to do this in place/using iteration? Currently the memory required increases with the depth of recursion

👍

Yes, I've made this change now for all of the recursive implementations in gipa.

Pratyush · 2020-09-01T17:16:15Z

ip_proofs/src/gipa.rs

+        }
+    }
+
+    fn recursive_verify(


Is there a way to reduce this to a single MSM, like what Bulletproofs does? Instead of essentially repeating the prover? This should provide great speed-ups for the verifier.

Yes, we can do something like that (looking at Section 3.1 of BBBPWM18).

I see that BBBPWM18 describes the same bit representation trick that I had emailed you all about in the TIPP context to calculate these scalars in linear time (Section 6.2 "Computing scalars").

I changed the verification to perform the multiexponentiation all at once at the end. However, it does not currently get the gains of using VariableMSM::multi_scalar_mul since we do not bind the commitment keys (LMC::Key, RMC::Key) to AffineCurve -- a downside of the generalization. Perhaps, we can add two inner products to the generic parameterization to do this? Not a super clean solution... Thoughts?

One way around this would be to add a "multi-scalar-mul" trait, and invoke that? That trait would have different implementations for fields vs for curves. (Maybe that belongs in zexe, and not here?)

We can leave that to a future PR; I filed an issue for it.

Pratyush · 2020-09-01T17:16:51Z

ip_proofs/src/gipa.rs

+    }
+}
+
+impl<IP, LMC, RMC, IPC, D> Clone for GIPAProof<IP, LMC, RMC, IPC, D>


If moving to the GIPAConfig idea, you can just derive this.

See other response.

Pratyush · 2020-09-01T17:22:17Z

ip_proofs/src/tipa/mod.rs

+    }
+}
+
+impl<IP, LMC, RMC, IPC, P, D> TIPA<IP, LMC, RMC, IPC, P, D>


Would it make the code more modular/abstracted if you used https://github.com/scipr-lab/poly-commit/tree/master/src/kzg10 ?

(I don't know the answer, because some computations might be different)

There are some extra computations we need. For example, we need to support shifting by a scalar r. That can still be made to work on top of a KZG commitment abstraction. However, the problem is that the library is fixed to commit to G1, whereas we need KZG commitments on both G1 and G2.

Pratyush · 2020-09-01T17:24:10Z

ip_proofs/src/tipa/mod.rs

+    }
+}
+
+pub fn structured_generators_scalar_power<G: Group>(


If not using the existing KZG10 impl, then you should use https://github.com/scipr-lab/zexe/blob/38b6f6f2c6b9a6a1b5e8eb90b1287c98b06520d1/algebra-core/src/msm/fixed_base.rs

👍 Changed parameterization to ProjectiveCurve

Pratyush · 2020-09-01T17:27:55Z

ip_proofs/src/tipa/structured_scalar_message.rs

+//TODO: Structured message is a special case of the non-committed message and does not rely on TIPA
+//TODO: Can support structured group element messages as well as structured scalar messages
+
+// TODO: Currently implemented by completely reusing TIPA, which means wasted prover and verifier effort on B


Maybe you can extract out the common code into a method on TIPA, and then use that method in both TIPA and here.

Yeah, I think something like that could work. The problem with using TIPA black-box as I am right now is that it attempts to calculate the structured KZG polynomial commitment proof for both left and right. We would want to extract the polynomial commitment proving and verification code from the main prove and verify methods, so that they can be called separately for left and right.

Then to do TIPA for left and a structured scalar message for right, we can first run GIPA using a PlaceholderCommitment that has null commitment keys and null commitments. Then run the TIPA polynomial commitment code on just left.

I haven't made this change yet, but commenting here to make note of this as a possible plan forward.

nirvantyagi · 2020-09-05T04:19:53Z

Thanks for the comments! Sorry I wasn't able to get to it this week. I'll respond soon!

nirvantyagi added 19 commits July 27, 2020 19:27

Rearrange to workspace with sipp package

c20f4e6

Pedersen and AFGHO16 commitments

e0bb31f

Simplified doubly homomorphic commitment trait by requiring vector fo…

b52495f

…rmats

Add extension field wrapper to generically support scalar multiplication

de59348

Inner product trait and implementations

7861fbd

Tests for Pedersen and AFGHO commitments

c1abeb4

Cargo format

c4041eb

Identity commitment

229a4d1

GIPA recursive prover

d44331d

GIPA verifier

b7a0398

Tests for GIPA and bug fixes to counter nonce and extension group add…

aef2973

…ition

Add setup fn to inner product commitment argument

b84fc56

TIPA structured setup inner product proofs and tests

9932a7a

Evaluate polynomial on KZG challenge using product form

f35e838

TIPA proofs with structured scalar messages

bef329b

Allow for SRS shift in TIPA for composition with aggregation protocols

5b168c3

Groth16 aggregation example

dd89f63

Remove inner product commitment proof abstraction from GIPA

221aaab

Benchmarks for inner products, GIPA, and TIPA

86a7398

Pratyush reviewed Sep 1, 2020

View reviewed changes

nirvantyagi added 2 commits September 3, 2020 17:24

Move Groth16 aggregation to library

9283252

Setup method for inner products used for Groth16 aggregation

e9e0ad9

nirvantyagi added 7 commits September 8, 2020 13:05

Derive clone for TIPA SRS

48330b2

Groth16 aggregation with slice input

0ddbe47

Implement clone for TIPA proofs

435e2a1

More efficient inner products with VariableMSM and sharing Miller loop

58c999b

Unroll recursive implementation in GIPA

a76f703

Delay GIPA verifier exponenetiations to end

ccbe6b7

Use FixedBaseMSM for structured setup

5b00bf8

This was referenced Sep 15, 2020

Add MSM trait, use it in GIPA verifier #4

Open

Extract out duplicated TIPA code into common functions #5

Open

Pratyush merged commit e45028f into arkworks-rs:master Sep 15, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GIPA, TIPP, MIPP, and example of Groth16 proof aggregation #3

GIPA, TIPP, MIPP, and example of Groth16 proof aggregation #3

nirvantyagi commented Aug 11, 2020

Pratyush commented Aug 26, 2020

nirvantyagi commented Aug 26, 2020

Pratyush Sep 1, 2020

nirvantyagi Sep 9, 2020

Pratyush Sep 1, 2020

nirvantyagi Sep 9, 2020

Pratyush Sep 1, 2020

nirvantyagi Sep 9, 2020

Pratyush Sep 1, 2020

nirvantyagi Sep 9, 2020

Pratyush Sep 1, 2020

nirvantyagi Sep 9, 2020

Pratyush Sep 15, 2020

Pratyush Sep 1, 2020

nirvantyagi Sep 9, 2020

Pratyush Sep 1, 2020 •

edited

Loading

nirvantyagi Sep 9, 2020

Pratyush Sep 1, 2020

nirvantyagi Sep 9, 2020

Pratyush Sep 1, 2020

nirvantyagi Sep 9, 2020

Pratyush Sep 15, 2020

Pratyush Sep 15, 2020

Pratyush Sep 1, 2020

nirvantyagi Sep 9, 2020

Pratyush Sep 1, 2020

nirvantyagi Sep 9, 2020

Pratyush Sep 1, 2020

nirvantyagi Sep 9, 2020 •

edited

Loading

Pratyush Sep 1, 2020

nirvantyagi Sep 9, 2020

nirvantyagi commented Sep 5, 2020

GIPA, TIPP, MIPP, and example of Groth16 proof aggregation #3

GIPA, TIPP, MIPP, and example of Groth16 proof aggregation #3

Conversation

nirvantyagi commented Aug 11, 2020

Pratyush commented Aug 26, 2020

nirvantyagi commented Aug 26, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Pratyush Sep 1, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nirvantyagi Sep 9, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nirvantyagi commented Sep 5, 2020

Pratyush Sep 1, 2020 •

edited

Loading

nirvantyagi Sep 9, 2020 •

edited

Loading