Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore(avm): improve column stats #11135

Merged
merged 1 commit into from
Jan 9, 2025
Merged

chore(avm): improve column stats #11135

merged 1 commit into from
Jan 9, 2025
+74 −36

Conversation

fcarreiro
Copy link
Contributor

@fcarreiro fcarreiro commented Jan 9, 2025 •
edited
Loading

Now that we use tight polynomials it makes sense to count things differently for our stats.

I'm gathering info in particular to see if it makes sense to use commit_sparse (which we are using now). From what I can see, probably not, or we should do it dynamically based on sparseness information.

Stats from the bulk test.

Median column fullness: 90%
Average column fullness: 66%
Fullness of all columns, ignoring empty ones:
  1: 100%  705: 100%  695: 100%  691: 100%  690: 100%  689: 100%  688: 100%  657: 100%  649: 100%  638: 100%  
637: 100%  636: 100%  632: 100%  268: 100%  267: 100%  194: 100%    0: 100%   25: 100%   15: 100%   20: 100%  
 11: 100%    8: 100%    2: 100%   66:  99%  175:  99%   75:  99%   90:  99%  101:  99%  108:  99%  117:  99%  
136:  99%  149:  99%  150:  99%  153:  99%  156:  99%  157:  99%  158:  99%  176:  99%  177:  99%  178:  99%  
186:  99%    3:  99%  196:  99%  308:  99%  307:  99%  302:  99%  282:  99%  275:  99%  274:  99%   35:  99%  
631:  99%  650:  99%   12:  99%   48:  99%    4:  99%  644:  98%  276:  98%  273:  98%    6:  96%  672:  96%  
666:  96%  673:  96%  658:  95%  120:  94%  139:  92%  141:  92%  143:  92%  140:  92%  144:  92%  121:  92%  
674:  91%   43:  91%  682:  91%   46:  91%  678:  91%  677:  91%  675:  91%  472:  90%  471:  90%  454:  90%  
445:  90%  470:  90%  455:  90%  456:  90%  457:  90%  458:  90%  459:  90%  469:  90%  468:  90%  460:  90%  
461:  90%  467:  90%  466:  90%  465:  90%  462:  90%  463:  90%  464:  90%  485:  90%  497:  90%  496:  90%  
495:  90%  494:  90%  493:  90%  492:  90%  491:  90%  490:  90%  489:  90%  488:  90%  487:  90%  486:  90%  
473:  90%  484:  90%  483:  90%  482:  90%  481:  90%  480:  90%  479:  90%  478:  90%  477:  90%  476:  90%  
475:  90%  474:  90%  401:  90%  426:  90%  425:  90%  424:  90%  423:  90%  422:  90%  421:  90%  400:  90%  
420:  90%  419:  90%  418:  90%  417:  90%  416:  90%  415:  90%  427:  90%  402:  90%  403:  90%  414:  90%  
404:  90%  405:  90%  413:  90%  412:  90%  411:  90%  410:  90%  409:  90%  408:  90%  406:  90%  439:  90%  
452:  90%  451:  90%  450:  90%  449:  90%  448:  90%  447:  90%  446:  90%  407:  90%  444:  90%  443:  90%  
442:  90%  441:  90%  440:  90%  453:  90%  438:  90%  437:  90%  398:  90%  436:  90%  435:  90%  434:  90%  
433:  90%  432:  90%  431:  90%  430:  90%  429:  90%  428:  90%  565:  90%  578:  90%  577:  90%  576:  90%  
575:  90%  574:  90%  573:  90%  572:  90%  571:  90%  570:  90%  569:  90%  568:  90%  567:  90%  566:  90%  
579:  90%  564:  90%  563:  90%  562:  90%  561:  90%  560:  90%  559:  90%  558:  90%  557:  90%  556:  90%  
555:  90%  554:  90%  553:  90%  593:  90%  629:  90%  627:  90%  626:  90%  625:  90%  624:  90%  623:  90%  
622:  90%  621:  90%  620:  90%  619:  90%  618:  90%  617:  90%  594:  90%  552:  90%  592:  90%  591:  90%  
590:  90%  589:  90%  587:  90%  586:  90%  585:  90%  584:  90%  583:  90%  582:  90%  581:  90%  580:  90%  
511:  90%  524:  90%  523:  90%  522:  90%  521:  90%  520:  90%  519:  90%  518:  90%  517:  90%  516:  90%  
515:  90%  514:  90%  513:  90%  512:  90%  525:  90%  510:  90%  509:  90%  508:  90%  507:  90%  506:  90%  
505:  90%  504:  90%  503:  90%  502:  90%  501:  90%  500:  90%  499:  90%  538:  90%  551:  90%  550:  90%  
549:  90%  548:  90%  547:  90%  546:  90%  545:  90%  544:  90%  543:  90%  542:  90%  541:  90%  540:  90%  
539:  90%  498:  90%  537:  90%  536:  90%  535:  90%  534:  90%  533:  90%  532:  90%  531:  90%  530:  90%  
529:  90%  528:  90%  527:  90%  526:  90%  352:  90%  343:  90%  344:  90%  345:  90%  346:  90%  347:  90%  
348:  90%  349:  90%  350:  90%  351:  90%  342:  90%  353:  90%  354:  90%  355:  90%  356:  90%  357:  90%  
358:  90%  359:  90%  360:  90%  333:  90%  399:  90%  326:  90%  327:  90%  328:  90%  329:  90%  330:  90%  
331:  90%  332:  90%  397:  90%  334:  90%  335:  90%  336:  90%  337:  90%  338:  90%  339:  90%  340:  90%  
341:  90%  388:  90%  380:  90%  381:  90%  382:  90%  383:  90%  384:  90%  385:  90%  386:  90%  387:  90%  
362:  90%  389:  90%  390:  90%  391:  90%  392:  90%  393:  90%  394:  90%  395:  90%  396:  90%  378:  90%  
379:  90%  361:  90%  363:  90%  364:  90%  365:  90%  366:  90%  367:  90%  368:  90%  369:  90%  370:  90%  
371:  90%  372:  90%  373:  90%  374:  90%  375:  90%  376:  90%  377:  90%   44:  89%  312:  88%  305:  88%  
 31:  87%   72:  86%  676:  86%   45:  85%  280:  84%  311:  84%   91:  83%  163:  80%  185:  80%  272:  78%  
190:  74%  199:  74%  182:  73%  187:  70%  132:  70%  125:  70%    5:  66%  197:  65%  180:  63%  119:  63%  
165:  60%  700:  60%  135:  57%   62:  56%   63:  53%   64:  53%   49:  51%  671:  50%  670:  50%  668:  50%  
667:  50%  680:  50%   18:  50%  145:  50%  146:  50%  148:  50%  174:  47%  701:  44%  287:  39%  281:  39%  
 41:  37%  588:  36%  192:  36%  164:  35%  198:  34%  285:  34%  181:  34%   37:  34%   38:  34%  151:  33%  
651:  33%  652:  33%  642:  32%  653:  32%  654:  32%  635:  32%  655:  32%  656:  32%  122:  29%  201:  28%  
236:  28%   39:  26%   67:  25%   40:  21%  203:  18%  123:  18%  286:  18%   53:  17%   10:  16%  283:  15%  
200:  13%  173:  13%  230:  13%  702:  13%  183:  13%  683:  11%  696:  11%   52:  10%  248:   9%  166:   9%  
 42:   8%   54:   8%  697:   8%    7:   7%  685:   7%  229:   7%  288:   7%  234:   6%  270:   5%   55:   4%  
235:   3%  278:   3%   33:   3%   71:   3%   56:   3%  167:   3%  693:   3%  218:   3%   50:   3%   28:   2%  
227:   2%  226:   2%  304:   1%   74:   1%   59:   1%    9:   1%  155:   1%  643:   1%   30:   1%  208:   1%  
646:   1%  665:   1%  238:   1%  239:   1%  692:   0%   27:   0%  687:   0%   58:   0%  686:   0%   57:   0%  
684:   0%  704:   0%  660:   0%  659:   0%   47:   0%  703:   0%   51:   0%  694:   0%  258:   0%  241:   0%  
243:   0%  244:   0%  245:   0%  246:   0%  247:   0%  249:   0%  250:   0%  251:   0%  252:   0%  253:   0%  
254:   0%  255:   0%  256:   0%  257:   0%  240:   0%  261:   0%  269:   0%  277:   0%  289:   0%  290:   0%  
291:   0%  292:   0%  293:   0%  294:   0%  295:   0%  296:   0%  297:   0%  303:   0%  309:   0%  212:   0%  
 68:   0%   73:   0%  152:   0%  160:   0%  161:   0%  162:   0%  188:   0%  189:   0%  195:   0%  204:   0%  
206:   0%  207:   0%  209:   0%  210:   0%  211:   0%   65:   0%  214:   0%  215:   0%  216:   0%  217:   0%  
219:   0%  220:   0%  222:   0%  223:   0%  224:   0%  225:   0%  231:   0%  232:   0%  233:   0%  237:   0%  
Details for 20 most sparse columns:
Column main_sel_op_msm: 2 non-zero entries out of 28048 (0%)
Column main_sel_op_l2gasleft: 1 non-zero entries out of 28904 (0%)
Column main_sel_op_l1_to_l2_msg_exists: 1 non-zero entries out of 35497 (0%)
Column main_sel_op_keccak: 1 non-zero entries out of 14710 (0%)
Column main_sel_op_get_contract_instance: 3 non-zero entries out of 28157 (0%)
Column main_sel_op_fee_per_l2_gas: 1 non-zero entries out of 28740 (0%)
Column main_sel_op_fee_per_da_gas: 1 non-zero entries out of 28825 (0%)
Column main_sel_op_fdiv: 40 non-zero entries out of 27798 (0%)
Column main_sel_op_external_return: 3 non-zero entries out of 36569 (0%)
Column main_sel_op_external_call: 1 non-zero entries out of 35820 (0%)
Column main_sel_op_emit_unencrypted_log: 3 non-zero entries out of 35147 (0%)
Column main_sel_op_emit_nullifier: 1 non-zero entries out of 35335 (0%)
Column main_sel_op_emit_note_hash: 1 non-zero entries out of 35291 (0%)
Column main_sel_op_emit_l2_to_l1_msg: 1 non-zero entries out of 35580 (0%)
Column alu_remainder: 6 non-zero entries out of 6708 (0%)
Column main_sel_op_debug_log: 31 non-zero entries out of 36202 (0%)
Column main_sel_op_dagasleft: 1 non-zero entries out of 28983 (0%)
Column main_sel_op_chain_id: 1 non-zero entries out of 28426 (0%)
Column main_sel_op_calldata_copy: 6 non-zero entries out of 36361 (0%)
Column main_sel_op_block_number: 1 non-zero entries out of 28580 (0%)

@fcarreiro Graphite App
Copy link
Contributor Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

@fcarreiro fcarreiro requested review from dbanks12 and removed request for IlyasRidhuan, jeanmon and Maddiaa0 January 9, 2025 18:48
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 9, 2025

Changes to circuit sizes

Generated at commit: bb1de30f3612841fa0372ba6b174be9f36c5ece1, compared to commit: 61c3e95aaa7fc6c6c4583242cf0263f39b29d084

🧾 Summary (100% most significant diffs)

Program ACIR opcodes (+/-) % Circuit size (+/-) %
rollup_base_private +60,716 ❌ +10.90% +132,884 ❌ +5.98%
private_kernel_tail +227 ❌ +4.43% +227 ❌ +0.83%
private_kernel_reset_0_0_64_0_0_0_0_0_0 +1 ❌ +0.01% +1 ❌ +0.00%
private_kernel_reset_64_0_0_0_0_0_0_0_0 +1 ❌ +0.01% +1 ❌ +0.00%
private_kernel_tail_to_public +1 ❌ +0.01% +1 ❌ +0.00%
private_kernel_reset_0_0_0_0_0_64_0_0_0 +1 ❌ +0.00% +1 ❌ +0.00%
rollup_base_public +71 ❌ +0.01% +57 ❌ +0.00%
private_kernel_inner +1 ❌ +0.00% +1 ❌ +0.00%
private_kernel_reset_0_0_0_0_64_0_0_0_0 +1 ❌ +0.01% +1 ❌ +0.00%
private_kernel_reset_0_64_0_0_0_0_0_0_0 +1 ❌ +0.00% +1 ❌ +0.00%
private_kernel_reset_0_0_0_64_0_0_0_0_0 +1 ❌ +0.00% +1 ❌ +0.00%
rollup_block_root +1 ❌ +0.00% +2 ❌ +0.00%
rollup_root 0 ➖ 0.00% -2 ✅ -0.00%
rollup_block_merge 0 ➖ 0.00% -4 ✅ -0.00%
rollup_block_root_empty -1,068 ✅ -51.90% -875,971 ✅ -99.49%
Full diff report 👇
Program ACIR opcodes (+/-) % Circuit size (+/-) %
rollup_base_private 617,798 (+60,716) +10.90% 2,355,616 (+132,884) +5.98%
private_kernel_tail 5,348 (+227) +4.43% 27,501 (+227) +0.83%
private_kernel_reset_0_0_64_0_0_0_0_0_0 13,114 (+1) +0.01% 36,896 (+1) +0.00%
private_kernel_reset_64_0_0_0_0_0_0_0_0 13,114 (+1) +0.01% 36,896 (+1) +0.00%
private_kernel_tail_to_public 16,335 (+1) +0.01% 42,934 (+1) +0.00%
private_kernel_reset_0_0_0_0_0_64_0_0_0 29,821 (+1) +0.00% 59,894 (+1) +0.00%
rollup_base_public 849,484 (+71) +0.01% 3,749,059 (+57) +0.00%
private_kernel_inner 46,279 (+1) +0.00% 89,088 (+1) +0.00%
private_kernel_reset_0_0_0_0_64_0_0_0_0 15,344 (+1) +0.01% 97,090 (+1) +0.00%
private_kernel_reset_0_64_0_0_0_0_0_0_0 30,010 (+1) +0.00% 262,750 (+1) +0.00%
private_kernel_reset_0_0_0_64_0_0_0_0_0 31,356 (+1) +0.00% 274,496 (+1) +0.00%
rollup_block_root 719,214 (+1) +0.00% 4,379,531 (+2) +0.00%
rollup_root 38,468 (0) 0.00% 14,601,781 (-2) -0.00%
rollup_block_merge 38,484 (0) 0.00% 1,962,780 (-4) -0.00%
rollup_block_root_empty 990 (-1,068) -51.90% 4,508 (-875,971) -99.49%

@fcarreiro fcarreiro merged commit 535a14c into master Jan 9, 2025
53 checks passed
@fcarreiro Graphite App
Copy link
Contributor Author

Merge activity

  • Jan 9, 2:54 PM EST: A user merged this pull request with Graphite.

@fcarreiro fcarreiro deleted the fc/avm-column-stats branch January 9, 2025 19:54
@AztecBot AztecBot mentioned this pull request Jan 9, 2025
Open
TomAFrench added a commit that referenced this pull request Jan 9, 2025
@TomAFrench
Merge branch 'master' into sync-noir
e791a61 
* master: (287 commits)
  feat: Sync from noir (#11051)
  chore(docs): Update tx concepts page (#10947)
  chore(docs): Edit Aztec.nr Guide section (#10866)
  chore: test:e2e defaults to no-docker (#10966)
  chore(avm): improve column stats (#11135)
  chore: Sanity checking of proving job IDs (#11134)
  feat: permutation argument optimizations  (#10960)
  feat: single tx block root rollup (#11096)
  refactor: prover db config (#11126)
  feat: monitor event loop lag (#11127)
  chore: Greater stability at 1TPS (#10981)
  chore: Jest reporters for CI (#11125)
  fix: Sequencer times out L1 tx at end of L2 slot (#11112)
  feat: browser chunking (#11102)
  fix: Added start/stop guards to running promise and serial queue (#11120)
  fix: Don't retransmit txs upon node restart (#11123)
  fix: Prover node aborts execution at epoch end (#11111)
  feat: blob sink in sandbox without extra process (#11032)
  chore: log number of instructions executed for call in AVM. Misc fix. (#11110)
  git subrepo push --branch=master noir-projects/aztec-nr
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dbanks12 dbanks12 dbanks12 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@fcarreiro @dbanks12