Skip to content
This repository has been archived by the owner on Aug 16, 2024. It is now read-only.

Simulate memory queues in parallel for BWG #166

Merged
merged 35 commits into from
Aug 15, 2024
Merged

Conversation

0xVolosnikov
Copy link
Contributor

@0xVolosnikov 0xVolosnikov commented Jul 23, 2024

What ❔

The greatest amount of time during BWG is spent simulating queues. At the moment, the queue simulation is single-threaded process. But different queues can be simulated in parallel, since they do not depend on each other.

This pull request does the following:

  • Move simulation of memory queue implicit part out of individual_circuits. Currently when processing memory-related individual circuits, a part of the queue is simulated for each of circuits. But a more efficient approach is to first collect all the implicit queries and simulate the entire memory queue before memory-related circuits.
  • Since we can now simulate the entire memory queue without interruption, including the implicit part, we also simulate a sorted version of the queue in parallel.
  • Part of witnesses produced during simulation is sent to storage immediately (to greatly reduce RAM usage).
  • Callstack simulation moved to separate thread.

As a result, a decrease in the time on the BWG by ~40% is observed locally.
Peak RAM decreased by ~30%

Checklist

  • PR title corresponds to the body of PR (we generate changelog entries from PRs).
  • Tests for the changes have been added / updated.
  • Documentation comments have been added / updated.
  • Code has been formatted via zk fmt and zk lint.

@0xVolosnikov 0xVolosnikov marked this pull request as draft July 23, 2024 08:24
@0xVolosnikov 0xVolosnikov marked this pull request as ready for review July 23, 2024 11:17
@0xVolosnikov 0xVolosnikov requested a review from EmilLuta July 23, 2024 11:21
@0xVolosnikov 0xVolosnikov marked this pull request as draft July 23, 2024 13:41
@0xVolosnikov 0xVolosnikov changed the title Simulate queues in parallel for witgen Simulate memory queues in parallel for BWG Aug 14, 2024
@0xVolosnikov
Copy link
Contributor Author

Should be used together with matter-labs/zksync-era#2652

@0xVolosnikov 0xVolosnikov marked this pull request as ready for review August 14, 2024 10:49
@EmilLuta EmilLuta merged commit ee8927a into v1.5.0 Aug 15, 2024
5 checks passed
@0xVolosnikov 0xVolosnikov deleted the vv-oracle-parallelization2 branch August 15, 2024 10:21
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants