Parallel vps' gas accounting rework #1835
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Describe your changes
Currently, during the parallel execution of vps (more specifically in the
try_fold
function we use theset
method ofVpsGas
to set the gas cost of the current Vp. This function only updates themax
field as it assumes theVpsGas
instance to have been just initialized and, therefore, be empty. From the rayon documentation though (https://docs.rs/rayon/latest/rayon/iter/trait.ParallelIterator.html#method.fold), it is stated that the sequence might be subdivided before being folded. In this case, more than on VP would be aggregated in one of these chunks and all of the VP following the first one would see a non-defaultVpsGas
instance, i.e. an instance with themax
field containing the gas cost of the previous vp. Given that the current implementation ofset
only overwrites themax
field, we would end up under assessing the gas cost of the transaction. The correct logic, in this case, would be to update the maximum (if needed) and push the cheaper vp cost to therest
field.I have tested this thing and it actually looks like rayon never produces subsequences with more than one element, i.e. all of the vps are allocated to their own subsequence and therefore do not incur in this problem. From some tests I saw that to trigger the misbehavior the collection needs to contain at least 8 elements, but this number is variable (depends from run to run) and also depends on the number of logical cpus allocated to rayon (which for Namada is non-constant, since we use half of the available logical cores). IMPORTANT: We might never come to such number of vps for a single transaction.
In the
set
method we also have a couple ofdebug_asserts
verifying that theVpsGas
instance is indeed empty (and therefore the current logic is correct).My concern is that if the internals of rayon change (which might even go unnoticed by SemVer), or if we start triggering more vps per transaction than we currently do, we might start to experience this issue on some machines. If we don't catch this with the
debug_asserts
and the code gets shipped we might end up with a non-consistent gas accounting which would eventually lead to a break in consensus.So this PR carries a small rework that ensures that the
identity
argument oftry_fold
is indeed an identity function and would guarantee the correct execution logic regardless of the rayon internals/execution context.Indicate on which release or other PRs this topic is based on
v0.22.0
Checklist before merging to
draft