Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add batch block processing result observer to block_processor #4167

Merged
merged 8 commits into from
Mar 5, 2023

Conversation

clemahieu
Copy link
Contributor

  • Implements the batch block processing observer from Add the ascending bootstrap client #4158.
  • Unifies notifications between block_processor observers 'processed' and 'batch_processed'.
  • New observer template which puts functionality in to its own file with minimal coupling.
  • This will be merge-committed to preserve functional change boundaries.

node::process_local cleanup

  • Can now block without needing to directly call ::process_one. Now it will be placed in to the block_processor queue.
  • Doesn't need to pass block_origin through process_one to process_live.
  • Return an optional since it may not be possible to return a result if the block_processor stops.

Block processor cleanup

  • Remove block_processor::wait_write workaround which was in effect trying to schedule database batches for node::process_local
  • Removes nano::block_origin as unneeded when replaced with observer
  • ::process_one is now fully encapsulated

block_processor::process_live cleanup

  • Removes block_origin as a parameter and uses a block_processor observer instead to do block publishing.
  • Removes unused parameters
  • Connect takes an enable flag which can be directly pulled from the node flags.

@clemahieu clemahieu added enhancement exclude from changelog Indicates the change is not relevant for appearing in the release changelog labels Mar 3, 2023
@clemahieu clemahieu added this to the V25.0 milestone Mar 3, 2023
@clemahieu clemahieu requested review from pwojcikdev and thsfs March 3, 2023 03:40
nano/node/blocking_observer.cpp Show resolved Hide resolved
nano/node/block_broadcast_strategy.cpp Outdated Show resolved Hide resolved
nano/node/block_broadcast_strategy.cpp Outdated Show resolved Hide resolved
@clemahieu clemahieu force-pushed the block_origin_remove branch from dcaf452 to 8cdaad7 Compare March 3, 2023 17:42
@clemahieu clemahieu force-pushed the block_origin_remove branch from 8cdaad7 to 4f7d249 Compare March 3, 2023 19:08
thsfs
thsfs previously approved these changes Mar 3, 2023
@pwojcikdev
Copy link
Contributor

Not sure about the way block origin is removed. You are trying to split data into multiple containers that naturally wants to be wrapped in a structure and move through the pipeline. It is more error prone and unnecessary, and in few places that it happens (election related code) it's a pain to work with. It's more or less the same workaround as with block arrival class, instead of tracking the source of each block (local/live/bootsrap) next to the block and then choosing broadcasting strategy based on that, we try to synchronise state across multiple components.

@clemahieu clemahieu force-pushed the block_origin_remove branch 2 times, most recently from b9c077d to d17339c Compare March 4, 2023 11:38
@clemahieu
Copy link
Contributor Author

clemahieu commented Mar 4, 2023

Not sure about the way block origin is removed. You are trying to split data into multiple containers that naturally wants to be wrapped in a structure and move through the pipeline. It is more error prone and unnecessary, and in few places that it happens (election related code) it's a pain to work with. It's more or less the same workaround as with block arrival class, instead of tracking the source of each block (local/live/bootsrap) next to the block and then choosing broadcasting strategy based on that, we try to synchronise state across multiple components.

I can see the reasoning for that and I think we'd be able to do something like that with the block_pipeline::context when merged. https://github.com/nanocurrency/nano-node/blob/6299819eaeaafcd302ca42170e6532de9272df6f/nano/node/block_pipeline/context.hpp

…he block_processor and broadcasting them on to the network.
@clemahieu clemahieu force-pushed the block_origin_remove branch 2 times, most recently from e22075d to 55d61b9 Compare March 4, 2023 11:53
@pwojcikdev
Copy link
Contributor

Not sure about the way block origin is removed. You are trying to split data into multiple containers that naturally wants to be wrapped in a structure and move through the pipeline. It is more error prone and unnecessary, and in few places that it happens (election related code) it's a pain to work with. It's more or less the same workaround as with block arrival class, instead of tracking the source of each block (local/live/bootsrap) next to the block and then choosing broadcasting strategy based on that, we try to synchronise state across multiple components.

I can see the reasoning for that and I think we'd be able to do something like that with the block_pipeline::context when merged. https://github.com/nanocurrency/nano-node/blob/6299819eaeaafcd302ca42170e6532de9272df6f/nano/node/block_pipeline/context.hpp

That looks like a perfect fit. The current solution is OK as a temporary measure until we get the block pipeline merged.

pwojcikdev
pwojcikdev previously approved these changes Mar 4, 2023
@clemahieu clemahieu force-pushed the block_origin_remove branch from d459479 to 8f9536b Compare March 4, 2023 15:08
@clemahieu clemahieu merged commit b7816ba into nanocurrency:develop Mar 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement exclude from changelog Indicates the change is not relevant for appearing in the release changelog
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants