Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: optimize blockstream processing #358

Merged
merged 27 commits into from
Nov 25, 2024

Conversation

mattp-swirldslabs
Copy link
Contributor

@mattp-swirldslabs mattp-swirldslabs commented Nov 20, 2024

Description:

  • Leveraging unparsed.proto to define bytes types at various levels from the top down to avoid marshaling all the objects
  • Converted standard PBJ types to *Unparsed
  • Changed BlockAsDirWriter and BlockAsDirReader to read/write Bytes
  • Added JProfiler debugging ability to update-env.sh and docker-compose.yml for troubleshooting
  • Fixed unit tests

Related issue(s):

Fixes #353 #356

Notes for reviewer:

Checklist

  • Documented (Code comments, README, etc.)
  • Tested (unit, integration, etc.)

@mattp-swirldslabs mattp-swirldslabs added Improvement Code changes driven by non business requirements Block Node Issues/PR related to the Block Node. labels Nov 20, 2024
@mattp-swirldslabs mattp-swirldslabs self-assigned this Nov 20, 2024
@mattp-swirldslabs mattp-swirldslabs added this to the 0.3.0 milestone Nov 20, 2024
@mattp-swirldslabs mattp-swirldslabs force-pushed the 00353-optimize-blockstream-processing branch from 2c8f15e to 4a8b020 Compare November 20, 2024 15:32
@mattp-swirldslabs mattp-swirldslabs marked this pull request as ready for review November 22, 2024 21:16
@mattp-swirldslabs mattp-swirldslabs requested a review from a team as a code owner November 22, 2024 21:16
Copy link
Contributor

@ata-nas ata-nas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On my first pass through the changes, looks good! Just a couple of questions.

AlfredoG87
AlfredoG87 previously approved these changes Nov 25, 2024
Copy link
Contributor

@AlfredoG87 AlfredoG87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 💯

Signed-off-by: Matt Peterson <[email protected]>
Signed-off-by: Matt Peterson <[email protected]>
Signed-off-by: Matt Peterson <[email protected]>
Signed-off-by: Matt Peterson <[email protected]>
Signed-off-by: Matt Peterson <[email protected]>
Signed-off-by: Matt Peterson <[email protected]>
Signed-off-by: Matt Peterson <[email protected]>
Signed-off-by: Matt Peterson <[email protected]>
Signed-off-by: Matt Peterson <[email protected]>
Signed-off-by: Matt Peterson <[email protected]>
Signed-off-by: Matt Peterson <[email protected]>
Signed-off-by: Matt Peterson <[email protected]>
Signed-off-by: Matt Peterson <[email protected]>
@mattp-swirldslabs mattp-swirldslabs force-pushed the 00353-optimize-blockstream-processing branch from 5d5773c to 65f3d5c Compare November 25, 2024 19:28
@mattp-swirldslabs mattp-swirldslabs changed the title feat: 00353 optimize blockstream processing feat: optimize blockstream processing Nov 25, 2024
Copy link
Contributor

@AlfredoG87 AlfredoG87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@mattp-swirldslabs mattp-swirldslabs merged commit 544d983 into main Nov 25, 2024
15 checks passed
@mattp-swirldslabs mattp-swirldslabs deleted the 00353-optimize-blockstream-processing branch November 25, 2024 20:18
Copy link

codecov bot commented Nov 25, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.72%. Comparing base (c2fb9d0) to head (65f3d5c).
Report is 1 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main     #358      +/-   ##
============================================
+ Coverage     94.62%   97.72%   +3.09%     
- Complexity      316      332      +16     
============================================
  Files            69       69              
  Lines          1228     1229       +1     
  Branches         84       86       +2     
============================================
+ Hits           1162     1201      +39     
+ Misses           55       19      -36     
+ Partials         11        9       -2     
Files with missing lines Coverage Δ
...erver/consumer/ConsumerStreamResponseObserver.java 98.11% <100.00%> (ø)
...ock/server/mediator/LiveStreamMediatorBuilder.java 100.00% <100.00%> (ø)
.../block/server/mediator/LiveStreamMediatorImpl.java 100.00% <100.00%> (ø)
...block/server/mediator/MediatorInjectionModule.java 100.00% <ø> (+25.00%) ⬆️
...block/server/mediator/SubscriptionHandlerBase.java 100.00% <100.00%> (ø)
...com/hedera/block/server/notifier/NotifierImpl.java 100.00% <ø> (ø)
...a/block/server/pbj/PbjBlockAccessServiceProxy.java 100.00% <100.00%> (+10.20%) ⬆️
...a/block/server/pbj/PbjBlockStreamServiceProxy.java 91.07% <100.00%> (+34.70%) ⬆️
...server/persistence/PersistenceInjectionModule.java 80.00% <ø> (ø)
...rver/persistence/StreamPersistenceHandlerImpl.java 100.00% <100.00%> (ø)
... and 6 more

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Block Node Issues/PR related to the Block Node. Improvement Code changes driven by non business requirements
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Evaluate "Unparsed" schema approach
3 participants