Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[spike] determine whether dedicated threads are needed for high-priority tasks #642

Open
Voxelot opened this issue Sep 24, 2022 · 0 comments

Comments

@Voxelot
Copy link
Member

Voxelot commented Sep 24, 2022

In the context of block production and importing, they are consensus critical tasks that consume a lot of resources. There is some concern about whether making these async tasks could lead to starvation if the tokio runtime is very busy, which for example, may cause unanticipated delays for a round leader during consensus and risk missing a slot.

One proposed solution is to spawn dedicated threads for high priority tasks to ensure they continue to progress regardless of other low-priority async runtime activity (e.g. network requests):

It is one of two heavies operations(the second is block commit). And so it should spawn a separate thread that fully focuses on it.

Originally posted by @xgreenx in #409 (comment)

Another option is to use multiple async runtimes, to partition high priority tasks away from low priority ones.

Both of these approaches will increase the complexity of the node and add additional threadpool overhead for integration testing, but may be a viable approach to help guarantee liveness in a busy network. The alternative is to ensure we carefully balance our tasks, and rate limit p2p or API related tasks that are lower priority within a unified async runtime. The latter approach may prove to be more robust than spawning lots of busy threads of various priorities and spamming the OS scheduler.

xgreenx added a commit that referenced this issue Jul 5, 2024
Related: #642

### Before requesting review
- [x] I have reviewed the code myself
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant