Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: cleanup and add config options #1807

Merged
merged 8 commits into from
Nov 22, 2019
Merged

runtime: cleanup and add config options #1807

merged 8 commits into from
Nov 22, 2019

Conversation

carllerche
Copy link
Member

@carllerche carllerche commented Nov 21, 2019

This patch finishes the cleanup as part of the transition to Tokio 0.2.
A number of changes were made to take advantage of having all Tokio
types in a single crate. Also, fixes using Tokio types from
spawn_blocking.

Many threads, one resource driver

Previously, in the threaded scheduler, a resource driver (mio::Poll /
timer combo) was created per thread. This was more or less fine, except
it required balancing across the available drivers. When using a
resource driver from outside of the thread pool, balancing is
tricky. The change was original done to avoid having a dedicated driver
thread.

Now, instead of creating many resource drivers, a single resource driver
is used. Each scheduler thread will attempt to "lock" the resource
driver before parking on it. If the resource driver is already locked,
the thread uses a condition variable to park. Contention should remain
low as, under load, the scheduler avoids using the drivers.

Add configuration options to enable I/O / time

New configuration options are added to runtime::Builder to allow
enabling I/O and time drivers on a runtime instance basis. This is
useful when wanting to create lightweight runtime instances to execute
compute only tasks.

Bug fixes

The condition variable parker is updated to the same algorithm used in
std. This is motivated by some potential deadlock cases discovered by
loom.

The basic scheduler is fixed to fairly schedule tasks. push_front was
accidentally used instead of push_back.

I/O, time, and spawning now work from within spawn_blocking closures.

Misc cleanup

The threaded scheduler is no longer generic over P :Park. Instead, it
is hard coded to a specific parker. Tests, including loom tests, are
updated to use Runtime directly. This provides greater coverage.

The blocking module is moved back into runtime as all usage is
within runtime itself.

This patch finishes the cleanup as part of the transition to Tokio 0.2.
A number of changes were made to take advantage of having all Tokio
types in a single crate. Also, fixes using Tokio types from
`spawn_blocking`.

* Many threads, one resource driver

Previously, in the threaded scheduler, a resource driver (mio::Poll /
timer combo) was created per thread. This was more or less fine, except
it required balancing across the available drivers. When using a
resource driver from **outside** of the thread pool, balancing is
tricky. The change was original done to avoid having a dedicated driver
thread.

Now, instead of creating many resource drivers, a single resource driver
is used. Each scheduler thread will attempt to "lock" the resource
driver before parking on it. If the resource driver is already locked,
the thread uses a condition variable to park. Contention should remain
low as, under load, the scheduler avoids using the drivers.

* Add configuration options to enable I/O / time

New configuration options are added to `runtime::Builder` to allow
enabling I/O and time drivers on a runtime instance basis. This is
useful when wanting to create lightweight runtime instances to execute
compute only tasks.

* Bug fixes

The condition variable parker is updated to the same algorithm used in
`std`. This is motivated by some potential deadlock cases discovered by
`loom`.

The basic scheduler is fixed to fairly schedule tasks. `push_front` was
accidentally used instead of `push_back`.

I/O, time, and spawning now work from within `spawn_blocking` closures.

* Misc cleanup

The threaded scheduler is no longer generic over `P :Park`. Instead, it
is hard coded to a specific parker. Tests, including loom tests, are
updated to use `Runtime` directly. This provides greater coverage.

The `blocking` module is moved back into `runtime` as all usage is
within `runtime` itself.
@hawkw
Copy link
Member

hawkw commented Nov 21, 2019

This was more or less fine, except
it required balancing across the available drivers. When using a
resource driver from outside of the thread pool, balancing is
tricky. The change was original done to avoid having a dedicated driver
thread.

If memory serves, there were at least some cases where having many drivers resulted in a performance advantage for IO-bound programs? Has there been any assessment about the performance impact of this change?

Copy link
Member

@hawkw hawkw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

based on our offline conversation, I think the move from a reactor per thread to a shared reactor is probably good. i had some questions about the code changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants