Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

rustdoc: DocFS: Replace rayon with threadpool and enable it for all targets #109139

Merged
merged 2 commits into from
Mar 16, 2023
Merged

rustdoc: DocFS: Replace rayon with threadpool and enable it for all targets #109139

merged 2 commits into from
Mar 16, 2023

Conversation

GuillaumeGomez
Copy link
Member

Fixes #109060.

Switching to threadpool makes it a bit simpler for us to wait for all tasks in DocFS directly in the Drop implementation. I'm also curious if making all the writes into a thread pool could improve run time for rustdoc on all other platforms than Windows as well.

I'll run a perf check to see.

cc @ehuss
r? @notriddle

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. labels Mar 14, 2023
@rustbot
Copy link
Collaborator

rustbot commented Mar 14, 2023

These commits modify the Cargo.lock file. Random changes to Cargo.lock can be introduced when switching branches and rebasing PRs.
This was probably unintentional and should be reverted before this PR is merged.

If this was intentional then you can ignore this comment.

@GuillaumeGomez
Copy link
Member Author

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

Sign in to view
@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Mar 14, 2023
@bors
Copy link
Contributor

bors commented Mar 14, 2023

⌛ Trying commit 50f7520 with merge b130c05870fe372ca741990c39f48aa621d01e8e...

@bors
Copy link
Contributor

bors commented Mar 14, 2023

☀️ Try build successful - checks-actions
Build commit: b130c05870fe372ca741990c39f48aa621d01e8e (b130c05870fe372ca741990c39f48aa621d01e8e)

@rust-timer

This comment has been minimized.

Sign in to view
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (b130c05870fe372ca741990c39f48aa621d01e8e): comparison URL.

Overall result: ✅ improvements - no action needed

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)		 - - 0
Regressions ❌
(secondary)		 - - 0
Improvements ✅
(primary)		 - - 0
Improvements ✅
(secondary)		 -0.4% [-0.5%, -0.4%] 5
All ❌✅ (primary) - - 0

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)		 2.0% [1.7%, 2.3%] 2
Regressions ❌
(secondary)		 3.7% [3.7%, 3.7%] 1
Improvements ✅
(primary)		 -3.5% [-3.5%, -3.5%] 1
Improvements ✅
(secondary)		 -3.0% [-3.0%, -3.0%] 1
All ❌✅ (primary) 0.2% [-3.5%, 2.3%] 3

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)		 - - 0
Regressions ❌
(secondary)		 - - 0
Improvements ✅
(primary)		 -3.4% [-3.4%, -3.4%] 1
Improvements ✅
(secondary)		 -5.2% [-6.0%, -4.4%] 4
All ❌✅ (primary) -3.4% [-3.4%, -3.4%] 1

@rustbot rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Mar 15, 2023
@GuillaumeGomez
Copy link
Member Author

Seems to have very little impact so seems acceptable to keep it enabled for all targets.

notriddle
notriddle reviewed Mar 15, 2023
View reviewed changes
src/librustdoc/docfs.rs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The doc comment at the top of this file is outdated now

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed the part about the other platforms. Good catch!

@notriddle
Copy link
Contributor

@bors r+ rollup

@bors
Copy link
Contributor

bors commented Mar 15, 2023

📌 Commit e667872 has been approved by notriddle

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 15, 2023
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Mar 15, 2023
@matthiaskrgr
Rollup merge of rust-lang#109139 - GuillaumeGomez:rustdoc-windows-wai…
ead92dd 
…t-for-write, r=notriddle

rustdoc: DocFS: Replace rayon with threadpool and enable it for all targets

Fixes rust-lang#109060.

Switching to `threadpool` makes it a bit simpler for us to wait for all tasks in `DocFS` directly in the `Drop` implementation. I'm also curious if making all the writes into a thread pool could improve run time for rustdoc on all other platforms than Windows as well.

I'll run a perf check to see.

cc `@ehuss`
r? `@notriddle`
@matthiaskrgr matthiaskrgr mentioned this pull request Mar 15, 2023
Closed
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Mar 16, 2023
@matthiaskrgr
Rollup merge of rust-lang#109139 - GuillaumeGomez:rustdoc-windows-wai…
49f1623 
…t-for-write, r=notriddle

rustdoc: DocFS: Replace rayon with threadpool and enable it for all targets

Fixes rust-lang#109060.

Switching to `threadpool` makes it a bit simpler for us to wait for all tasks in `DocFS` directly in the `Drop` implementation. I'm also curious if making all the writes into a thread pool could improve run time for rustdoc on all other platforms than Windows as well.

I'll run a perf check to see.

cc ``@ehuss``
r? ``@notriddle``
@matthiaskrgr matthiaskrgr mentioned this pull request Mar 16, 2023
Closed
@matthiaskrgr matthiaskrgr mentioned this pull request Mar 16, 2023
Merged
bors added a commit to rust-lang-ci/rust that referenced this pull request Mar 16, 2023
@bors
Auto merge of rust-lang#109206 - matthiaskrgr:rollup-oev8ax6, r=matth…
7ac4b82 
…iaskrgr

Rollup of 10 pull requests

Successful merges:

 - rust-lang#108875 (rustdoc: fix type search for `Option` combinators)
 - rust-lang#108971 (error-msg: impl better suggestion for `E0532`)
 - rust-lang#109139 (rustdoc: DocFS: Replace rayon with threadpool and enable it for all targets)
 - rust-lang#109151 (Assert def-kind is correct for alias types)
 - rust-lang#109158 (error-msg: expand suggestion for `unused_def` lint)
 - rust-lang#109166 (make `define_opaque_types` fully explicit)
 - rust-lang#109171 (Some cleanups in our normalization logic)
 - rust-lang#109180 (Unequal → Not equal)
 - rust-lang#109185 (rustdoc: remove `std::` from primitive intra-doc link tooltips)
 - rust-lang#109192 (Mention UEFI target promotion in release notes for 1.67.0)

Failed merges:

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit 6cf2f47 into rust-lang:master Mar 16, 2023
@rustbot rustbot added this to the 1.70.0 milestone Mar 16, 2023
@GuillaumeGomez GuillaumeGomez deleted the rustdoc-windows-wait-for-write branch March 16, 2023 13:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@notriddle notriddle notriddle left review comments

Assignees

@notriddle notriddle

Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue.
Projects
None yet
Milestone
1.70.0
Development

Successfully merging this pull request may close these issues.

Rustdoc does not wait for DocFS writes to complete before exiting on Windows
5 participants
@GuillaumeGomez @rustbot @rust-timer @bors @notriddle