Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement REPL replayer for Windows #28608

Merged
merged 1 commit into from
Aug 14, 2018
Merged

Implement REPL replayer for Windows #28608

merged 1 commit into from
Aug 14, 2018

Conversation

KristofferC
Copy link
Member

@KristofferC KristofferC commented Aug 11, 2018

Review without whitespace: https://github.com/JuliaLang/julia/pull/28608/files?w=1

After some JuliaCon hacking with @Keno, we came up with something that replays the REPL script on Windows. The trick is to fool the spawned Julia process into believing it is being started from Cygwin by giving it a named pipe with the name satisfying a certain pattern. This makes us create TTYs for the 3 standard streams instead of e.g.IOStreams.

Since we are only faking being Cygwin we need to add a small patch to the Terminal REPL code to avoid it
trying to set some settings to the terminal that we don't care about when doing precompilation generation.

Note that this PR seems to increase the time of the precompile generation while, in fact, time has only been moved from loading the Pkg precompile statements from one place to another.

I also added a "overhead" measurement to the output:

Generating precompile statements... 1076 generated in  97.362301 seconds (overhead  59.582277 seconds)

This shows how much time we "waste" by dynamically collecting precompile statements compared to having a perfect file with precompile statements available. Since this is significant, it might be worth some extra logic to have a "build mode for developers" where the previous precompile file is tried to be used and only regenerate a new one if using the old one fails. This would be a non-default setting so build bots and CI would always regenerate from scratch.

One thing that would be good to figure out is that at the end when the REPL script has been executed, calling exit() on Windows does not shut down the process, so here we just forcefully kill it. According to @vtjnash, this would happen if we have Windows is waiting for us to finish reading from a stream, but it feels like the @async block that just keeps reading from master should deal with this...

@KristofferC KristofferC added the compiler:precompilation Precompilation of modules label Aug 11, 2018
@KristofferC
Copy link
Member Author

The buildbot binary for this commit is at https://julialangnightlies-s3.julialang.org/bin/winnt/x64/1.0/julia-b8b07a227b-win64.exe and performance is good.

@KristofferC KristofferC changed the title WIP: Implement REPL replayer for Windows Implement REPL replayer for Windows Aug 13, 2018
@KristofferC
Copy link
Member Author

I'll restart AV a couple of times to see that this is stable on Windows and then I plan to merge unless someone objects. Keeping the Pkg precompile stuff up to date is super annoying since there are some quite big refactorings in e.g. the Pkg REPL code.

@KristofferC KristofferC merged commit 3fe2d08 into master Aug 14, 2018
@KristofferC KristofferC deleted the kc/precompile_win branch August 14, 2018 10:02
bors bot added a commit to JuliaLang/Pkg.jl that referenced this pull request Aug 14, 2018
648: Remove precompile things from repo r=KristofferC a=KristofferC

(cherry picked from commit 3fe2d08c1546c41ade0459122905016527efa03f)

Can remove this now when JuliaLang/julia#28608 is fixed.

Co-authored-by: Kristoffer Carlsson <[email protected]>
KristofferC added a commit that referenced this pull request Aug 19, 2018
KristofferC added a commit that referenced this pull request Aug 19, 2018
@KristofferC KristofferC mentioned this pull request Aug 19, 2018
KristofferC added a commit that referenced this pull request Aug 19, 2018
KristofferC added a commit that referenced this pull request Sep 6, 2018
KristofferC added a commit that referenced this pull request Sep 8, 2018
KristofferC added a commit that referenced this pull request Feb 11, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler:precompilation Precompilation of modules
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant