Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

setupEnv: Run some processes concurrently #2346

Merged
merged 2 commits into from
Aug 6, 2016

Conversation

sjakobi
Copy link
Member

@sjakobi sjakobi commented Jul 6, 2016

This should make most commands slightly faster.
On my machine, stack path takes ~20% less time than before, a no-op
stack build in a single-package project ~15%.

@sjakobi
Copy link
Member Author

sjakobi commented Jul 6, 2016

So, what I actually don't know, is whether this safe, especially querying ghc-pkg asynchronously.

Can someone with more experience with asynchronous programming take a look?

BTW here's some debug output from a stack version on top of my better-process-logging branch for #2327:

~ $ stack-bpl-async -v path --dist-dir --global-pkg-db
Version 1.1.3, Git revision 7934d6ce5eb9b841a6f248b427e2e8812fd69845 (dirty) (3753 commits) x86_64 hpack-0.14.0
2016-07-06 19:40:57.493231: [debug] Checking for project config at: /home/simon/stack.yaml @(Kkm9qBx54bb6fPiRXOOC4T:Stack.Config src/Stack/Config.hs:823:9)
2016-07-06 19:40:57.493629: [debug] Checking for project config at: /home/stack.yaml @(Kkm9qBx54bb6fPiRXOOC4T:Stack.Config src/Stack/Config.hs:823:9)
2016-07-06 19:40:57.493763: [debug] Checking for project config at: /stack.yaml @(Kkm9qBx54bb6fPiRXOOC4T:Stack.Config src/Stack/Config.hs:823:9)
2016-07-06 19:40:57.493909: [debug] No project config file found, using defaults. @(Kkm9qBx54bb6fPiRXOOC4T:Stack.Config src/Stack/Config.hs:845:13)
2016-07-06 19:40:57.496410: [debug] Checking whether stack was built with libgmp4 @(Kkm9qBx54bb6fPiRXOOC4T:Stack.Config src/Stack/Config.hs:327:5)
2016-07-06 19:40:57.496909: [debug] Run process: /usr/bin/ldd /home/simon/.local/bin/stack-bpl-async @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
2016-07-06 19:40:57.512142: [debug] Process finished in 15 ms: /usr/bin/ldd /home/simon/.local/bin/stack-bpl-async @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
2016-07-06 19:40:57.512561: [debug] Stack was not built with libgmp4 @(Kkm9qBx54bb6fPiRXOOC4T:Stack.Config src/Stack/Config.hs:331:14)
2016-07-06 19:40:57.512688: [info] Run from outside a project, using implicit global project config @(Kkm9qBx54bb6fPiRXOOC4T:Stack.Config src/Stack/Config.hs:462:13)
2016-07-06 19:40:57.514316: [info] Using resolver: nightly-2016-06-21 from implicit global project's config file: /home/simon/.stack/global-project/stack.yaml @(Kkm9qBx54bb6fPiRXOOC4T:Stack.Config src/Stack/Config.hs:476:32)
2016-07-06 19:40:57.514602: [debug] Trying to decode /home/simon/.stack/build-plan-cache/x86_64-linux/nightly-2016-06-21.cache @(Kkm9qBx54bb6fPiRXOOC4T:Data.Store.VersionTagged src/Data/Store/VersionTagged.hs:51:5)
2016-07-06 19:40:57.527888: [debug] Success decoding /home/simon/.stack/build-plan-cache/x86_64-linux/nightly-2016-06-21.cache @(Kkm9qBx54bb6fPiRXOOC4T:Data.Store.VersionTagged src/Data/Store/VersionTagged.hs:60:13)
2016-07-06 19:40:57.534286: [debug] Getting system compiler version @(Kkm9qBx54bb6fPiRXOOC4T:Stack.Setup src/Stack/Setup.hs:345:17)
2016-07-06 19:40:57.534606: [debug] Run process: /usr/bin/ghc --info @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
2016-07-06 19:40:57.560165: [debug] Process finished in 25 ms: /usr/bin/ghc --info @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
2016-07-06 19:40:57.563992: [debug] Asking GHC for its version @(Kkm9qBx54bb6fPiRXOOC4T:Stack.Setup.Installed src/Stack/Setup/Installed.hs:97:13)
2016-07-06 19:40:57.564233: [debug] Getting Cabal package version @(Kkm9qBx54bb6fPiRXOOC4T:Stack.GhcPkg src/Stack/GhcPkg.hs:165:5)
2016-07-06 19:40:57.564356: [debug] Getting global package database location @(Kkm9qBx54bb6fPiRXOOC4T:Stack.GhcPkg src/Stack/GhcPkg.hs:48:5)
2016-07-06 19:40:57.564437: [debug] Run process: /home/simon/.stack/programs/x86_64-linux/ghc-8.0.1/bin/ghc --numeric-version @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
2016-07-06 19:40:57.565233: [debug] Run process: /home/simon/.stack/programs/x86_64-linux/ghc-8.0.1/bin/ghc-pkg --no-user-package-db field --simple-output Cabal version @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
2016-07-06 19:40:57.565852: [debug] Run process: /home/simon/.stack/programs/x86_64-linux/ghc-8.0.1/bin/ghc-pkg --no-user-package-db list --global @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
2016-07-06 19:40:57.602662: [debug] Process finished in 37 ms: /home/simon/.stack/programs/x86_64-linux/ghc-8.0.1/bin/ghc-pkg --no-user-package-db field --simple-output Cabal version @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
2016-07-06 19:40:57.606331: [debug] Process finished in 34 ms: /home/simon/.stack/programs/x86_64-linux/ghc-8.0.1/bin/ghc-pkg --no-user-package-db list --global @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
2016-07-06 19:40:57.607750: [debug] Process finished in 43 ms: /home/simon/.stack/programs/x86_64-linux/ghc-8.0.1/bin/ghc --numeric-version @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
2016-07-06 19:40:57.607943: [debug] Resolving package entries @(Kkm9qBx54bb6fPiRXOOC4T:Stack.Setup src/Stack/Setup.hs:226:5)
2016-07-06 19:40:57.608657: [debug] Getting global package database location @(Kkm9qBx54bb6fPiRXOOC4T:Stack.GhcPkg src/Stack/GhcPkg.hs:48:5)
2016-07-06 19:40:57.608962: [debug] Run process: /home/simon/.stack/programs/x86_64-linux/ghc-8.0.1/bin/ghc-pkg --no-user-package-db list --global @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
2016-07-06 19:40:57.631108: [debug] Process finished in 21 ms: /home/simon/.stack/programs/x86_64-linux/ghc-8.0.1/bin/ghc-pkg --no-user-package-db list --global @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
global-pkg-db: /home/simon/.stack/programs/x86_64-linux/ghc-8.0.1/lib/ghc-8.0.1/package.conf.d
dist-dir: .stack-work/dist/x86_64-linux/Cabal-1.24.0.0

This should make most commands slightly faster.
On my machine, `stack path` takes ~20% less time than before, a no-op
`stack build` in a single-package project ~15%.
@sjakobi
Copy link
Member Author

sjakobi commented Jul 21, 2016

So, what I actually don't know, is whether this safe, especially querying ghc-pkg asynchronously.

Now, I figure there's no reason why ghc-pkg wouldn't be able to perform two reads concurrently.

@mgsloan
Copy link
Contributor

mgsloan commented Aug 6, 2016

Makes sense!

@mgsloan mgsloan merged commit 099ca78 into commercialhaskell:master Aug 6, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants