Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use patched filelock with CLOEXEC #4787

Merged
merged 1 commit into from
Apr 30, 2019
Merged

Use patched filelock with CLOEXEC #4787

merged 1 commit into from
Apr 30, 2019

Conversation

snoyberg
Copy link
Contributor

Specifically, we're adding in this commit:

snoyberg/filelock@4f08049

Without this change, very often on large builds we end up in a situation
where:

  • Multiple threads are working in parallel
  • Thread A needs to work with the database
  • Thread A takes a file lock on the database
  • Thread B forks a child process, which inherits the database lock
  • Thread A releases the file lock
  • Thread C attempts to take the file lock, but is blocked by B's child

As a demonstration, lsof showed me this on my system:

COMMAND     PID    USER   FD   TYPE DEVICE SIZE/OFF       NODE NAME
stack     20216 michael   12w   REG    1,4        0 8613331527 stack.sqlite3.pantry-write-lock
Cabal-sim 22736 michael   12w   REG    1,4        0 8613331527 stack.sqlite3.pantry-write-lock
ghc       22800 michael   12w   REG    1,4        0 8613331527 stack.sqlite3.pantry-write-lock

With this change in place, I witnessed 0 cases of the file lock not
acquired message from Stack on a large build.

Specifically, we're adding in this commit:

snoyberg/filelock@4f08049

Without this change, very often on large builds we end up in a situation
where:

* Multiple threads are working in parallel
* Thread A needs to work with the database
* Thread A takes a file lock on the database
* Thread B forks a child process, which inherits the database lock
* Thread A releases the file lock
* Thread C attempts to take the file lock, but is blocked by B's child

As a demonstration, lsof showed me this on my system:

COMMAND     PID    USER   FD   TYPE DEVICE SIZE/OFF       NODE NAME
stack     20216 michael   12w   REG    1,4        0 8613331527 stack.sqlite3.pantry-write-lock
Cabal-sim 22736 michael   12w   REG    1,4        0 8613331527 stack.sqlite3.pantry-write-lock
ghc       22800 michael   12w   REG    1,4        0 8613331527 stack.sqlite3.pantry-write-lock

With this change in place, I witnessed 0 cases of the file lock not
acquired message from Stack on a large build.
@snoyberg snoyberg merged commit 595e9ea into master Apr 30, 2019
@snoyberg snoyberg deleted the cloexec-filelock branch April 30, 2019 16:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant