Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix segfault: capture rust heap references before yeilding to OCaml #6833

Merged
merged 3 commits into from
Nov 23, 2020

Conversation

mrmr1993
Copy link
Member

This PR fixes the segmentation fault reported in #6824.

The 'header' (ie. data pointer + length) of the input vectors were stored in the OCaml heap. This was fine when they were run synchronously, but the async version allows the OCaml GC to run while the function is also running. A badly timed GC can move the header and leave rust's reference pointing at arbitrary data, and dereferencing the vector's data pointer may then result in a segfault.

This PR changes the behaviour to store only a (rust heap) pointer in the OCaml heap. As long as we capture the pointer before yielding to OCaml, we know that we hold a reference to the intended data.

Checklist:

  • Document code purpose, how to use it
    • Mention expected invariants, implicit constraints
  • Tests were added for the new behavior
    • Document test purpose, significance of failures
    • Test names should reflect their purpose
  • All tests pass (CI will check this if you didn't)
  • Serialized types are in stable-versioned modules
  • Does this close issues? List them:

@mrmr1993 mrmr1993 added the ci-build-me Add this label to trigger a circle+buildkite build for this branch label Nov 23, 2020
@mrmr1993 mrmr1993 requested a review from a team as a code owner November 23, 2020 21:01
@mergify mergify bot merged commit 38c10db into develop Nov 23, 2020
@mergify mergify bot deleted the feature/fix-prover-segfault branch November 23, 2020 23:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci-build-me Add this label to trigger a circle+buildkite build for this branch ready-to-merge-into-develop
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants