-
Notifications
You must be signed in to change notification settings - Fork 804
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Simplify ReleasePool, remove parking_lot dependency. #899
Conversation
@davidhewitt Since you wrote this part, can you review this? I'm curious if there's any specific reason that I'm missing for why
This PR changes those things, but let me know if i'm missing something here. (In that case we should probably add a comment about it.) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@m-ou-se thanks so much for all of the cleanups you have been pushing for pyo3 today!
The original two vector code existed long before me, though I was the most recent person to touch it up 😄 . My theory is that there were two reasons for this:
- While the
pointers_to_drop
mutex is locked, other Rust threads which might be droppingPyObject
s orPy<T>
might be blocked from making progress. In all honesty it seems like a very rare case that other threads would be doing anything withPyObject
s at the same time, as no other thread would have the GIL. - When the
Py_DECREF
call is made, arbitrary drop code can run, which might lead to furtherPyObject
being dropped. It used to be the case before a recent change of mine that thesePyObject
would be put straight into the release pool. If the release poolMutex
was still locked then the thread would deadlock.
This second reason in particular forced the need for a two-vector solution, and would only hold the mutex while swapping between them so that the Py_DECREF
calls could then be made without the mutex locked. (Using *mut Vec
made it cheap to swap between the two vectors.)
But as I recently changed PyObject
s to not use the release pool if the gil is held (see register_pointer
function), this deadlock problem no longer applies. (Unless people are using the unsafe function assume_gil_aquired
when pyo3 doesn't actually have any GILPool
set. And they'll be getting much more noticeable crashes if they're doing that!)
As for set_len(0)
- that's just something I missed in refactoring, using .clear()
looks very sensible to me.
So, in summary, thanks for cleaning this up - I'm in favour of this change 👍
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you!
As @davidhewitt says, we need this simplification as the situtation changed.
Replaces the parking_lot Mutex by a simple AtomicBool spinlock, and simplifies the ReleasePool to contain one Vec instead of pointers to two Vecs.
Thanks for the update! |
This pull request should have been changed to avoid spinlocks, see #916 |
Replaces the parking_lot Mutex by a simple AtomicBool spinlock, and simplifies the ReleasePool to contain one Vec instead of pointers to two Vecs.