-
-
Notifications
You must be signed in to change notification settings - Fork 646
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Expose intrinsics as decorated Python functions instead #20874
Conversation
487d30d
to
1b7c9f1
Compare
1b7c9f1
to
7dbb37d
Compare
7dbb37d
to
33ff599
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice. I think this looks good, but it's at the limit of my understanding.
Playing back, it looks like the PR has a few different moving parts:
- setup the
PyGeneratorResponseNativeCall
type to store a Rust future in a Python object - support extracting and awaiting that future in
Task::generate
(plusGeneratorResponse::NativeCall
andgenerator_send
etc.) - change how the intrinsics are defined and registered in Rust, which includes removing the
Intrinsic
andIntrinsics
types and theRule::Intrinsic
handling - rewrite all the intrinsics to:
- query the thread-local task context since they no longer get it as an arg
- wrap their internal future in
PyGeneratorResponseNativeCall
(which requires a few other adjustments, due to type inference, etc.?)
- define some normal
@rule
s as wrappers
Then, because we've fully switched over in the definitions, I think this code is being actively exercised by the full test suite. (I'm raising this because: if this was instead a "preparation" that wasn't being used, we might discover a whole bunch of issues when we start using them. But, because they're being used directly by the old test-suite, that's a strong indication that things fit together and there's no behaviour change.)
Have I got that correct (or at least correct enough)?
Approved assuming I have.
That is all exactly right. Thank you @huonw ! ... I apologize for such a skimpy PR description. |
The switch in #20874 to exposing intrinsics as python functions meant that running an InteractiveProcess now occurs under a separate engine Task than the goal that invokes it. This introduced a bug where the `restartable` property was applied to the InteractiveProcess intrinsic's Task, and not, as previously, to the goal's Task. This change ensures that the restartability applies correctly, via new rule helpers. Fixes #21243.
The switch in pantsbuild#20874 to exposing intrinsics as python functions meant that running an InteractiveProcess now occurs under a separate engine Task than the goal that invokes it. This introduced a bug where the `restartable` property was applied to the InteractiveProcess intrinsic's Task, and not, as previously, to the goal's Task. This change ensures that the restartability applies correctly, via new rule helpers. Fixes pantsbuild#21243.
The switch in pantsbuild#20874 to exposing intrinsics as python functions meant that running an InteractiveProcess now occurs under a separate engine Task than the goal that invokes it. This introduced a bug where the `restartable` property was applied to the InteractiveProcess intrinsic's Task, and not, as previously, to the goal's Task. This change ensures that the restartability applies correctly, via new rule helpers. Fixes pantsbuild#21243.
) (#21319) The switch in #20874 to exposing intrinsics as python functions meant that running an InteractiveProcess now occurs under a separate engine Task than the goal that invokes it. This introduced a bug where the `restartable` property was applied to the InteractiveProcess intrinsic's Task, and not, as previously, to the goal's Task. This change ensures that the restartability applies correctly, via new rule helpers. Fixes #21243.
Remove
intrinsics
facility from the engine, and instead expose intrinsics inintrinsics.py
via a new native call response for generators. This solves an issue in #19730 that there were no obvious@rule
functions to call for intrinsics.The native call response wraps a Rust future, which we trampoline to execute on the Rust runtime, and then return a value to the generator. It should work generically for any use case where we would like
@rule
code to await native code, which opens the door to fixing #11329 by makingWorkspace.write_digest
async.