-
Notifications
You must be signed in to change notification settings - Fork 2.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fail gracefully on bad SabreDAG
construction
#10724
Conversation
One or more of the the following people are requested to review this:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great.
Is this the first bit of Rust-based unit testing we've got in accelerate
?
It is the first bit of testing (I didn't want to write a Python test that's testing an internal Rust interface), but weirdly it works completely fine locally for me and fails in CI. I'll have to look into the setup. |
On closer inspection of that error message, I'm guessing/hoping the problem is just that Azure's setup Python action doesn't add Python's dynamic libraries path to the default linker path, but if we get to that point from within a Python interpreter session then it's obviously already loaded into memory. |
d3d2eb8
to
64c1982
Compare
This is a private, internal Rust type, but it doesn't cost us anything (meaningful) to bounds-check the accessors and ensure we fail gracefully rather than panicking if we ever make a mistake in Python space. This more faithfully fulfills the return value of `SabreDAG::new`, which previously was an effectively infallible `Result`.
3fc2160
to
ab282f3
Compare
Alright, the issue in the end was that PyO3 was finding the correct place to link in the library and on Linux (or at least the CI machines) that's a dynamic library. However, for reasons that I don't fully understand, it doesn't write out the library path into the Since I don't 100% understand why PyO3 doesn't write out the |
Pull Request Test Coverage Report for Build 6025520902
💛 - Coveralls |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Thanks for enabling Rust-only testing 😄.
In retrospect, I'm fairly sure that PyO3 doesn't automatically write out the path of the dynamic Python library detected during the build into the rpath because that would couple the output executable to the directory structure of the machine that built it, which isn't useful during packaging as that's not a fixed point. So what I did of ensuring that the library path is set up correctly during the run is (I think) the correct thing to do. |
* Fail gracefully on bad `SabreDAG` construction This is a private, internal Rust type, but it doesn't cost us anything (meaningful) to bounds-check the accessors and ensure we fail gracefully rather than panicking if we ever make a mistake in Python space. This more faithfully fulfills the return value of `SabreDAG::new`, which previously was an effectively infallible `Result`. * Run `cargo fmt` (cherry picked from commit 1606ca3) # Conflicts: # Cargo.toml # crates/accelerate/src/sabre_layout.rs # crates/accelerate/src/sabre_swap/sabre_dag.rs
* Fail gracefully on bad `SabreDAG` construction This is a private, internal Rust type, but it doesn't cost us anything (meaningful) to bounds-check the accessors and ensure we fail gracefully rather than panicking if we ever make a mistake in Python space. This more faithfully fulfills the return value of `SabreDAG::new`, which previously was an effectively infallible `Result`. * Run `cargo fmt` (cherry picked from commit 1606ca3)
* Fail gracefully on bad `SabreDAG` construction This is a private, internal Rust type, but it doesn't cost us anything (meaningful) to bounds-check the accessors and ensure we fail gracefully rather than panicking if we ever make a mistake in Python space. This more faithfully fulfills the return value of `SabreDAG::new`, which previously was an effectively infallible `Result`. * Run `cargo fmt` (cherry picked from commit 1606ca3) Co-authored-by: Jake Lishman <[email protected]>
Summary
This is a private, internal Rust type, but it doesn't cost us anything (meaningful) to bounds-check the accessors and ensure we fail gracefully rather than panicking if we ever make a mistake in Python space. This more faithfully fulfills the return value of
SabreDAG::new
, which previously was an effectively infallibleResult
.Details and comments
Stems from #10712 (comment). We might not need to bother with this, but if we did, this is probably what it'd look like.
The extra test shim stuff is just because PyO3/Rust can't link statically against CPython for the test suite; usually
extension-module
s link dynamically against Python with the particular shared library getting linked during Python's initialisation of the module.