Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port QPY serialization to Rust #13131

Open
mtreinish opened this issue Sep 11, 2024 · 0 comments
Open

Port QPY serialization to Rust #13131

mtreinish opened this issue Sep 11, 2024 · 0 comments
Labels
mod: qpy Related to QPY serialization Rust This PR or issue is related to Rust code in the repository type: feature request New feature or request
Milestone

Comments

@mtreinish
Copy link
Member

What should we add?

Now that we have most of the circuit model in Rust it should be possible to port the QPY into Rust. We have enough of the data model in Rust now to iterate over the circuit data in rust and generate the payload as we go. There are still places where there will be python interaction, mainly around custom gates and pulse/calibrations. But we can just call out to Python functions as necessary to handle that.

The actual QPY format is packed structs with network endianess and it is pretty natural to express this in rust. The payloads defined in: https://github.com/Qiskit/qiskit/blob/main/qiskit/qpy/formats.py would be expressed as something like:

#[repr(packed)]
struct FileHeader {
    preface: [c_char; 6],
    qpy_version: c_uchar,
    major_version: c_uchar,
    minor_version: c_uchar,
    patch_version: c_uchar,
    num_programs: c_ulonglong
}

in rust.

Although we'll have to use something like the byteorder crate: https://docs.rs/byteorder/latest/byteorder/ to specify the endianess when reading or writing the QPY payload. Although it might be easier to use something like serde to handle the serialization for us and then just work with rust native structs: https://serde.rs/data-format.html

@mtreinish mtreinish added type: feature request New feature or request mod: qpy Related to QPY serialization Rust This PR or issue is related to Rust code in the repository labels Sep 11, 2024
@mtreinish mtreinish added this to the 1.3.0 milestone Sep 11, 2024
mtreinish added a commit to mtreinish/qiskit-core that referenced this issue Oct 22, 2024
With the release of symengine 0.13.0 we discovered a version dependence
on the payload format used for serializing symengine expressions. This
was worked around in Qiskit#13251 but this is not a sustainable solution and
only works for symengine 0.11.0 and 0.13.0 (there was no 0.12.0). While
there was always the option to use sympy to serialize the underlying
symbolic expression (there is a `use_symengine` flag on `qpy.dumps` you
can set to `False` to do this) the sympy serialzation has several
tradeoffs most importantly is much higher runtime overhead. To solve
the issue moving forward a qiskit native representation of the parameter
expression object is necessary for serialization.

This commit bumps the QPY format version to 13 and adds a new
serialization format for ParameterExpression objects. This new format
is a serialization of the API calls made to ParameterExpression that
resulted in the creation of the underlying object. To facilitate this
the ParameterExpression class is expanded to store an internal "replay"
record of the API calls used to construct the ParameterExpression
object. This internal list is what gets serialized by QPY and then on
deserialization the "replay" is replayed to reconstruct the expression
object. This is a different approach to the previous QPY representations
of the ParameterExpression objects which instead represented the internal
state stored in the ParameterExpression object with the symbolic
expression from symengine (or a sympy copy of the expression). Doing
this directly in Qiskit isn't viable though because symengine's internal
expression tree is not exposed to Python directly. There isn't any
method (private or public) to walk the expression tree to construct
a serialization format based off of it. Converting symengine to a sympy
expression and then using sympy's API to walk the expression tree is a
possibility but that would tie us to sympy which would be problematic
for Qiskit#13267 and Qiskit#13131, have significant runtime overhead, and it would
be just easier to rely on sympy's native serialization tools.

The tradeoff with this approach is that it does increase the memory
overhead of the `ParameterExpression` class because for each element
in the expression we have to store a record of it. Depending on the
depth of the expression tree this also could be a lot larger than
symengine's internal representation as we store the raw api calls made
to create the ParameterExpression but symengine is likely simplifying
it's internal representation as it builds it out. But I personally think
this tradeoff is worthwhile as it ties the serialization format to the
Qiskit objects instead of relying on a 3rd party library. This also
gives us the flexibility of changing the internal symbolic expression
library internally in the future if we decide to stop using symengine
at any point.

Fixes Qiskit#13252
@raynelfss raynelfss modified the milestones: 1.3.0, 2.0.0 Oct 24, 2024
github-merge-queue bot pushed a commit that referenced this issue Nov 6, 2024
* Add Qiskit native QPY ParameterExpression serialization

With the release of symengine 0.13.0 we discovered a version dependence
on the payload format used for serializing symengine expressions. This
was worked around in #13251 but this is not a sustainable solution and
only works for symengine 0.11.0 and 0.13.0 (there was no 0.12.0). While
there was always the option to use sympy to serialize the underlying
symbolic expression (there is a `use_symengine` flag on `qpy.dumps` you
can set to `False` to do this) the sympy serialzation has several
tradeoffs most importantly is much higher runtime overhead. To solve
the issue moving forward a qiskit native representation of the parameter
expression object is necessary for serialization.

This commit bumps the QPY format version to 13 and adds a new
serialization format for ParameterExpression objects. This new format
is a serialization of the API calls made to ParameterExpression that
resulted in the creation of the underlying object. To facilitate this
the ParameterExpression class is expanded to store an internal "replay"
record of the API calls used to construct the ParameterExpression
object. This internal list is what gets serialized by QPY and then on
deserialization the "replay" is replayed to reconstruct the expression
object. This is a different approach to the previous QPY representations
of the ParameterExpression objects which instead represented the internal
state stored in the ParameterExpression object with the symbolic
expression from symengine (or a sympy copy of the expression). Doing
this directly in Qiskit isn't viable though because symengine's internal
expression tree is not exposed to Python directly. There isn't any
method (private or public) to walk the expression tree to construct
a serialization format based off of it. Converting symengine to a sympy
expression and then using sympy's API to walk the expression tree is a
possibility but that would tie us to sympy which would be problematic
for #13267 and #13131, have significant runtime overhead, and it would
be just easier to rely on sympy's native serialization tools.

The tradeoff with this approach is that it does increase the memory
overhead of the `ParameterExpression` class because for each element
in the expression we have to store a record of it. Depending on the
depth of the expression tree this also could be a lot larger than
symengine's internal representation as we store the raw api calls made
to create the ParameterExpression but symengine is likely simplifying
it's internal representation as it builds it out. But I personally think
this tradeoff is worthwhile as it ties the serialization format to the
Qiskit objects instead of relying on a 3rd party library. This also
gives us the flexibility of changing the internal symbolic expression
library internally in the future if we decide to stop using symengine
at any point.

Fixes #13252

* Remove stray comment

* Add format documentation

* Add release note

* Add test and fix some issues with recursive expressions

* Add int type for operands

* Add dedicated subs test

* Pivot to stack based postfix/rpn deserialization

This commit changes how the deserialization works to use a postfix
stack based approach. Operands are push on the stack and then popped off
based on the operation being run. The result of the operation is then
pushed on the stack. This handles nested objects much more cleanly than
the recursion based approach because we just keep pushing on the stack
instead of recursing, making the accounting much simpler. After the
expression payload is finished being processed there will be a single
value on the stack and that is returned as the final expression.

* Apply suggestions from code review

Co-authored-by: Elena Peña Tapia <[email protected]>

* Change DERIV to GRAD

* Change side kwarg to r_side

* Change all the v4s to v13s

* Correctly handle non-commutative operations

This commit fixes a bug with handling the operand order of subtraction,
division, and exponentiation. These operations are not commutative but
the qpy deserialization code was treating them as such. So in cases
where the argument order was reversed qpy was trying to flip the
operands around for code simplicity and this would result in incorrect
behavior. This commit fixes this by adding explicit op codes for the
reversed sub, div, and pow and preserving the operand order correctly
in these cases.

* Fix lint

---------

Co-authored-by: Elena Peña Tapia <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
mod: qpy Related to QPY serialization Rust This PR or issue is related to Rust code in the repository type: feature request New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants