Add conversion from Python int to either long or BigInt properly

[cmpute]

Sometimes it's convenient to have a conversion to proper type in rust, i.e. if the Python int is larger than 2^64, then convert to BigInt, if smaller, then convert to native i64. This could reduce some overhead if you want to be able to deal with big number and with small number efficiently at the same time.

The following code is what I figured out and it's working, but it might be better if it's integrated into the support for num_bigint

use pyo3::{ffi, AsPyPointer};
use pyo3::types::{PyLong, PyType};
use std::os::raw::c_int;

enum IntTypes {
    Small(i64),
    Big(BigInt),
}

fn parse_int(ob: &PyAny) -> PyResult<IntTypes> {
    let py = ob.py();
    unsafe {
        let num: Py<PyLong> = Py::from_owned_ptr_or_err(py, ffi::PyNumber_Index(ob.as_ptr()))?;
        let mut overflow: c_int = 0;
        let v = ffi::PyLong_AsLongLongAndOverflow(num.as_ptr(), &mut overflow);
        if v == -1 && PyErr::occurred(py) {
            Err(PyErr::fetch(py))
        } else if overflow != 0 {
            let num: BigInt = ob.extract()?;
            Ok(IntTypes::Big(num))
        } else {
            Ok(IntTypes::Small(v))
        }
    }
    // then use ToPyObject
}

[davidhewitt]

Thank you for the proposal. I would recommend instead solving this using #[derive(FromPyObject)] - this will generate an implementation which will first try to extract a Small, and if that fails, try a Big.

#[derive(FromPyObject)]
enum IntTypes {
    Small(i64),
    Big(BigInt),
}

As the above is straightfoward (although not super well documented, sorry), I don't think there's need to add a more specific function for this to PyO3.

[cmpute]

@davidhewitt Thanks for the information! I indeed found that the documentation for enums is quite lacky. This solution is quite neat!

However I think using my code snippet above will reduce the overhead for conversion (without depending on Python throwing errors), although it might not be critical in most cases.

[davidhewitt]

I agree that your snippet will be more efficient, however I think a small enough optimisation that at this point I don't feel a strong need to add this to PyO3. If this proves to be a popular request, I'm happy to change my mind later.

FYI - If you tweaked your parse_int function to be an impl FromPyObject for IntTypes then you'll be able to use IntTypes directly in #[pyfunction] and #[pymethods] arguments.

[cmpute]

Just a quick note, I found about 0.2 s improvement on my laptop with 10000 evaluations of a function taking IntTypes as input.

[davidhewitt]

After #2068 this performance gap should hopefully be a lot smaller 🤞

[cmpute]

Cool! Thanks for the efforts!