Passing numpy arrays without memory duplication

[ebranlard]

I'm trying to figure out a way to pass numpy arrays directly to a cdll (wrapped with pyclibrary), without the user having to manually do the type conversion, and without having any memory duplication.

I was wondering if you had any thoughts on that and if your framework could handle it smoothly. I'm aware of the build_array function provided by c_library, but I believe it would be more transparent for the users if they can use numpy arrays directly. I believe it's fine to impose a dtype to the users though.

Test case
Here's my way of doing it so far, I have the following fortran dll calls, that adds vector x and y and returns a vector add, all of them of size n, with the following signature:

void testlib_addvec(double* x, double* y, double* add, int* n);

On the python side, I create number arrays, that I then have to convert to c_double arrays using a small function to_c, and then I pass it to the library (which is a CLibrary object):

import numpy as np
from pyclibrary.c_library import CLibrary
lib=CLibrary(libfile,[headerfile])

def to_c(var):
        n=var.shape[0]
        var_c=(c_double*n)(0)
        var_c[0:n]=var[0:n]
        return var_c
n=3
x    = np.ones(n) 
y    = np.ones(n)+2
z    = np.zeros(n)
x_c  = to_c(x)
y_c  = to_c(y)
z_c  = to_c(z) 
res=lib.testlib_addvec(x_c, y_c, z_c, c_int(n))
print('x:', res['x'][:])
print('y:', res['y'][:])
print('z:', res['add'][:])

Questions

Is there anyways we can make this call more "transparent" for the user, without the need for the explicit type conversion?

Can we avoid any memory duplication by asking the user to create numpy arrays of a given type?

I'm guessing one solution could be to handle the conversion within pyclibrary directly.

I tried with x_c = x.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), which seems to work for the input arguments, but then gives a MemoryError if I try to access res['x'] and it won't work for the returned value add. Since it's a fortran dll, everything is passed by reference, so there is not real notion of input and outputs though.

[MatthieuDartiailh]

In this case it perhaps does not matter but you should be careful to pass F ordered arrays since this likely what your library expect.
I know I have used the ctypes attributes in the past. I will check how. Do not hesitate to ping me.