RFC: Reverse Array dims in no-copy conversion from Julia to Numpy

[kmsquire]

I've been working with image data and PyPlot.imshow, and finding that I have to munge the data more than I would like in order to get it to display:

PyPlot.imshow(permutedims(img.data, (3,2,1))

Digging into this, I found that reversing the dims and strides in the PyObject constructor let me display the image (more) directly with PyPlot.imshow(img.data).

Now it's likely that this change would cause issues with other arrays being passed to Numpy, so I don't expect this to be merged. But I'm wondering if a version which reverses the dims/strides can be made available somehow?

[stevengj]

The PyObject constructor tells Python the correct ordering of the dimensions; it sounds like you just don't like the behavior of matplotlib's imshow?

[kmsquire]

I guess. Here's some more context: when I read a frame from a video, for example, I'll get a 3x960x540 array. Passing this directly to imshow doesn't work:

julia> using AV

julia> f = AV.open("/home/kevin/Videos/chunk_0.mp4")
AVCapture("/home/kevin/Videos/chunk_0.mp4", ...)

julia> img = read(f)
3x960x540 Array{Uint8,3}:
[:, :, 1] =
...

julia> PyPlot.imshow(img)
ERROR: PyError (PyObject_Call) <type 'exceptions.TypeError'>
TypeError('Invalid dimensions for image data',)
  File "/usr/lib/pymodules/python2.7/matplotlib/pyplot.py", line 2892, in imshow
    imlim=imlim, resample=resample, url=url, **kwargs)
  File "/usr/lib/pymodules/python2.7/matplotlib/axes.py", line 7300, in imshow
    im.set_data(X)
  File "/usr/lib/pymodules/python2.7/matplotlib/image.py", line 429, in set_data
    raise TypeError("Invalid dimensions for image data")

 in pyerr_check at /home/kevin/.julia/v0.3/PyCall/src/exception.jl:58
 in pycall at /home/kevin/.julia/v0.3/PyCall/src/PyCall.jl:85
 in imshow at /home/kevin/.julia/v0.3/PyPlot/src/PyPlot.jl:250

julia> size(img)
(3,960,540)

julia> strides(img)
(1,3,2880)

I find that reordered dimensions to (540,960,3) and strides to (2880,3,1) (which this PR does) lets matplotlib's imshow interpret the array correctly as an image, without modifying the data. This makes sense, as the image is already row-major order.

So, one thought I had was to have a parameter to the PyObject constructor allowing reordering the dimensions, and then to allow PyPlot to use this parameter when calling imshow. (Using an Image type rather than a raw array would actually make the color dimension and orientation easier to ascertain.)

Thoughts? Is there a better way to deal with this directly in PyPlot?

[stevengj]

I see. This seems one of those arbitrary limitations of matplotlib that I keep running into, and I usually try to work around it to make the Julia interface more flexible.

I think it would be reasonable to write an imshow{T<:Number}(x::AbstractArray{T,3}; kws...) = ... method in PyPlot that checks whether the first dimension or the last dimension of x is 3 and reorders the dimensions if possible.

But you're right that this looks like it would ideally require some support in PyCall: there should be a low-level variant of the PyObject(x) constructor that allows you to re-interpret a Julia array x as a row-major array with reversed dimensions (without copying the data). Would be easier if Julia had a row-major array type, of course (JuliaLang/LinearAlgebra.jl#84).

[kmsquire]

I've updated this PR so that the PyObject constructor looks like

function PyObject{T<:NPY_TYPES}(a::StridedArray{T}, revdims::Bool=false)

Let me know if this works, or if you would prefer a different interface.

I also just submitted JuliaPy/PyPlot.jl#73, which uses this constructor for imshow when the color dimension appears first. The combination of patches works for me.

That PR shouldn't be merged until this one is tagged.

[kmsquire]

That PR shouldn't be merged until this one is tagged.

Well, I guess both could remain untagged as long as REQUIRES is updated when they do get tagged...

[stevengj]

Hi @kmsquire, sorry I lost track of this. This looks good to merge given a rebase and a test.

[kmsquire]

Me too. ;-) Rebased.

[kmsquire]

I'll add a test in a little while.

[stevengj]

I would say rather than "Julia arrays are in column-major order, but in some cases it is useful to pass them to Python as row-major arrays simply by reversing the dimensions. For example, although NumPy works with both row-major and column-major data, some Python libraries like OpenCV seem to require row-major data (the default in NumPy). In such cases, use PyObject(array, true)."

And then add a similar comment to the README.md file.

[coveralls]

Coverage increased (+0.2%) to 59.84% when pulling 827d6c2 on kmsquire:reverse_dims into 2ead45a on stevengj:master.

[stevengj]

Ping, any chance of a test? (The AppVeyor failure looks like an unrelated network problem.)

[stevengj]

Probably just something like

if PyCall.npy_initialized
    let A = rand(2,3,4), o = PyObject(A, true), A2 = convert(PyAny, o)
        @test A == permutedims(A2, [3,2,1])
    end
end

[dmrd]

This should not silently fall back on failure to array2py(a) if revdims=true. It produces inconsistent behavior. Is there some way array2py could take revdims as an extra arg as well?

[stevengj]

Good catch. For now, it can throw an error if revdims=true but in the future array2py could implement this too.

[dmrd]

Reproduction: https://gist.github.com/dmrd/b47c89c045279973258f997eeff1e863

Can we add BitArray--> Array{uint8} conversion as a default? Numpy doesn't seem to do bit arrays.

[stevengj]

@dmrd, a converter for BitArray, probably to Array{Bool}, is a good idea, but should be filed as a separate issue or PR. The patch could probably be as simple as a line PyObject(a::BitArray) = PyObject(Array{Bool}(a)) plus a test case.

[stevengj]

Closing in favor of JuliaLang/julia#265

[kmsquire]

Thanks, @StevenG, and sorry not to be more responsive.

[stevengj]

No problem, thanks for doing most of the work on this!