Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

world counter problem calling Julia functions from inside cfunctions (since pull #17057) #19790

Closed
luchr opened this issue Dec 31, 2016 · 8 comments

Comments

@luchr
Copy link

luchr commented Dec 31, 2016

I cooked up a minimal example (a modified example from the julia docs) in order to show my problem:

func_dict = Dict{Int,Function}()

function mycustomsort{T}(a::T, b::T)
  return func_dict[1](a,b)
end

# Last chance to define mycompare here(!!)

const mycustomsort_c = cfunction(mycustomsort, Cint, (Ref{Cdouble}, Ref{Cdouble}) )

# Too late; the world has changed!
function mycompare{T}(a::T, b::T)
  return convert(Cint, a<b ? -1 : a>b ? +1 : 0)::Cint
end

func_dict[1] = mycompare

A = [1.0,5,2,3]

ccall(:qsort, Void, (Ptr{Cdouble}, Csize_t, Csize_t, Ptr{Void}), 
        A, length(A), sizeof(eltype(A)), mycustomsort_c)
println(A)

One gets the output:

ERROR: LoadError: MethodError: no method matching mycompare(::Float64, ::Float64)
The applicable method may be too new: running in world age 20393, while current world is 20394.

Yes in the (constructed) example above, one can use the workaround to define mycompare before mycustomsort_c. But in real world examples this is not always possible.

Real world example: I'm the author of the ODEInterface package (available for Julia v0.4, v0.5, and trunk before 23.12.2016). There the idea is, that the user can write julia functions (e.g. the right-hand side of an ODE) and give this julia-functions to the ODEInterface. Inside the ODEInterface Fortran codes are called. These Fortran-Codes call Callback-functions that (in my case) are cfunctions of the ODEInterface that in turn try to call the user-given Julia function. Since Pull #17057 this is not possible anymore, because the ODEInterface (and hence the cfunctions) are typically loaded/compiled before the user defines the problems he/she wants to solve.

Any idea how I can solve this problem?

Julia Version 0.6.0-dev.1704
Commit 0a0c41c* (2016-12-27 16:34 UTC)
Platform Info:
  OS: Linux (x86_64-redhat-linux)
  CPU: Intel(R) Core(TM) i7-4700MQ CPU @ 2.40GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.7.1 (ORCJIT, haswell)
@tkelman
Copy link
Contributor

tkelman commented Dec 31, 2016

duplicate of #19774 ?

@stevengj
Copy link
Member

Yes, this is a duplicate. For now, the workaround is to use eval, but with #19784 I define an invokelatest function that makes things a bit easier.

@yuyichao
Copy link
Contributor

Note that for this usecase, you should not cache the cfunction yourself.

@luchr
Copy link
Author

luchr commented Dec 31, 2016

Sorry, yuyichao, I don't understand your comment (but I want to understand it):

  1. In the ODEInterface I have to call compiled Fortran codes and supply callback-functions. I thought the only way to do this is to call cfunction, save the result in a global const and use this (code-)pointer as callback-functions/pointers for the Fortran code (as input).
  2. What I am "caching" (in a dict) are the julia functions given by the users. There no cfunction-calls are involved.

Are there better ways to do this?

@vtjnash
Copy link
Member

vtjnash commented Dec 31, 2016

Yichao is suggesting that it is unnecessary to cache the cfunction in a global

@yuyichao
Copy link
Contributor

Right. I've just realized that typeinf is not handling that correctly so I'm trying to fix that.

@yuyichao
Copy link
Contributor

yuyichao commented Jan 1, 2017

See JuliaMath/Cubature.jl#23 for what I think packages with a similar usage should do. The integrands function in that PR should be compiled to returning one of 8 constant pointers and it should be automatically recompiled when new functions are defined.

@luchr
Copy link
Author

luchr commented Jan 1, 2017

I think I've understood your idea now and the impact (now julia knows at compile-time what to do for d.integrand_func in the callback, because there will be one cfunction-compilation per different integrand). Wow.
If I manage to do the same in the ODEInterface package, this will be a huge improvement.

Thank you very much

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants