import module on worker process different path

[augustinyiptong]

Hello,

I have a master process on a windows machine and a worker process on a linux machine. The paths to the modules directory are different.

This doesnt work:

@everywhere import Compat 

WARNING: deserialization checks failed while attempting to load cache from C:\Users\augustin.yiptong\.julia\lib\v0.5\Compat.ji
LoadError: On worker 2:
SystemError: opening file /usr/bin/C:\Users\augustin.yiptong\.julia\v0.5\Compat\src\Compat.jl: Invalid argument
 [inlined code] from error.jl:34
 in open at iostream.jl:90
...

I can get around this by copying code from loading.jl, hard coding the correct path, and running it on worker process:

content_r = @spawn open(readbytes, "/home/ubuntu/.julia/lib/v0.5/Compat.ji")
r = @spawn ccall(:jl_restore_incremental_from_buf, Any, (Ptr{UInt8},Int), fetch(content_r), sizeof(fetch(content_r)))

Importing modules in the linux box REPL works fine.

Should the imports on the worker process determine the paths locally instead of getting import paths from the master?

Thanks, Gus

---

versioninfo()
Julia Version 0.5.0-dev+1158
Commit 20786d2* (2015-11-05 14:13 UTC)
Platform Info:
  System: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i7-2640M CPU @ 2.80GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Sandybridge)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.3

r = @spawn versioninfo()
fetch(r)
    From worker 2:  Julia Version 0.5.0-dev+433
    From worker 2:  Commit e60fd18 (2015-09-29 15:39 UTC)
    From worker 2:  Platform Info:
    From worker 2:    System: Linux (x86_64-linux-gnu)
    From worker 2:    CPU: Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
    From worker 2:    WORD_SIZE: 64
    From worker 2:    BLAS: libopenblas (NO_LAPACK NO_LAPACKE DYNAMIC_ARCH NO_AFFINITY Sandybridge)
    From worker 2:    LAPACK: liblapack.so.3
    From worker 2:    LIBM: libopenlibm
    From worker 2:    LLVM: libLLVM-3.3

[tkelman]

Similar to JuliaLang/Distributed.jl#22, trying to connect heterogenous operating systems isn't going to work very well. You're better off just ssh'ing into the Linux machine and running the master node from there.

[nolsie]

Hi,

I am attempting to start a worker process on a linux machine with a windows machine as the master but I cannot get it to work. I have tried all combinations of

addprocs(["user@host"], dir="/path/to/julia")
SSH is working fine, linux to linux with Julia also works fine.

Any help would be greatly appreciated.

Thanks,

Noel

[yuyichao]

It's not supported and please use the discourse for questions.