[release 0.6] stack overflow when using pyjulia

[haflo1]

probably related to #22320

On windows:
After upgrading to Julia 0.6 and Python 3.6
when I call
julia.Julia() from Python

I get exception:

Traceback (most recent call last):
  File "C:/Ron/git/pyvis/Utilities/jini.py", line 8, in <module>
    j = julia.Julia(debug =True)
  File "C:\Anaconda3\lib\site-packages\julia\core.py", line 292, in __init__
    self.api.jl_init(jl_init_path)
OSError: exception: stack overflow

Any workarounds or steps to resolve this issue would be appreciated.

[TsurHerman]

update:
We found a workaround, I am posting it here for easy referencing.

based on: #22320 , https://stackoverflow.com/a/20744680

1. install visual studio 2017 (community edition) if not already installed
   visual studio 2017 adds several alternatives for cmd.exe ,which adds all nesseceary paths (AKA vcvars64.bat)

2. start "x64 Native Tools Command Prompt for VS 2017"

3. cd to python.exe directory

4. editbin python.exe /STACK:8000000

Works like un-holy magic

[Keno]

So one option here is to run the initialization on a different thread (for which we control the stack size). That'd require some rejiggering of the embedding API (to register an additional thread as being managed/switching the main thread), but would probably be an easier backport to 0.6. @vtjnash thoughts?

[vtjnash]

Fixed on master. I think the workaround posted above is acceptable for v0.6?

[Keno]

People keep running into this and asking them to download the VS dev tools and change their python executable is not an acceptable solution. We should fix this on 0.6 one way or another. Adding some features to the embedding API to be able to do the initialization on a different thread doesn't seem so hard.

[Keno]

Another option would be automatically stack switch if we detect that the stack isn't large enough.

[moustachio-belvedere]

I had same issue but with Python 3.5 (Anaconda distribution.)

I followed steps of TsurHerman but it just changed the type of error thrown. Now I get:

WARNING: redefining constant JULIA_HOME

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x1b32c930 -- PyUnicodeUCS2_DecodeLatin1 at C:\Users\louis.kaplan\.julia\v0.6\Conda\deps\us
r\python27.DLL (unknown line)
while loading no file, in expression starting on line 0
PyUnicodeUCS2_DecodeLatin1 at C:\Users\louis.kaplan\.julia\v0.6\Conda\deps\usr\python27.DLL (unknown line)
PyObject_GenericGetAttrWithDict at C:\Users\louis.kaplan\.julia\v0.6\Conda\deps\usr\python27.DLL (unknown line)
PyEval_EvalFrameEx at C:\Anaconda3\python35.dll (unknown line)
PyEval_GetFuncDesc at C:\Anaconda3\python35.dll (unknown line)
PyEval_GetFuncDesc at C:\Anaconda3\python35.dll (unknown line)
PyEval_EvalFrameEx at C:\Anaconda3\python35.dll (unknown line)
PyEval_GetFuncDesc at C:\Anaconda3\python35.dll (unknown line)
PyEval_GetFuncDesc at C:\Anaconda3\python35.dll (unknown line)
PyEval_EvalFrameEx at C:\Anaconda3\python35.dll (unknown line)
PyEval_GetFuncDesc at C:\Anaconda3\python35.dll (unknown line)
PyEval_GetFuncDesc at C:\Anaconda3\python35.dll (unknown line)
PyEval_EvalFrameEx at C:\Anaconda3\python35.dll (unknown line)
PyEval_EvalFrameEx at C:\Anaconda3\python35.dll (unknown line)
PyFunction_SetAnnotations at C:\Anaconda3\python35.dll (unknown line)
PyObject_Call at C:\Anaconda3\python35.dll (unknown line)
PyMethod_New at C:\Anaconda3\python35.dll (unknown line)
PyObject_Call at C:\Anaconda3\python35.dll (unknown line)
PyType_Ready at C:\Anaconda3\python35.dll (unknown line)
PyType_Modified at C:\Anaconda3\python35.dll (unknown line)
PyObject_Call at C:\Anaconda3\python35.dll (unknown line)
PyEval_GetFuncDesc at C:\Anaconda3\python35.dll (unknown line)
PyEval_GetFuncDesc at C:\Anaconda3\python35.dll (unknown line)
PyEval_EvalFrameEx at C:\Anaconda3\python35.dll (unknown line)
PyEval_EvalFrameEx at C:\Anaconda3\python35.dll (unknown line)
PyRun_FileExFlags at C:\Anaconda3\python35.dll (unknown line)
PyRun_FileExFlags at C:\Anaconda3\python35.dll (unknown line)
PyRun_SimpleFileExFlags at C:\Anaconda3\python35.dll (unknown line)
PyRun_AnyFileExFlags at C:\Anaconda3\python35.dll (unknown line)
Py_hashtable_copy at C:\Anaconda3\python35.dll (unknown line)
Py_Main at C:\Anaconda3\python35.dll (unknown line)
unknown function (ip: 000000001C28125B)
BaseThreadInitThunk at C:\Windows\system32\kernel32.dll (unknown line)
RtlUserThreadStart at C:\Windows\SYSTEM32\ntdll.dll (unknown line)
Allocations: 1573454 (Pool: 1572243; Big: 1211); GC: 0
Press any key to continue . . .

Sorry if this is comment is missing any information. This is the first time I have submitted a bug report to an open source project. Let me know if I can provide any more useful info.

[Keno]

Looks like somewhere along the line it accidentally picked up python 2.7. Make sure you have the latest versions of PyCall and pyjulia which have some fixes that might help.

[moustachio-belvedere]

Thanks. Reinstalled PyCall and pyjulia and seems to be working now. I think I had recent (if not latest versions) but it might have been that I didn't set up the correct environment variable ENV["PYTHON"] in Julia before installing PyCall. Did that this time so could have been what did the trick.

[KristofferC]

Seem closed by #23255

[ghost]

In [2]: j = julia.Julia()
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
<ipython-input-2-f420be51b3fc> in <module>()
----> 1 j = julia.Julia()

C:\Users\durjoysen.maitra\Desktop\pyjulia\julia\core.py in __init__(self, init_julia, jl_runtime_path, jl_init_path, debug)
    302             self.api.jl_init_with_image.argtypes = [char_p, char_p]
    303             self._debug("calling jl_init_with_image(%s, %s)" % (jl_init_path, image_file))
--> 304             self.api.jl_init_with_image(jl_init_path, image_file.encode("utf-8"))
    305             self._debug("seems to work...")
    306

OSError: exception: stack overflow

I have Julia 0.6 and Python 3.5.2. I am still getting this error :(

[KristofferC]

Do you have julia 0.6.2?

[ghost]

No, julia 0.6.0

julia> versioninfo()
Julia Version 0.6.0
Commit 903644385b* (2017-06-19 13:05 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.9.1 (ORCJIT, skylake)

[KristofferC]

I think the patch that fixed this is only available in 0.6.2. You can download it from https://julialang.org/downloads/.