Theano on Windows with Python 3.5 probably don't work

[nouiz]

see: http://stevedower.id.au/blog/building-for-python-3-5/

Maybe we should switch to MSVC directly and not mingw? Or just wait to have mingw support it?

[patrikhuber]

That's a very nice write-up! I haven't gone through the whole blog post, but I just wanted to add that I'm successfully (modulo a few minor issues) using Theano with msys2-mingw64 and python3.5. I've installed their mingw64 python3.5, numpy, scipy, pip3 from pacman, and then installed Theano from pip3.

[ghost]

Does mingw64 python work with CUDA?

[patrikhuber]

@LexSong: I don't know, I'm not using CUDA currently, but msys2's Python has been really great so far and it looks pretty standard. Without having any experience with Python and CUDA, I'd say "Probably, why not".

[nouiz]

"Probably, why not" on windows with CUDA make me smile. I hope it is now as
easy is it could be, history told me it isn't as easy as we would like. But
someone need to try it to know. Here we won't have time shortly to try it,
so if someone try it, please report result. This can help other people.

On Sun, Dec 20, 2015 at 4:18 PM, Patrik Huber [email protected]
wrote:

@LexSong https://github.com/LexSong: I don't know, I'm not using CUDA
currently, but msys2's Python has been really great so far and it looks
pretty standard. Without having any experience with it, I'd say "Probably,
why not".

—
Reply to this email directly or view it on GitHub
#3376 (comment).

[patrikhuber]

Hehe yea, I can only imagine! I currently can't try with CUDA, maybe in the future. Hopefully other people can chime in :-) It would be great if it worked with msys2-mingw64, since msys2 is awesome and useful for many purposes, so you don't have to have several mingw distributions and each of them comes with their own gcc and takes up space etc. (e.g. most Python distributions for Windows come with their own mingw).

[ErnstDinkelmann]

Something interesting that may just help others.
My environment:
Win8.1, 64bit, Python 3.5, Anaconda4 install, MS Visual Studio 12.0, CUDA 7.5, OpenBLAS

I also experienced the C/C++ compiler and importing issues when I had
cxx = C:\TDM-GCC-64\bin\g++.exe
This is recommended in a number of installation tutorials online.

The errors I experienced:
File "C:\Program Files (x86)\JetBrains\PyCharm Community Edition 2016.1.2\helpers\pydev_pydev_bundle\pydev_import_hook.py", line 21, in do_import
module = self._system_import(name, _args, *_kwargs)
ImportError: DLL load failed: A dynamic link library (DLL) initialization routine failed.

However, when I changed to:
cxx = C:\MinGw64\mingw64\bin\g++.exe
Not sure I've seen this recommended, The g++ files are significantly different in size, so they must be a lot different in content.
Everything seems to work, at least for the scripts I'm running on scikit-neuralnetwork (sknn)

For completeness, my entire .theanorc.txt file content (stored in c:\Users\ is

[global]
floatX = float32
device = gpu
base_compiledir = C:\TheanoOperational
cxx = C:\MinGw64\mingw64\bin\g++.exe

[nvcc]
flags = -LC:\Anaconda3\libs
fastmath = True
compiler_bindir = C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\cl.exe

[cuda]
root = C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v7.5

[blas]
ldflags=-LC:\openblas -lopenblas

Important to note, even with correctly pre-compiled files in the base_compiledir done with the second g++.exe, the initial g++.exe was unable to import them correctly.

Some interest results to show that it does work in fact:
device=gpu, cxx = C:\MinGw64\mingw64\bin\g++.exe
Time: 120 secs

device=cpu, cxx =
(ie no g++ compiler)
Time: 80 secs

device=cpu, cxx = C:\MinGw64\mingw64\bin\g++.exe
Time: 40 secs

The GPU result just means I have a slow GPU compared to my CPU for this specific type of task, which is true (Corei7-2670QM vs Nvidia NVS 4200).
However, when only looking at the CPU results, hell yeah. And that's with a relatively large amount of overhead on a small run.

Hope this can help someone. May just be a fluke, but it works.

[mingwandroid]

@ErnstDinkelmann is C:\MinGw64 an MSYS2 installation?

[ErnstDinkelmann]

@mingwandroid
Out of my depth, but I will try to help.
It seems I got confused with different online sources. Essentially the procedure I followed is summarised here. But clearly I also used others (hence the C:\TDM-GCC-64\bin\g++.exe which is not even mentioned in this article)
http://rosinality.ncity.net/doku.php?id=python:installing_theano

I definitely had to run "msys.bat" (and run other commands within that window), but first had to edit it, as it did not want to execute on my pc at first. Something to do with being unable to find the WD (working directory).

For completeness, here's the contents of my msys.bat, in case someone also struggles with a similar error:

rem @echo off
set HOME=%USERPROFILE%

if "%1" == "GOTO:" goto %2

if NOT "x%WD%" == "x" set WD=

goto _WindowsNT

:_Windows

if "x%COMSPEC%" == "x" set COMSPEC=command.com
start /min %COMSPEC% /e:4096 /c %0 GOTO: _Resume %0 %1 %2 %3 %4 %5 %6 %7 %8 %9
goto EOF

:_Resume
for %%F in (1 2 3) do shift
if NOT EXIST %WD%msys-1.0.dll set WD=.\bin\

:_WindowsNT

if not "x%PROCESSOR_ARCHITECTURE%" == "xAMD64" goto _NotX64
set COMSPEC=%WINDIR%\SysWOW64\cmd.exe
%COMSPEC% /c %0 %1 %2 %3 %4 %5 %6 %7 %8 %9
goto EOF
:_NotX64

if NOT EXIST %WD%msys-1.0.dll set WD=.\bin\

set MSYSCON=unknown
if "x%1" == "x-norxvt" set MSYSCON=sh.exe
if "x%1" == "x--norxvt" set MSYSCON=sh.exe
if "x%1" == "x-rxvt" set MSYSCON=rxvt.exe
if "x%1" == "x--rxvt" set MSYSCON=rxvt.exe
if "x%1" == "x-mintty" set MSYSCON=mintty.exe
if "x%1" == "x--mintty" set MSYSCON=mintty.exe
if NOT "x%MSYSCON%" == "xunknown" shift

if "x%MSYSCON%" == "xunknown" set MSYSCON=sh.exe

if "x%MSYSTEM%" == "x" set MSYSTEM=MINGW32
if "%1" == "MINGW32" set MSYSTEM=MINGW32
if "%1" == "MSYS" set MSYSTEM=MSYS

if NOT "x%DISPLAY%" == "x" set DISPLAY=

if "x%MSYSCON%" == "xmintty.exe" goto startmintty
if "x%MSYSCON%" == "xrxvt.exe" goto startrxvt
if "x%MSYSCON%" == "xsh.exe" goto startsh

:unknowncon
echo %MSYSCON% is an unknown option for msys.bat.
pause
exit 1

:notfound
echo Cannot find the rxvt.exe or sh.exe binary -- aborting.
pause
exit 1

:startmintty
if NOT EXIST %WD%mintty.exe goto startsh
start %WD%mintty /bin/bash -l
exit

:startrxvt
if NOT EXIST %WD%rxvt.exe goto startsh


if "x%MSYSBGCOLOR%" == "x" set MSYSBGCOLOR=White
if "x%MSYSFGCOLOR%" == "x" set MSYSFGCOLOR=Black
if "x%MINGW32BGCOLOR%" == "x" set MINGW32BGCOLOR=LightYellow
if "x%MINGW32FGCOLOR%" == "x" set MINGW32FGCOLOR=Navy
if "%MSYSTEM%" == "MSYS" set BGCOLOR=%MSYSBGCOLOR%
if "%MSYSTEM%" == "MSYS" set FGCOLOR=%MSYSFGCOLOR%
if "%MSYSTEM%" == "MINGW32" set BGCOLOR=%MINGW32BGCOLOR%
if "%MSYSTEM%" == "MINGW32" set FGCOLOR=%MINGW32FGCOLOR%

start %WD%rxvt -backspacekey � -sl 2500 -fg %FGCOLOR% -bg %BGCOLOR% -sr -fn Courier-12 -tn msys -geometry 80x25 -e /bin/sh --login -i
exit

:startsh
if NOT EXIST %WD%sh.exe goto notfound
start %WD%sh --login -i
exit

:EOF

The above file also only ran when I did NOT do it as administrator. If I did "run it as administrator", it did not leave the msys command window open.

Other issues I experienced was folder write rights issues due to our corporate policy, so I ended up installing everything in C: root, rather than in program files, etc etc. That sorted out a lot.

Another tutorial that helped in the end, which I found quite helpful:
http://simranmetric.com/category/tutorials/

[ruslanagit]

I have successfully installed theano release 0.8.2 with Python 3.5
It took me a few days to learn how to make it work, but at the end it does work. Most tests pass (I had up to 15 failures out of 21,724 tests that run), and Keras examples run smoothly on top of it.
Now, I tried to upgrade my theano to development version, but it is currently blocked with the message that theano is not supported on windows with python 3.5
It's getting hard to work with 2 versions, since I am also doing some fixes for theano. Now I need to maintain patches for release version, and also merge fixes into development branch (which I cannot even run to test...).
Could you please open development version for python 3.5 too?
I can contribute my installation instructions, if needed.

[nouiz]

You can just comment the raising of that error. Then it should continue to
work. At least, I don't know what could have broken it.

What did you do to make Theano work with python 3.5? We don't have time for
this, but if you or someone else make Theano support Python 3.5 on windows,
then we can remove the assert at the top. Otherwise, people have strange
errors that don't tell what the problem is.

On Fri, Jul 22, 2016 at 12:52 PM, ruslanagit [email protected]
wrote:

I have successfully installed theano release 0.8.2 with Python 3.5
It took me a few days to learn how to make it work, but at the end it does
work. Most tests pass (I had up to 15 failures out of 21,724 tests that
run), and Keras examples run smoothly on top of it.
Now, I tried to upgrade my theano to development version, but it is
currently blocked with the message that theano is not supported on windows
with python 3.5
It's getting hard to work with 2 versions, since I am also doing some
fixes for theano. Now I need to maintain patches for release version, and
also merge fixes into development branch (which I cannot even run to
test...).
Could you please open development version for python 3.5 too?
I can contribute my installation instructions, if needed.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#3376 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AALC-4XDIYSEeBtAi2oR4mM5hou3WU8Kks5qYPVkgaJpZM4F6-XI
.

[ruslanagit]

Well, the main problem with Python 3.5 on windows, I guess, is that mingw and libpython are not available (not complied with Python 3.5), so you cannot run $ conda install mingw libpython
step.
So you either need to downgrade to Python 3.4 (was not an option for me) and then follow standard instructions for installing Theano on Windows, or make a few tricks to make theano work with Python 3.5. For me the following steps worked on Windows 10 with Anaconda3 and Python 3.5:

1. Install mingw from https://sourceforge.net/projects/mingw-w64/
2. Add the bin directory of mingw to PATH, and make sure there is no other gcc compiler in PATH (i.e. TDM-GCC is not there).
3. In .theanorc file add
   
   [gcc]
   
   cxxflags = -shared -I"[TDM-GCC path]\include" -I"[TDM-GCC path]\x86_64-w64-mingw32\include"
   
   You should update paths to TDM-GCC according to your system.
   Note, that TDM include directory is required, since the compilation will fail with mingw include directories for python 3.5 (I think they would work for Python 2, but I am not 100% sure)
4. Create libpython35.a manually and copy it to appropriate directory. For example:

• Create temp directory
• Copy python35.dll (I took it from Anaconda3 folder) into created directory
• Navigate into created directory
• Run:
  
  gendef python35.dll
• Run:
  
  dlltool --dllname python35.dll --def python35.def --output-lib libpython35.a
• Copy libpython35.a into Anaconda3\libs

All other installations/configurations were done as described in the guide for installaing Theano on Windows.

[junk-anon]

Hi, I am trying to get theano to work with PyCharm. I switched from 3.5 to 3.4 and have tried some of the things above but none seem to be working. I'm running into 2 errors:

nvcc : fatal error : Wrong syntax in file C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v5.5\bin/nvcc.profile: "2010"
ERROR (theano.sandbox.cuda): Failed to compile cuda_ndarray.cu: ('nvcc return status', 4294967295, 'for cmd', 'nvcc -shared -O3 -Xlinker /DEBUG -D HAVE_ROUND -m64 -Xcompiler -DCUDA_NDARRAY_CUH=mc72d035fdf91890f3b36710688069b2e,-DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION,/Zi,/MD -I"C:\Lasagne\WinPython\python-3.4.4.amd64\lib\site-packages\theano\sandbox\cuda" -I"C:\Lasagne\WinPython\python-3.4.4.amd64\lib\site-packages\numpy\core\include" -I"C:\Lasagne\WinPython\python-3.4.4.amd64\include" -I"C:\Lasagne\WinPython\python-3.4.4.amd64\lib\site-packages\theano\gof" -L"C:\Lasagne\WinPython\python-3.4.4.amd64\libs" -L"C:\Lasagne\WinPython\python-3.4.4.amd64" -o C:\Users\Anon\AppData\Local\Theano\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_42_Stepping_7_GenuineIntel-3.4.4-64\cuda_ndarray\cuda_ndarray.pyd mod.cu -lcublas -lpython34 -lcudart')

Interestingly this error showed up in Spyder and Pycharm but after adding this:

compiler-bindir = C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin

which I found in another thread, CUDA worked in Spyder but stayed broken in Pycharm.

And the second is that it is not recognizing the .theanorc file which I figured out trying to install it before and getting help with the author of the link below. I made that file and theano works with Spyder and the installation instructions found here:

https://github.com/Lasagne/Lasagne/wiki/From-Zero-to-Lasagne-on-Windows-7-(64-bit)

I really prefer PyCharm but don't know where to put the settings or how to make it find .theanorc

Does anybody have any ideas please?

Thanks

[lamblin]

Maybe PyCharm looks for your home directory in a different place? You can try to use os.path.expanduser('~') to see where it is trying to look the .theanorc by default.
You can also set the THEANORC environment variable explicitly.

[junk-anon]

Thank you I'll try that tonight. What do you mean by set the theanorc environmental variable explicitly please?

[junk-anon]

Got some time to try it. Found my path to simply be c:\Users\Anon so put .theanorc there.

May be a little closer but not sure. It now gives the following error for the loading of CUDA:

nvcc : fatal error : Compiler 'cl' in PATH different than the one specified with -ccbin

I'm not sure where -ccbin is defined.

I also found suggestions to change my .theanorc file and found that adding the following helped me get to this new error:

[nvcc]
flags=--use-local-env --cl-version=2010

So now I am hopeful that if I can find the -ccbin path definition then it might work. Google hasn't been helpful so far but still working on it.

Does anyone have other ideas please?

BTW I'd like to change my default path which I found from the "os.path.expanduser('~')" suggestion from c:\users\anon to a more python related directory. How do I change that path please?

[ErnstDinkelmann]

@junk-anon I'm not able to help you debug your issue as I have no idea whats going wrong, but here is the full installation steps that I followed written out in a simple manner in case I have to re-do it at some stage.. Maybe something in here helps you.:

Installing Theano (the underlying package for most NN wrappers)

References:

the most helpful: http://simranmetric.com/category/tutorials/
other that may help (specific 64 bit also): http://rosinality.ncity.net/doku.php?id=python:installing_theano
more generic (also covers 32 bit, but also other things): http://deeplearning.net/software/theano/install_windows.html
Other:
https://www.kaggle.com/c/otto-group-product-classification-challenge/forums/t/13973/a-few-tips-to-install-theano-on-windows-64-bits
https://lepisma.github.io/articles/2015/07/30/up-with-theano-and-cuda/
http://stackoverflow.com/questions/25729969/installing-theano-on-windows-8-with-gpu-enabled

General

First, read the first two references to get a feel for what needs to be done.
Then read the below so that you are aware of things that might need to change.

This installation worked for Win 8.1, 64 Bit, Python 3.5
When running things, run as administrator, except for the .bat file... for some reason that does not work.
See comment below about where not to install certain things

Prorgams to Install And Folder where I installed (which in some cases should avoid being in folders like Windows, Program Files, Users, etc, where there may be permission issues)

AnaConda c:\AnaConda3\ Required
MinGw64 c:\MinGw64\ Required
msys c:\MinGw64\msys\ Required
Visual Studio 12 default as specified by installer Required
CUDA 7.5 default as specified by installer Optional if your GPU is CUDA supported
cuDNN manual work (see notes later) Optional and related to CUDA.
OpenBLAS c:\OpenBLAS\ Optional if you want your calculations to run faster.

Also created a folder "C:\TheanoOperational", which you will see is referenced in the setup files below as the folder where theano will pre-compile C-Code

conda install mingw libpython (as an alternative to get all dependencies did not work for me at the time as the mingw library was not updated at that point to support Python 3.5 (which was the version I ran with).

MSYS NOTE

use the msys.bat file that I commented previously in this thread. I made a single change, because it did not want to run on my ocmputer.

cuDNN from CUDA (optional)

The only thing that worked for me was to manually copy the CuDNN files (in the SIP folder) into the appropriate folders in your CUDA installation.
For example, if your CUDA installation is in "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v7.5", copy files in the zip to the corresponding folders.

Full set of environmental variables (some are created automatically by their installers):

When changing paths, make sure to restart the computer. The PATHS enviromental variable are not updated properly in Windows until a restart! This bit me - wasted a lot of time. Never knew it, but it really made a difference on at least one occasion.

CUDA_PATH C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v7.5
CUDA_PATH_V7_5 C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v7.5
CUDA_ROOT C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v7.5
NUMBER_OF_PROCESSORS 8 but you need to set equal to your own number of logical processors on your CPU (not # cores, # logical processors)
OMP_NUM_THREADS 6 set to about 3/4 of num processors
OPENBLAS_NUM_THREADS 6
PATH C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v7.5\bin;
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v7.5\libnvvp;
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v7.5;
C:\MinGw64\MinGw64\bin;
C:\Anaconda3;
C:\Anaconda3\Scripts;
C:\Anaconda3\Library\bin;
C:\GnuWin32\bin;
C:\OpenBLAS;
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin
ie, the above needs to be in your path. Many will be create by the installers, but double check.

Content of ".theanorx.txt" settings file (to be saved in C:\Users\ See http://deeplearning.net/software/theano/library/config.html for more info

[global]
floatX = float32
device = gpu
base_compiledir = C:\TheanoOperational
cxx = C:\MinGw64\mingw64\bin\g++.exe
openmp = True
openmp_elementwise_minsize = 200000
mode = FAST_RUN
optimizer = fast_run

[nvcc]
flags = -LC:\Anaconda3\libs
fastmath = True
compiler_bindir = C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\cl.exe

[dnn]
enabled = auto

[cuda]
root = C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v7.5

[blas]
ldflags=-LC:\openblas -lopenblas

STUFF THAT WAS PROBABLY REDUNDANT/UNECESSARY

environmental variable
LD_LIBRARY_PATH C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v7.5;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v7.5\bin;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v7.5\lib\x64;C:\OpenBLAS

[junk-anon]

Thank you very much!

I'm uninstalling stuff tonight and will try to install again and will use this to help. I'll update when I know if I got Pycharm to work.

[nouiz]

With the new conda packages, it work.