Failed to install python library, python 3.5 in Debian

[charlescearl]

Hi,
I attempted to install the python libraries in

 Debian 3.16.43-2+deb8u5

with

  conda 4.4.8
  Python 3.5.5 
 [GCC 7.2.0] on linux

I create the conda environment

conda-env create -n test-vw-35 python=3.5 -f environment.yaml

where the additional packages are given in environment.yaml

- defaults
- conda-forge
dependencies:
- ipython=6.3
- jupyter=1.0
- numpy=1.14
- pandas=0.22
- pyarrow=0.9
- scikit-learn=0.19
- scipy=1.0
- python=3.5
- cython=0.28

After creating the environment, I run

(test-vw-35) >~/test-vw-env$ pip install vowpalwabbit

and the compilation fails at

  /usr/bin/g++ -shared -Wl,--export-dynamic pylibvw.o -L /usr/local/lib -L /usr/lib -L /usr/lib/x86_64-linux-gnu -L/home/charles/.conda/envs/test-vw-35/lib/python3.5/config-3.5m -L/home/charles/.conda/envs/test-vw-35/lib -lpython3.5m -lpthread -ldl  -lutil -lrt -lm  -Xlinker -export-dynamic ../vowpalwabbit/libvw.a ../vowpalwabbit/liballreduce.a -L /usr/local/lib -L /usr/lib -L /usr/lib/x86_64-linux-gnu -l boost_program_options -l pthread -l z -l boost_python -o pylibvw.so
  lto1: fatal error: bytecode stream generated with LTO version 6.0 instead of the expected 3.0
  compilation terminated.
  lto-wrapper: /usr/bin/g++ returned 1 exit status
  /usr/bin/ld: lto-wrapper failed
  collect2: error: ld returned 1 exit status
  Makefile:55: recipe for target 'pylibvw.so' failed

Any suggestions?

[mlisovyi]

The very last paragraph of the Troubleshooting section mentions that there are issues with the VW build with conda added to the PATH. Could your problem be related to that? I'm a beginner with VW and was lucky to get it installed despite presence of conda.
P.S. After a quick google search, this anaconda issue might be relevant and might give you a hint on how to approach it.

[arielf]

Indeed. Finding where anaconda python lives is the easy part.

Linking (C and C++) binary modules under $HOME/.conda/... which may not have the same ABI or even API as the ones vw uses (the standard API+ABIs of the runtime environment) is bound to cause other issues.

The links provided by @mlisovyi say that anaconda requires its users to fully switch to their environment, including their tool-chain (compiler, linker, libraries, pip, etc.)... This means vw and its library, need to be built fully by anaconda env approved/supported tools. One can't mix anaconda with the standard (e.g. Ubuntu) tool-chain and expect it to work.

I would say that this is more an anaconda issue than a vw (which uses a standard env) issue.

[JohnLangford]

Closing this for now---as @arielf says, this seems like an anaconda issue.

[charlescearl]

Works for me in Python 3.6 with conda build script in https://github.com/JohnLangford/vowpal_wabbit/tree/master/python