Installation of bears requires libxml2

[coala-bot]

This dependency was introduced by recently added bears which use Python package lxml.
Installing lxml from pypi requires libxml2 headers, which are typically not found on Windows.
This is going to be a major problem until coala/coala-bears#1000 is fixed.
Temporarily, we can improve good instructions for Windows users on how to install lxml.
difficulty/low

Opened via gitter by @jayvdb

[kbhagi]

1. started here : https://coala.io/#!/getinvolved
2. Next here http://api.coala.io/en/latest/Developers/Newcomers_Guide.html
3. Confused between what action to take next : https://coala.io/install & http://coala.io/devsetup
4. Then went here http://docs.coala.io/en/latest/Users/Install.html
5. Followed recommendation to install coala,coala-bears inside virtual-env
6. installed virtual-env,pip
7. activated virtual-env
8. installed coala inside activate virtual-env coala_env
9. tried to install coala-bears inside coala_env
10. started getting errors which says unable to install libxml2 & some version conflicts
11. tried to install system-wide outside of virtual-env on cmd with command pip install coala-bears
12. cloned the repsoitory coala,coala-bears
13. cd into coala folder, tried pip install . --> worked successfully, installed it successfully.
14. cd into coala-bears folder, tried pip install . ---> got lots of erros which I am posting isntallation error log below :

9999999999)
Requirement already satisfied: coala-utils~=0.6.2 in c:\program files\python36\l
ib\site-packages (from coala>=0.12.0.dev20170604215434->coala-bears==0.12.0.dev9
9999999999999)
Requirement already satisfied: unidiff~=0.5.2 in c:\program files\python36\lib\s
ite-packages (from coala>=0.12.0.dev20170604215434->coala-bears==0.12.0.dev99999
999999999)
Requirement already satisfied: PyPrint~=0.2.6 in c:\program files\python36\lib\s
ite-packages (from coala>=0.12.0.dev20170604215434->coala-bears==0.12.0.dev99999
999999999)
Requirement already satisfied: setuptools>=17.0 in c:\program files\python36\lib
\site-packages (from coala>=0.12.0.dev20170604215434->coala-bears==0.12.0.dev999
99999999999)
Requirement already satisfied: appdirs~=1.4 in c:\program files\python36\lib\sit
e-packages (from coala>=0.12.0.dev20170604215434->coala-bears==0.12.0.dev9999999
9999999)
Requirement already satisfied: stevedore>=1.17.1 in c:\program files\python36\li
b\site-packages (from bandit~=1.2->coala-bears==0.12.0.dev99999999999999)
Requirement already satisfied: six>=1.9.0 in c:\program files\python36\lib\site-
packages (from bandit~=1.2->coala-bears==0.12.0.dev99999999999999)
Requirement already satisfied: GitPython>=1.0.1 in c:\program files\python36\lib
\site-packages (from bandit~=1.2->coala-bears==0.12.0.dev99999999999999)
Requirement already satisfied: polib>=1.0.8 in c:\program files\python36\lib\sit
e-packages (from dennis~=0.8->coala-bears==0.12.0.dev99999999999999)
Requirement already satisfied: docopt>=0.6.1 in c:\program files\python36\lib\si
te-packages (from html-linter~=0.3.0->coala-bears==0.12.0.dev99999999999999)
Requirement already satisfied: template-remover in c:\program files\python36\lib
\site-packages (from html-linter~=0.3.0->coala-bears==0.12.0.dev99999999999999)
Requirement already satisfied: jsonschema!=2.5.0,>=2.4 in c:\program files\pytho
n36\lib\site-packages (from nbformat~=4.1->coala-bears==0.12.0.dev99999999999999
)
Requirement already satisfied: jupyter-core in c:\program files\python36\lib\sit
e-packages (from nbformat~=4.1->coala-bears==0.12.0.dev99999999999999)
Requirement already satisfied: ipython-genutils in c:\program files\python36\lib
\site-packages (from nbformat~=4.1->coala-bears==0.12.0.dev99999999999999)
Requirement already satisfied: traitlets>=4.1 in c:\program files\python36\lib\s
ite-packages (from nbformat~=4.1->coala-bears==0.12.0.dev99999999999999)
Requirement already satisfied: future in c:\program files\python36\lib\site-pack
ages (from proselint~=0.7.0->coala-bears==0.12.0.dev99999999999999)
Requirement already satisfied: snowballstemmer in c:\program files\python36\lib
site-packages (from pydocstyle~=2.0->coala-bears==0.12.0.dev99999999999999)
Requirement already satisfied: astroid>=1.5.1 in c:\program files\python36\lib\s
ite-packages (from pylint~=1.6->coala-bears==0.12.0.dev99999999999999)
Requirement already satisfied: mccabe in c:\program files\python36\lib\site-pack
ages (from pylint~=1.6->coala-bears==0.12.0.dev99999999999999)
Requirement already satisfied: colorama; sys_platform == "win32" in c:\program f
iles\python36\lib\site-packages (from pylint~=1.6->coala-bears==0.12.0.dev999999
99999999)
Requirement already satisfied: docutils in c:\program files\python36\lib\site-pa
ckages (from pyroma~=2.2.0->coala-bears==0.12.0.dev99999999999999)
Requirement already satisfied: mando<0.4,>=0.3 in c:\program files\python36\lib
site-packages (from radon==1.4.0->coala-bears==0.12.0.dev99999999999999)
Requirement already satisfied: chardet>=2.3.0 in c:\program files\python36\lib\s
ite-packages (from vim-vint~=0.3.12->coala-bears==0.12.0.dev99999999999999)
Collecting ansicolor~=0.2.4 (from vim-vint~=0.3.12->coala-bears==0.12.0.dev99999
999999999)
Using cached ansicolor-0.2.4-py2.py3-none-any.whl
Requirement already satisfied: idna<2.6,>=2.5 in c:\program files\python36\lib\s
ite-packages (from requests~=2.12->coala>=0.12.0.dev20170604215434->coala-bears=
=0.12.0.dev99999999999999)
Requirement already satisfied: certifi>=2017.4.17 in c:\program files\python36\l
ib\site-packages (from requests~=2.12->coala>=0.12.0.dev20170604215434->coala-be
ars==0.12.0.dev99999999999999)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in c:\program files\python3
6\lib\site-packages (from requests~=2.12->coala>=0.12.0.dev20170604215434->coala
-bears==0.12.0.dev99999999999999)
Requirement already satisfied: sarge~=0.1.4 in c:\program files\python36\lib\sit
e-packages (from dependency-management~=0.4.0->coala>=0.12.0.dev20170604215434->
coala-bears==0.12.0.dev99999999999999)
Requirement already satisfied: pyparsing in c:\program files\python36\lib\site-p
ackages (from packaging~=16.8->coala>=0.12.0.dev20170604215434->coala-bears==0.1
2.0.dev99999999999999)
Requirement already satisfied: termcolor~=1.1.0 in c:\program files\python36\lib
\site-packages (from PyPrint~=0.2.6->coala>=0.12.0.dev20170604215434->coala-bear
s==0.12.0.dev99999999999999)
Requirement already satisfied: pbr!=2.1.0,>=2.0.0 in c:\program files\python36\l
ib\site-packages (from stevedore>=1.17.1->bandit~=1.2->coala-bears==0.12.0.dev99
999999999999)
Requirement already satisfied: gitdb2>=2.0.0 in c:\program files\python36\lib\si
te-packages (from GitPython>=1.0.1->bandit~=1.2->coala-bears==0.12.0.dev99999999
999999)
Requirement already satisfied: decorator in c:\program files\python36\lib\site-p
ackages (from traitlets>=4.1->nbformat~=4.1->coala-bears==0.12.0.dev999999999999
99)
Requirement already satisfied: wrapt in c:\program files\python36\lib\site-packa
ges (from astroid>=1.5.1->pylint~=1.6->coala-bears==0.12.0.dev99999999999999)
Requirement already satisfied: lazy-object-proxy in c:\program files\python36\li
b\site-packages (from astroid>=1.5.1->pylint~=1.6->coala-bears==0.12.0.dev999999
99999999)
Requirement already satisfied: smmap2>=2.0.0 in c:\program files\python36\lib\si
te-packages (from gitdb2>=2.0.0->GitPython>=1.0.1->bandit~=1.2->coala-bears==0.1
2.0.dev99999999999999)
Building wheels for collected packages: coala-bears, lxml
Running setup.py bdist_wheel for coala-bears ... done
Stored in directory: C:\Users\mindwarrior\AppData\Local\pip\Cache\wheels\4a\7f
\b1\975dc7655c8c1f955e9c35e2a3cbf33faedc19f8729ab6dae1
Running setup.py bdist_wheel for lxml ... error
Complete output from command "c:\program files\python36\python.exe" -u -c "imp
ort setuptools, tokenize;file='C:\Users\MINDWA1\AppData\Local\Temp\pi
p-build-00n_zh3t\lxml\setup.py';f=getattr(tokenize, 'open', open)(file);co
de=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec')
)" bdist_wheel -d C:\Users\MINDWA1\AppData\Local\Temp\tmptsregsflpip-wheel- --p
ython-tag cp36:
Building lxml version 3.6.0.
Building without Cython.
ERROR: b"'xslt-config' is not recognized as an internal or external command,\r
\noperable program or batch file.\r\n"
** make sure the development packages of libxml2 and libxslt are installed **

Using build configuration of libxslt
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-3.6
creating build\lib.win-amd64-3.6\lxml
copying src\lxml\builder.py -> build\lib.win-amd64-3.6\lxml
copying src\lxml\cssselect.py -> build\lib.win-amd64-3.6\lxml
copying src\lxml\doctestcompare.py -> build\lib.win-amd64-3.6\lxml
copying src\lxml\ElementInclude.py -> build\lib.win-amd64-3.6\lxml
copying src\lxml\pyclasslookup.py -> build\lib.win-amd64-3.6\lxml
copying src\lxml\sax.py -> build\lib.win-amd64-3.6\lxml
copying src\lxml\usedoctest.py -> build\lib.win-amd64-3.6\lxml
copying src\lxml_elementpath.py -> build\lib.win-amd64-3.6\lxml
copying src\lxml_init_.py -> build\lib.win-amd64-3.6\lxml
creating build\lib.win-amd64-3.6\lxml\includes
copying src\lxml\includes_init_.py -> build\lib.win-amd64-3.6\lxml\includes

creating build\lib.win-amd64-3.6\lxml\html
copying src\lxml\html\builder.py -> build\lib.win-amd64-3.6\lxml\html
copying src\lxml\html\clean.py -> build\lib.win-amd64-3.6\lxml\html
copying src\lxml\html\defs.py -> build\lib.win-amd64-3.6\lxml\html
copying src\lxml\html\diff.py -> build\lib.win-amd64-3.6\lxml\html
copying src\lxml\html\ElementSoup.py -> build\lib.win-amd64-3.6\lxml\html
copying src\lxml\html\formfill.py -> build\lib.win-amd64-3.6\lxml\html
copying src\lxml\html\html5parser.py -> build\lib.win-amd64-3.6\lxml\html
copying src\lxml\html\soupparser.py -> build\lib.win-amd64-3.6\lxml\html
copying src\lxml\html\usedoctest.py -> build\lib.win-amd64-3.6\lxml\html
copying src\lxml\html_diffcommand.py -> build\lib.win-amd64-3.6\lxml\html
copying src\lxml\html_html5builder.py -> build\lib.win-amd64-3.6\lxml\html
copying src\lxml\html_setmixin.py -> build\lib.win-amd64-3.6\lxml\html
copying src\lxml\html_init_.py -> build\lib.win-amd64-3.6\lxml\html
creating build\lib.win-amd64-3.6\lxml\isoschematron
copying src\lxml\isoschematron_init_.py -> build\lib.win-amd64-3.6\lxml\iso
schematron
copying src\lxml\lxml.etree.h -> build\lib.win-amd64-3.6\lxml
copying src\lxml\lxml.etree_api.h -> build\lib.win-amd64-3.6\lxml
copying src\lxml\includes\c14n.pxd -> build\lib.win-amd64-3.6\lxml\includes
copying src\lxml\includes\config.pxd -> build\lib.win-amd64-3.6\lxml\includes
copying src\lxml\includes\dtdvalid.pxd -> build\lib.win-amd64-3.6\lxml\include
s
copying src\lxml\includes\etreepublic.pxd -> build\lib.win-amd64-3.6\lxml\incl
udes
copying src\lxml\includes\htmlparser.pxd -> build\lib.win-amd64-3.6\lxml\inclu
des
copying src\lxml\includes\relaxng.pxd -> build\lib.win-amd64-3.6\lxml\includes

copying src\lxml\includes\schematron.pxd -> build\lib.win-amd64-3.6\lxml\inclu
des
copying src\lxml\includes\tree.pxd -> build\lib.win-amd64-3.6\lxml\includes
copying src\lxml\includes\uri.pxd -> build\lib.win-amd64-3.6\lxml\includes
copying src\lxml\includes\xinclude.pxd -> build\lib.win-amd64-3.6\lxml\include
s
copying src\lxml\includes\xmlerror.pxd -> build\lib.win-amd64-3.6\lxml\include
s
copying src\lxml\includes\xmlparser.pxd -> build\lib.win-amd64-3.6\lxml\includ
es
copying src\lxml\includes\xmlschema.pxd -> build\lib.win-amd64-3.6\lxml\includ
es
copying src\lxml\includes\xpath.pxd -> build\lib.win-amd64-3.6\lxml\includes
copying src\lxml\includes\xslt.pxd -> build\lib.win-amd64-3.6\lxml\includes
copying src\lxml\includes\etree_defs.h -> build\lib.win-amd64-3.6\lxml\include
s
copying src\lxml\includes\lxml-version.h -> build\lib.win-amd64-3.6\lxml\inclu
des
creating build\lib.win-amd64-3.6\lxml\isoschematron\resources
creating build\lib.win-amd64-3.6\lxml\isoschematron\resources\rng
copying src\lxml\isoschematron\resources\rng\iso-schematron.rng -> build\lib.w
in-amd64-3.6\lxml\isoschematron\resources\rng
creating build\lib.win-amd64-3.6\lxml\isoschematron\resources\xsl
copying src\lxml\isoschematron\resources\xsl\RNG2Schtrn.xsl -> build\lib.win-a
md64-3.6\lxml\isoschematron\resources\xsl
copying src\lxml\isoschematron\resources\xsl\XSD2Schtrn.xsl -> build\lib.win-a
md64-3.6\lxml\isoschematron\resources\xsl
creating build\lib.win-amd64-3.6\lxml\isoschematron\resources\xsl\iso-schematr
on-xslt1
copying src\lxml\isoschematron\resources\xsl\iso-schematron-xslt1\iso_abstract
_expand.xsl -> build\lib.win-amd64-3.6\lxml\isoschematron\resources\xsl\iso-sche
matron-xslt1
copying src\lxml\isoschematron\resources\xsl\iso-schematron-xslt1\iso_dsdl_inc
lude.xsl -> build\lib.win-amd64-3.6\lxml\isoschematron\resources\xsl\iso-schemat
ron-xslt1
copying src\lxml\isoschematron\resources\xsl\iso-schematron-xslt1\iso_schematr
on_message.xsl -> build\lib.win-amd64-3.6\lxml\isoschematron\resources\xsl\iso-s
chematron-xslt1
copying src\lxml\isoschematron\resources\xsl\iso-schematron-xslt1\iso_schematr
on_skeleton_for_xslt1.xsl -> build\lib.win-amd64-3.6\lxml\isoschematron\resource
s\xsl\iso-schematron-xslt1
copying src\lxml\isoschematron\resources\xsl\iso-schematron-xslt1\iso_svrl_for
_xslt1.xsl -> build\lib.win-amd64-3.6\lxml\isoschematron\resources\xsl\iso-schem
atron-xslt1
copying src\lxml\isoschematron\resources\xsl\iso-schematron-xslt1\readme.txt -

build\lib.win-amd64-3.6\lxml\isoschematron\resources\xsl\iso-schematron-xslt1
running build_ext
building 'lxml.etree' extension
creating build\temp.win-amd64-3.6
creating build\temp.win-amd64-3.6\Release
creating build\temp.win-amd64-3.6\Release\src
creating build\temp.win-amd64-3.6\Release\src\lxml
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe /c
/nologo /Ox /W3 /GL /DNDEBUG /MD -Isrc\lxml\includes "-Ic:\program files\python
36\include" "-Ic:\program files\python36\include" "-IC:\Program Files (x86)\Micr
osoft Visual Studio 14.0\VC\INCLUDE" "-IC:\Program Files (x86)\Microsoft Visual
Studio 14.0\VC\ATLMFC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include
\10.0.10240.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\includ
e\um" "-IC:\Program Files (x86)\Windows Kits\8.1\include\shared" "-IC:\Program
Files (x86)\Windows Kits\8.1\include\um" "-IC:\Program Files (x86)\Windows Kits
\8.1\include\winrt" /Tcsrc\lxml\lxml.etree.c /Fobuild\temp.win-amd64-3.6\Releas
e\src\lxml\lxml.etree.obj -w
cl : Command line warning D9025 : overriding '/W3' with '/w'
lxml.etree.c
src\lxml\includes\etree_defs.h(14): fatal error C1083: Cannot open include fil
e: 'libxml/xmlversion.h': No such file or directory
Compile failed: command 'C:\Program Files (x86)\Microsoft Visual Studio 14.0
\VC\BIN\x86_amd64\cl.exe' failed with exit status 2
creating Users
creating Users\MINDWA1
creating Users\MINDWA1\AppData
creating Users\MINDWA1\AppData\Local
creating Users\MINDWA1\AppData\Local\Temp
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe /c
/nologo /Ox /W3 /GL /DNDEBUG /MD -I/usr/include/libxml2 "-IC:\Program Files (x8
6)\Microsoft Visual Studio 14.0\VC\INCLUDE" "-IC:\Program Files (x86)\Microsoft
Visual Studio 14.0\VC\ATLMFC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10
include\10.0.10240.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1
\include\um" "-IC:\Program Files (x86)\Windows Kits\8.1\include\shared" "-IC:\P
rogram Files (x86)\Windows Kits\8.1\include\um" "-IC:\Program Files (x86)\Windo
ws Kits\8.1\include\winrt" /TcC:\Users\MINDWA1\AppData\Local\Temp\xmlXPathInit
ov510k8m.c /FoUsers\MINDWA1\AppData\Local\Temp\xmlXPathInitov510k8m.obj
xmlXPathInitov510k8m.c
C:\Users\MINDWA~1\AppData\Local\Temp\xmlXPathInitov510k8m.c(1): fatal error C1
083: Cannot open include file: 'libxml/xpath.h': No such file or directory

---

---

Could not find function xmlCheckVersion in library libxml2. Is libxml2 install
ed?

---

---

error: command 'C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN
\x86_amd64\cl.exe' failed with exit status 2

---

Failed building wheel for lxml
Running setup.py clean for lxml
Successfully built coala-bears
Failed to build lxml
Installing collected packages: coala, apertium-lint, lxml, memento-client, pydoc
style, rstcheck, ansicolor, vim-vint, coala-bears
Found existing installation: coala 0.11.0
Uninstalling coala-0.11.0:
Exception:
Traceback (most recent call last):
File "c:\program files\python36\lib\shutil.py", line 538, in move
os.rename(src, real_dst)
PermissionError: [WinError 5] Access is denied: 'c:\program files\python36\li
b\site-packages\coala-0.11.0.dist-info\description.rst' -> 'C:\Users\MINDWA
~1\AppData\Local\Temp\pip-0iatizw2-uninstall\program files\python36\lib\
site-packages\coala-0.11.0.dist-info\description.rst'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "c:\program files\python36\lib\site-packages\pip\basecommand.py", line 21
5, in main
status = self.run(options, args)
File "c:\program files\python36\lib\site-packages\pip\commands\install.py", li
ne 342, in run
prefix=options.prefix_path,
File "c:\program files\python36\lib\site-packages\pip\req\req_set.py", line 77
8, in install
requirement.uninstall(auto_confirm=True)
File "c:\program files\python36\lib\site-packages\pip\req\req_install.py", lin
e 754, in uninstall
paths_to_remove.remove(auto_confirm)
File "c:\program files\python36\lib\site-packages\pip\req\req_uninstall.py", l
ine 115, in remove
renames(path, new_path)
File "c:\program files\python36\lib\site-packages\pip\utils_init_.py", line
267, in renames
shutil.move(old, new)
File "c:\program files\python36\lib\shutil.py", line 553, in move
os.unlink(src)
PermissionError: [WinError 5] Access is denied: 'c:\program files\python36\li
b\site-packages\coala-0.11.0.dist-info\description.rst'

[ramoska]

We need to resolve coala/coala-bears#1000 first, since lxml dependency:

• is not so easy to install (you need libxml2 and libxslt1 installed before)
• is only required by one Bear
• issue will be almost solved by solving Bear dependencies should be optional dependencies in setup.py coala-bears#1000

After blocking issue is solved we can update documentation how to install optional pip packages.

[jayvdb]

This is a documentation task to explain lxml. It valid even after coala/coala-bears#1000, as installing lxml on Windows is not easy, as it often depends on libxml development files.

[Makman2]

Unassigned per personal request.

[MorElmaliach]

I would like to work on this.
BTW, an easy fix that did the job for me was using: sudo apt-get install zlib1g-dev, and this was after installing libxml2 and libxslt1. This was ofcourse done on linux, but it's a VM, so I'll try to go ahead and tweak with this on my windows 10 computer to try and find a solution for win users.