Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bear dependencies should be optional dependencies in setup.py #1000

Open
jayvdb opened this issue Nov 16, 2016 · 16 comments
Open

Bear dependencies should be optional dependencies in setup.py #1000

jayvdb opened this issue Nov 16, 2016 · 16 comments

Comments

@jayvdb
Copy link
Member

jayvdb commented Nov 16, 2016

It should be possible to install this package without any bear dependencies.

setup.py shoukd use extras for each Python dependency that bears use.
And then an additional extra "alldeps" which includes all bear deps.
Once done, people should migrate to using pip install coala-bears[alldeps] and then we can remove them from the default deps.

Or we could create a coala-bears-minimal package which has no bear deps.

@sils
Copy link
Member

sils commented Nov 16, 2016

I guess they won't be installed by default then?

@jayvdb
Copy link
Member Author

jayvdb commented Dec 15, 2016

right, which will speed up installation and reduce headaches for all.

@jayvdb
Copy link
Member Author

jayvdb commented Dec 22, 2016

There are actually two sets of bear dependencies.

Direct imports

bears/python/PEP8Bear.py and bears/python/PEP8NotebookBear.py imports autopep8 directly.
bears/natural_language/LanguageToolBear.py imports guess_language and language_check directly.
bears/python/PyCommentedCodeBear.py imports eradicate directly.
bears/general/InvalidLinkBear.py imports requests directly.
etc

Indirect imports

bears/cmake/CMakeLintBear.py invokes cmakelint, which depends on cmakelint
etc.

IMO these are all optional. Without them, the bear wont run, but coala will run and other bears can be used.

Which means that dependency_management may be the only actual requirement, as almost every bear needs it, and even that requirement will probably be relocated into the main coala setup.py.

@Makman2
Copy link
Member

Makman2 commented Dec 22, 2016

how would I install optional packages for a bear (or even all bears if I want to) conveniently?

@jayvdb
Copy link
Member Author

jayvdb commented Dec 22, 2016

how would I install optional packages for a bear (or even all bears if I want to) conveniently?

See SomeProject[foo, bar] in https://pip.pypa.io/en/stable/reference/pip_install/#requirement-specifiers

For all bears, pip install coala-bears[alldeps] as mentioned in the issue description.

We can make as many extras as we like, with as many overlapping subsets as we want.

e.g. bear specific extras

pip install coala-bears[bear1,bear2,etc]

and language specific extras

pip install coala-bears[python,php,etc]

And a user could mix them like

pip install coala-bears[python,InferBear,etc]

@Makman2
Copy link
Member

Makman2 commented Dec 24, 2016

For all bears, pip install coala-bears[alldeps] as mentioned in the issue description.

ah sorry seems I've missed that in the text :)

@jayvdb
Copy link
Member Author

jayvdb commented Dec 24, 2016

blocked on #1044

@Mixih
Copy link
Member

Mixih commented Dec 26, 2016

#1044 complete. Pending implementation on ci

@jayvdb
Copy link
Member Author

jayvdb commented Dec 26, 2016

A lot of auditing required between requirements.txt and the generated version; e,g. #1212

@jayvdb
Copy link
Member Author

jayvdb commented Dec 26, 2016

And PipRequirements (all of package_management) is not ready for this. See https://gitlab.com/coala/package_manager/issues/6

@jayvdb jayvdb self-assigned this Dec 31, 2016
@jayvdb
Copy link
Member Author

jayvdb commented Dec 31, 2016

Now, the only requirements needed by coala-bears is coala. I think it would be good if we kept it that way.

I've done most of the fixes to python dependencies (see #1231 and #1230), and then will PR a "switch to using generated bear-requirements.txt".

There are then some extra GCI tasks I can create to improve setup.py and the requirements generator.

jayvdb added a commit to jayvdb/coala-bears that referenced this issue Jan 1, 2017
Remove all bear requirements from requirements.txt,
and use .ci/generate_bear_requirements.py to create
and verify that the bear-requirements is identical to
the PipRequirement found in the bears.

Update the PipRequirement in the bears to match the
versions currently being installed, and also remove
the PipRequirement for libclang-py, as this
dependency is provided by coala.

Updates setup.py to add an extra 'alldeps' that is be used
by CI to transition to explicitly requesting additional
dependencies so 100% coverage and `coala --ci` work correctly.

Uses pip to install wheels, as it supports extras while
setup.py would require a non-standardised command line option
to be added.

Closes coala#1000
@jayvdb jayvdb reopened this Jan 1, 2017
@jayvdb
Copy link
Member Author

jayvdb commented Jan 1, 2017

Not quite solved.

jayvdb added a commit to jayvdb/coala that referenced this issue Jan 1, 2017
Before we can remove the python dependencies from
the coala-bears install requirements, the other coala
repositories need to switch to using the 'alldeps'
extra.

Related to coala/coala-bears#1000
@jayvdb jayvdb added this to the 0.10 milestone Jan 18, 2017
@Mixih
Copy link
Member

Mixih commented Jan 22, 2017

@jayvdb status? do we still need to remove things?

@jayvdb
Copy link
Member Author

jayvdb commented Jan 22, 2017

Ya, we need to remove things from setup.py .
Ideally we fix the repos that depend on bears being automatically installed.
e.g. the docker, but it is broken atm due to coala/docker-coala-base#87

jayvdb added a commit to jayvdb/docker-coala-base that referenced this issue Feb 1, 2017
coala-bears will soon stop depending on all of the Python
dependencies that are required for the bears written in
Python.  To install all Python dependencies, an 'alldeps'
extra needs to be used.

Related to coala/coala-bears#1000
jayvdb added a commit to jayvdb/documentation that referenced this issue Feb 1, 2017
@jayvdb
Copy link
Member Author

jayvdb commented Feb 1, 2017

Ideally we fix the repos that depend on bears being automatically installed.

coala and the docker have been fixed to use alldeps.
coala-atom CI installs coala-bears, which may need fixing, but .. CircleCI is broken anyway: coala/coala-atom#38
Presumably other editor plugin repos also need to be checked.

@jayvdb
Copy link
Member Author

jayvdb commented May 28, 2017

This problem has caused a huge breakage as VintBear is broken until Vimjas/vint#217 is fixed. Everything fails as can be seen at coala/coala#4283

gosom pushed a commit to gosom/coala-bears that referenced this issue Jul 15, 2017
Remove all bear requirements from requirements.txt,
and use .ci/generate_bear_requirements.py to create
and verify that the bear-requirements is identical to
the PipRequirement found in the bears.

Update the PipRequirement in the bears to match the
versions currently being installed, and also remove
the PipRequirement for libclang-py, as this
dependency is provided by coala.

Updates setup.py to add an extra 'alldeps' that is be used
by CI to transition to explicitly requesting additional
dependencies so 100% coverage and `coala --ci` work correctly.

Uses pip to install wheels, as it supports extras while
setup.py would require a non-standardised command line option
to be added.

Closes coala#1000
yzgyyang pushed a commit to yzgyyang/dependency_management that referenced this issue Apr 13, 2018
ApertiumLintBear in the bears 0.12 pre-release requires many
host library and utilities, via package libxslt1-dev or similar.
As it is not used, it is more efficient to remove it, and
the hard dependency should be removed before 0.12 release
due to coala/coala-bears#1000 .

Also undo the VintBear workaround, which is unnecessary after 0.11.1.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests

5 participants