Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pylint seems broken on the latest base64-lts 16.4.1 #476

Closed
sjorge opened this issue Mar 20, 2017 · 7 comments
Closed

pylint seems broken on the latest base64-lts 16.4.1 #476

sjorge opened this issue Mar 20, 2017 · 7 comments
Assignees
@mamash

Comments

@sjorge
Copy link

sjorge commented Mar 20, 2017

I originally wanted to open this against saltstack because I was having issues with there saltpylint, but after 2 rebuilds of the zone an traced it to py27-pylint from the 2016Q4 set.

Bellow is the entire issues wrote over at there repo before figuring out it needs to go here, I will summery my finding above the Description section...

It seems that py27-pylint-1.6.1 pulls in py27-astroid-1.3.6
When running pylint-2.7 it seems that it boils down to a few missing dependencies, base-lts 15.4.x never had those issues, it worked out of the box... but more importantly it seems to want a newer version of py27-astroid.

Removing the packages and switching to the ones provided by pip works fine. If I installed the same pylint version via pip, it pulls in a newer astroid too!

Collecting pylint==1.6.1
  Downloading pylint-1.6.1-py2.py3-none-any.whl (562kB)
    100% |################################| 563kB 819kB/s 
Requirement already satisfied: six in /opt/local/lib/python2.7/site-packages (from pylint==1.6.1)
Requirement already satisfied: isort>=4.2.5 in /opt/local/lib/python2.7/site-packages (from pylint==1.6.1)
Requirement already satisfied: mccabe in /opt/local/lib/python2.7/site-packages (from pylint==1.6.1)
Requirement already satisfied: configparser; python_version == "2.7" in /opt/local/lib/python2.7/site-packages (from pylint==1.6.1)
Requirement already satisfied: backports.functools-lru-cache; python_version == "2.7" in /opt/local/lib/python2.7/site-packages (from pylint==1.6.1)
Collecting astroid<1.5.0,>=1.4.5 (from pylint==1.6.1)
  Using cached astroid-1.4.9-py2.py3-none-any.whl
Requirement already satisfied: lazy-object-proxy in /opt/local/lib/python2.7/site-packages (from astroid<1.5.0,>=1.4.5->pylint==1.6.1)
Requirement already satisfied: wrapt in /opt/local/lib/python2.7/site-packages (from astroid<1.5.0,>=1.4.5->pylint==1.6.1)
Installing collected packages: astroid, pylint
Successfully installed astroid-1.4.9 pylint-1.6.1

So I think that py27-astroid needs to be updated to a newer version to resolve this issue.

Description of Issue/Question

After redoplying my dev environment I'm having issues with the newer saltpylint it pulled in.

  1. lets run pylint against logadm.py for example
Traceback (most recent call last):
  File "/opt/local/bin/pylint-2.7", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/opt/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3019, in <module>
    @_call_aside
  File "/opt/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3003, in _call_aside
    f(*args, **kwargs)
  File "/opt/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3032, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/opt/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 655, in _build_master
    ws.require(__requires__)
  File "/opt/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 963, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/opt/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 849, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'backports.functools_lru_cache' distribution was not found and is required by pylint

A missing dependency? Seems easy to fix.

# pip install backports.functools_lru_cache
Collecting backports.functools_lru_cache
  Downloading backports.functools_lru_cache-1.3-py2.py3-none-any.whl
Installing collected packages: backports.functools-lru-cache
Successfully installed backports.functools-lru-cache-1.3
  1. lets try again
Traceback (most recent call last):
  File "/opt/local/bin/pylint-2.7", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/opt/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3019, in <module>
    @_call_aside
  File "/opt/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3003, in _call_aside
    f(*args, **kwargs)
  File "/opt/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3032, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/opt/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 655, in _build_master
    ws.require(__requires__)
  File "/opt/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 963, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/opt/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 849, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'configparser' distribution was not found and is required by pylint

More deps missing, ok then...

# pip install configparser
Collecting backports.functools_lru_cache
  Downloading backports.functools_lru_cache-1.3-py2.py3-none-any.whl
Installing collected packages: backports.functools-lru-cache
Successfully installed backports.functools-lru-cache-1.3
  1. we try again
Traceback (most recent call last):
  File "/opt/local/bin/pylint-2.7", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/opt/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3019, in <module>
    @_call_aside
  File "/opt/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3003, in _call_aside
    f(*args, **kwargs)
  File "/opt/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3032, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/opt/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 655, in _build_master
    ws.require(__requires__)
  File "/opt/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 963, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/opt/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 849, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'mccabe' distribution was not found and is required by pylint

# pip install mccabe
Collecting mccabe
  Downloading mccabe-0.6.1-py2.py3-none-any.whl
Installing collected packages: mccabe
Successfully installed mccabe-0.6.1

I think something in the dependency tree is broken, not necessarily in saltpylint though... we soldier on.

  1. we try again...
Traceback (most recent call last):
  File "/opt/local/bin/pylint-2.7", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/opt/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3019, in <module>
    @_call_aside
  File "/opt/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3003, in _call_aside
    f(*args, **kwargs)
  File "/opt/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3032, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/opt/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 655, in _build_master
    ws.require(__requires__)
  File "/opt/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 963, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/opt/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 849, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'isort>=4.2.5' distribution was not found and is required by pylint

:/

# pip install  'isort>=4.2.5'
Collecting isort>=4.2.5
  Downloading isort-4.2.5-py2.py3-none-any.whl (40kB)
    100% |################################| 40kB 977kB/s 
Installing collected packages: isort
Successfully installed isort-4.2.5
  1. rinse and repeat
Traceback (most recent call last):
  File "/opt/local/bin/pylint-2.7", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/opt/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3019, in <module>
    @_call_aside
  File "/opt/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3003, in _call_aside
    f(*args, **kwargs)
  File "/opt/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3032, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/opt/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 657, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/opt/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 670, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/opt/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 849, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'astroid<1.5.0,>=1.4.5' distribution was not found and is required by pylint

Wait hold on? py27-astroid-1.3.6 got pulled in by py27-pylint-1.6.1 ...
Well uninstall the package and install it via pip then ...

# pkgin remove py27-astroid-1.3.6
2 packages to delete: 

py27-pylint-1.6.1 py27-astroid-1.3.6

proceed ? [Y/n] y
removing py27-pylint-1.6.1...
removing py27-astroid-1.3.6...
pkg_install warnings: 0, errors: 0
reading local summary...
processing local summary...
# pip install  'astroid<1.5.0,>=1.4.5'
Collecting astroid<1.5.0,>=1.4.5
  Downloading astroid-1.4.9-py2.py3-none-any.whl (213kB)
    100% |################################| 215kB 1.6MB/s 
Collecting lazy-object-proxy (from astroid<1.5.0,>=1.4.5)
  Downloading lazy-object-proxy-1.2.2.tar.gz
Collecting wrapt (from astroid<1.5.0,>=1.4.5)
  Downloading wrapt-1.10.10.tar.gz
Requirement already satisfied: six in /opt/local/lib/python2.7/site-packages (from astroid<1.5.0,>=1.4.5)
Building wheels for collected packages: lazy-object-proxy, wrapt
  Running setup.py bdist_wheel for lazy-object-proxy ... done
  Stored in directory: /root/.cache/pip/wheels/06/d4/af/71508d0c69292f8c369ba3ae83cd85fabcd6956fd5709633b4
  Running setup.py bdist_wheel for wrapt ... done
  Stored in directory: /root/.cache/pip/wheels/22/a5/9f/ca577d0e7b917b681fdbb7b36a41a8a9ae76fb7fec146438dc
Successfully built lazy-object-proxy wrapt
Installing collected packages: lazy-object-proxy, wrapt, astroid
Successfully installed astroid-1.4.9 lazy-object-proxy-1.2.2 wrapt-1.10.10
# pip install pylint
Collecting pylint
  Downloading pylint-1.6.5-py2.py3-none-any.whl (577kB)
    100% |################################| 583kB 969kB/s 
Requirement already satisfied: six in /opt/local/lib/python2.7/site-packages (from pylint)
Requirement already satisfied: isort>=4.2.5 in /opt/local/lib/python2.7/site-packages (from pylint)
Requirement already satisfied: mccabe in /opt/local/lib/python2.7/site-packages (from pylint)
Requirement already satisfied: configparser; python_version == "2.7" in /opt/local/lib/python2.7/site-packages (from pylint)
Requirement already satisfied: backports.functools-lru-cache; python_version == "2.7" in /opt/local/lib/python2.7/site-packages (from pylint)
Requirement already satisfied: astroid<1.5.0,>=1.4.5 in /opt/local/lib/python2.7/site-packages (from pylint)
Requirement already satisfied: lazy-object-proxy in /opt/local/lib/python2.7/site-packages (from astroid<1.5.0,>=1.4.5->pylint)
Requirement already satisfied: wrapt in /opt/local/lib/python2.7/site-packages (from astroid<1.5.0,>=1.4.5->pylint)
Installing collected packages: pylint
Successfully installed pylint-1.6.5

(All the other deps above did not have packages, so that is why they also came from pip)

  1. lets try again, success
    pylint --rcfile=.testing.pylintrc salt/modules/logadm.py

Setup

pkgsrc:

  • py27-OpenSSL-16.0.0
  • py27-Tk-2.7.12nb4
  • py27-apache-libcloud-0.17.0
  • py27-asn1-0.1.9
  • py27-astroid-1.3.6
  • py27-backports.ssl_match_hostname-3.5.0.1
  • py27-backports_abc-0.4
  • py27-certifi-2015.11.20.1
  • py27-cffi-1.9.1
  • py27-cherrypy-8.1.0
  • py27-cparser-2.17
  • py27-crypto-2.6.1nb3
  • py27-cryptography-1.7.1
  • py27-curl-7.19.3.1nb2
  • py27-dateutil-2.5.3
  • py27-enum34-1.1.6
  • py27-expat-2.7.12
  • py27-gnupg-0.3.2nb2
  • py27-idna-2.1
  • py27-ipaddress-1.0.17
  • py27-jinja2-2.8
  • py27-logilab-common-1.2.0
  • py27-m2crypto-0.22.3nb1
  • py27-markupsafe-0.23
  • py27-msgpack-0.4.6
  • py27-mysql56db-1.2.5
  • py27-pep8-1.6.2
  • py27-pip-9.0.1
  • py27-pylint-1.6.1
  • py27-requests-2.12.4
  • py27-setuptools-32.1.0
  • py27-setuptools_scm-1.15.0
  • py27-simplejson-3.8.2
  • py27-singledispatch-3.4.0.3
  • py27-six-1.10.0
  • py27-sqlite3-2.7.12nb8
  • py27-tornado-4.3
  • py27-yaml-3.12
  • py27-zmq-16.0.2

pip:

  • SaltPyLint (2017.3.9)

Steps to Reproduce Issue

pylint-2.7 --rcfile=.testing.pylintrc salt/modules/logadm.py
@mamash mamash self-assigned this Mar 20, 2017
@mamash
Copy link

mamash commented Mar 20, 2017

Yep, it doesn't work in 2016Q4, as confirmed in joyent/pkgsrc/issues/454. I'd fixed that in trunk, but typically only security fixes go back into LTS.

jsonn@6665f9e

@jperkin
Copy link
Collaborator

jperkin commented Mar 20, 2017

If it's safe enough we should backport it, assuming all the DEPENDS can be satisfied.

@mamash
Copy link

mamash commented Mar 20, 2017

It's two new packages added and one updated.

@mamash
Copy link

mamash commented Mar 20, 2017

I didn't realize it's actually being used by a toolchain like this, thought it's a manual dev side tool. I'll see about a backport for 2016Q4 since we're carrying that branch around for some time.

@sjorge
Copy link
Author

sjorge commented Mar 20, 2017 •
edited
Loading

Ha I didn't realize I already reported a simular one before! Well seems more than one dep was bad in 2016Q4 for py-lint.

And yep, a few python bits use it here and they causing other stuff to break to.

@mamash
Copy link

mamash commented Mar 20, 2017

I backported the update to 2016Q4 today, have a go at it please.

@sjorge
Copy link
Author

sjorge commented Mar 21, 2017

Works fine now after redeploying!

@sjorge sjorge closed this as completed Mar 21, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mamash mamash

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jperkin @mamash @sjorge