Next step in pylint's gamification and user experience

[Pierre-Sassoulas]

This is a follow up to #2399 and relate to #5403. It was discussed in detail in #4538 (comment)

Right now when starting to use pylint the score can be really bad and we might be dumping thousands of warning on users. This is demoralizing, even for code quality oriented persons and hurting adoption for legacy projects.

The experience I have right now in starting to use pylint on a code base is overwhelming, I have to use pylint .|grep -o "([a-z\-]*)$"|cut -d "(" -f2|sort|uniq|sed s/\)/,/ to disable all the existing warning and start from somewhere. Then I need to think hard before enabling them again to know where to start. The way I see it the pylint guide would add the disable automatically, maybe in pylintrc, maybe in code directly, then it would rank the warning and tell you exactly what to do next and provide a feeling of progression with level up and "encouragements". Congratulation you do not have any error level warning, Congratulation the code follow SOLID principles, Congratulation we do not have any performance improvement to provide, Congratulation your code did not regress after upgrading to pylint 2.10.1, you were already that good, Congratulation you've got a perfect code according to pylint's opinion etc. A lot of variables to follow that could be in the pylintrc.

The first step is Implementing baseline so only new message are raised : #5403. This way pylint can be adopted easily on a legacy codebase.

Then I think we could create a pylint "guide". What I mean by that is upgrading the pylintrc step by step to fix the legacy messages starting by the more significant one (error, then vicious warning, then warning, then important convention, then opinionated convention, then adding extensions...). This way there would be a clear path to clean code and nothing overwhelming along the way. This would permit to add "levels" and "achievements" and improve gamification. The score do not need to be bad, it can be a good score for this level (i.e. good jobs there are no errors, do you want to increase the quality of your code ?). Pylint would be friendlier and it would help user to increase their code base quality.

[Hornwitser]

The experience I had using Pylint on an existing codebase was 1500 messages being reported and no way to sort through it, no way to determine what has changed or compare with previous runs and no way to filter by the messages or modules I want to work on. I ended up writing an html interface to display the reports based on the JSON output that has all of these features.

[nickdrozd]

Perhaps this would be a good time to revisit the topic of what's enabled by default. It would be interesting to know what checks are typically disabled. For myself, I always disable the ones that require docstrings. In my experience those warnings account for a lot of the noise.

[Pierre-Sassoulas]

@nickdrozd you can discuss this in #3512