diff --git a/README.md b/README.md index ee00a93a9..61055b804 100644 --- a/README.md +++ b/README.md @@ -25,3 +25,4 @@ in the `cEP-0000.md` document. | [cEP-0022](cEP-0022.md) | coala-quickstart Green Mode | This cEP provides the features, needs, requirements and implementation of the green-mode being added to coala-quickstart. | | [cEP-0025](cEP-0025.md) | Integrate pyflakes AST into coala | This cEP describes the details of the process of integrating the pyflakes-enhanced-AST into coala as part of [GSoC'18 project](https://summerofcode.withgoogle.com/projects/#5549789140221952). | | [cEP-0027](cEP-0027.md) | coala Bears Testing API | This cEP describes the implementation process of `BaseTestHelper` class and `GlobalBearTestHelper` class to improve testing API of coala as a part of the [GSoC'18 project](https://summerofcode.withgoogle.com/projects/#6625036551585792). | +| [cEP-0031](cEP-0031.md) | Improve Generic Bear Quality | This cEP describes the improvement in the generic bears currently available along with the implementation of the new bears as a part of the [GSoC'19 project](https://summerofcode.withgoogle.com/projects/#4866569388163072). | diff --git a/cEP-0031.md b/cEP-0031.md new file mode 100644 index 000000000..87533c4d8 --- /dev/null +++ b/cEP-0031.md @@ -0,0 +1,254 @@ +# Improve Generic Bear Quality + +| Metadata | | +| -------- | ------------------------------------------- | +| cEP | 31 | +| Version | 0.1 | +| Title | Improve Generic Bear Quality | +| Authors | Bhushan Khanale | +| Status | Proposed | +| Type | Feature | + +## Abstract + +This cEP describes the improvement in the generic bears currently available +along with the implementation of the new bears as a part of the +[GSoC 2019 Project][project]. + +## Introduction + +coala has few generic bears which have the potential to perform better +fixing some of the issues in them. Some of these issues are as follows: + +- [coala/coala-bears#2092][2092]: Pygmented AnnotationBear + Pygments is a Python library for syntax highlighting of source code. + Therefore it contains complete syntax definitions of any relevant + programming language and a custom parser framework for tokenizing source + code of all those languages. This gives the opportunity of reducing the + current complex implementation of the bear into a much simpler one. + +- [coala/coala-bears#644][644]: Ignore doc comments/doc strings + Comments or docstrings do not have any set standards while the + IndentationBear does not ignore them in a file. IndentationBear checks for + correct indentation in the program statements since that is crucial for + proper code style. Docstrings/comments, on the other hand, do not have any + such compulsion. So ideally docstrings should be ignored by the bear. coala + already uses documentation extraction algorithms in DocumentationStyleBear. + Using the existing algorithms and constructing ignore ranges out of the + extracted comments inside the IndentationBear, this issue can be solved. + +- [coala/coala-bears#502][502]: MarkdownBear is not very helpful + On running MarkdownBear all we get is the result message 'The text does not + comply to the set style.'. This is not very helpful for most users. We will + be using a plugin to solve this issue. + +- [coala/coala-bears#1897][1897]: Show PEP8 error description + On running PEP8Bear the default message is 'The code does not comply to + PEP8.' which is not very helpful for the user. This issue will be solved + by implementing upstream changes to autopep8. + +- [hhatto/autopep8#227][227]: Create diff for a specific pep8 issue + User should have a choice of selecting the issue he wants to generate a fix + for. Although the issue is a part of the autopep8, we will be solving + this as a part of upstream changes to autopep8. + +Along with fixing these issues, new bears including `OutdatedDependencyBear`, +`FileExistsBear`, `RegexLintBear`, `FileModeBear` and `RequirementsCheckBear` +will be implemented. + +## Implementation of new bears + +### 1. OutdatedDependencyBear + +Issue: [coala/coala-bears#2445][2445] + +New functions to the existing package manager would be the best way of +implementing this. + +- For pip requirements, we could use the PyPI JSON API which provides the + version info by specifying the `package_name`. + +```py +class PipRequirement(PackageRequirement): + def get_version_info(self): + # This would get the package info in json format + url = 'https://pypi.python.org/pypi/%s/json' % (self.package,) + + # Get the required data + data = json.load() + versions = data['releases'].keys().sort(key = StrictVersion) + return versions +``` + +- For npm dependencies, we can use the npm cli command `npm outdated` which + produces an output as below: + +``` +$ npm outdated +Package Current Wanted Latest Location +glob 5.0.15 5.0.15 6.0.1 test-outdated-output +nothingness 0.0.3 git git test-outdated-output +npm 3.5.1 3.5.2 3.5.1 test-outdated-output +local-dev 0.0.3 linked linked test-outdated-output +once 1.3.2 1.3.3 1.3.3 test-outdated-output +``` + +The output can be directly parsed and can be used to get the list of all +outdated dependencies. + +### 2. FileExistBear + +Issue: [coala/coala-bears#527][527] + +The bear will be responsible to check if the files exist in the project +directory. The name of the files can be passed as a regex to the bear, to +which the bear will check their existence. + +```py +class FileExistBear(LocalBear): + def run(self, required_files: tuple,): + for file in required_files: + if not os.path.is_file(): + message = 'File {} not found.'.format(file) + yield Result.from_values(origin=self, message=message,) +``` + +### 3. RegexLintBear + +Issue: [coala/coala-bears#1532][1532] + +The task of the bear is to check the regex in strings. This can be done using +the AnnotationBear to detect all the regex strings and then check for the valid +regex through the prepared algorithm for each type. + +For Python, the bear can be written as follows: + +```py +def is_regex(code, language): + """ + Returns true if the string is a regex. + """ + +class RegexLintBear(LocalBear): + def run(self, file, filename, language,): + strings = run_local_bear(AnnotationBear) + for string in strings: + if is_regex(string, language): + try: + re.compile(string) + except re.error as Exception: + message = 'This is not a valid regex.' + yield Result.from_values(origin=self, message=message) +``` + +### 4. FileModeBear + +Issue: [coala/coala-bears#2370][2370] + +The bear will check the permissions on the files provided by the user +and to ensure that the file permissions are the one that is expected. + +The bear will be used as follows: + +``` +[all.mode] +bears = FileModeBear +filemode = rw + +[all.shell.mode] +bears = FileModeBear +filemode = rwx +``` + +The bear will first find the permissions of the files specified by the user and +then if the permissions are not suitable, the bear will try to change those +permissions into the expected ones. If the bear doesn't have enough permissions +to do so then the bear will let the user know about this. + +```py +class FileModeBear(LocalBear): + def run(self, file, filename, filemode,): + stats = os.stat(filename) + stats_read = bool(stats.st_mode & stat.S_IRUSR) + stats_write = bool(stats.st_mode & stat.S_IWUSR) + + mode = {'r': stats_read, 'w': stats_write} + ch_mode = {'r': stat.S_IRUSR, 'w': stat.S_IWUSR} + + # Similarly, by bitwise operations on the st_mode, the relevant + # information can be found out. + + change_mode = 0 + for char in filemode: + if not mode[char]: + change_mode += ch_mode[char] + if not change_mode: + os.chmod(change_mode) +``` + +### 5. RequirementsCheckBear + +Issue: [coala/coala-bears#1113][1113] + +The bear will be focused on Python only since they are most prone for +conflicting requirements. + +The implementation is based on the recursive check for the requirements of each +package in requirement.txt file of the project. The requirements are checked +through PyPI's JSON API. + +```py +class RequirementsCheckBear(LocalBear): + def collect_requirements(): + """ + This will get the requirements through the requirements.txt file and + through requirements folder and pip files. + """ + packages = get_packages() + + for package in packages: + requirements = get_requirements(package) + packages.append(r for r in requirements) + + return packages + + def get_requirements(package_name): + """ + This will get the requirements of a package through the PyPI's JSON API. + """ + url = 'https://pypi.python.org/pypi/%s/json' % (package_name,) + + data = requests.get(url).json() + + return data['info']['requires_dist'] + + def run(self, file, filaname,): + packages = get_requirements() + conflicts = detect_conflicts() + if conflicts: + for conflict in conflicts: + message = '{} is conflicting package.'.format(conflict) + yield Result.from_values(origin=self, message=message,) +``` + +[project]: https://summerofcode.withgoogle.com/projects/#4866569388163072 + +[227]: https://github.com/hhatto/autopep8/issues/227 + +[502]: https://github.com/coala/coala-bears/issues/502 + +[527]: https://github.com/coala/coala-bears/issues/527 + +[551]: https://github.com/coala/projects/issues/551 + +[644]: https://github.com/coala/coala-bears/issues/644 + +[1113]: https://github.com/coala/coala-bears/issues/1113 + +[1532]: https://github.com/coala/coala-bears/issues/1532 + +[1897]: https://github.com/coala/coala-bears/issues/1897 + +[2092]: https://github.com/coala/coala-bears/issues/2092 + +[2445]: https://github.com/coala/coala-bears/issues/2445