Skip to content

Commit

Permalink
cEP-0031: Improve Generic Bear Quality
Browse files Browse the repository at this point in the history 
Closes #183
  • Loading branch information
@bkhanale
bkhanale committed May 16, 2019
1 parent 93c7cce commit a1736da
Showing 1 changed file with 239 additions and 0 deletions.
239 changes: 239 additions & 0 deletions cEP-0031.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,239 @@
# Improve Generic Bear Quality

| Metadata | |
| -------- | ------------------------------------------- |
| cEP | 31 |
| Version | 0.1 |
| Title | Improve Generic Bear Quality |
| Authors | Bhushan Khanale <mailto:[email protected]> |
| Status | Proposed |
| Type | Feature |

## Abstract

This cEP describes the improvement in the generic bears currently available
along with the implementation of the new bears as a part of the
[GSoC 2019 Project][project].

## Introduction

coala has few generic bears which have the potential to perform better
fixing some of the issues in them. Some of these issues include [#2092][2092],
[#644][644], [#502][502], [#1897][1897], [#227][227].
Along with fixing these issues, new bears including `OutdatedDependencyBear`,
`FileExistsBear`, `RegexLintBear`, `FileModeBear` and `RequirementsCheckBear`
will be implemented.

This project also introduces changes in order to interlink `PEP8Bear` and
`PycodestyleBear` as a part of the [#551][551] proposal.

## Implementation of new bears

### 1. OutdatedDependencyBear

Issue: [coala/coala-bears#2445][2445]

New functions to the existing package manager would be the best way of
implementing this.

For pip requirements, we could use the PyPI JSON API which provides the
version info by specifying the `package_name`.

```py
class PipRequirement(PackageRequirement):
def get_version_info(self):
# This would get the package info in json format
url = 'https://pypi.python.org/pypi/%s/json' % (self.package,)

# Get the required data
data = json.load()
versions = data['releases'].keys().sort(key = StrictVersion)
return versions
```

For npm dependencies, we can use the npm cli command `npm outdated` which
produces an output as below:

```
$ npm outdated
Package Current Wanted Latest Location
glob 5.0.15 5.0.15 6.0.1 test-outdated-output
nothingness 0.0.3 git git test-outdated-output
npm 3.5.1 3.5.2 3.5.1 test-outdated-output
local-dev 0.0.3 linked linked test-outdated-output
once 1.3.2 1.3.3 1.3.3 test-outdated-output
```

The output can be directly parsed and can be used to get the list of all
outdated dependencies.

### 2. FileExistBear

Issue: [coala/coala-bears#527][527]

The bear will be responsible to check if the files exist in the project
directory. The name of the files can be passed as a regex to the bear, to
which the bear will check their existence.

```py
class FileExistBear(LocalBear):
def run(self, required_files: tuple,):
for file in required_files:
if not os.path.is_file():
message = 'File {} not found.'.format(file)
yield Result.from_values(origin=self, message=message,)
```

### 3. RegexLintBear

Issue: [coala/coala-bears#1532][1532]

The task of the bear is to check the regex in strings. This can be done using
the AnnotationBear to detect all the regex strings and then check for the valid
regex through the prepared algorithm for each type.

For Python, the bear can be written as follows:

```py
def is_regex(code, language):
"""
Returns true if the string is a regex.
"""

class RegexBear(LocalBear):
def run(self, file, filename, language,)
strings = run_local_bear(AnnotationBear)
for string in strings:
if is_regex(string, language):
try:
re.compile(string)
except re.error as Exception:
message = 'This is not a valid regex.'
yield Result.from_values(origin=self, message=message)
```

### 4. FileModeBear

Issue: [coala/coala-bears#2370][2370]

The bear will check the permissions on the files provided by the user
and to ensure that the file permissions are the one that is expected.

The bear will be used as follows:

```
[all.mode]
bears = FileModeBear
filemode = rw
[all.shell.mode]
bears = FileModeBear
filemode = rwx
```

The bear will first find the permissions of the files specified by the user and
then if the permissions are not suitable the bear will try to change those
permissions into the expected ones. If the bear doesn't have enough permissions
to do so then the bear will let the user know about this.

```py
class FileModeBear(LocalBear):
def run(self, file, filename, filemode,):
st = os.stat(filename)
st_read = bool(st.st_mode & stat.S_IRUSR)
st_write = bool(st.st_mode & stat.S_IWUSR)

mode = {'r': st_read, 'w': st_write}
ch_mode = {'r': stat.S_IRUSR, 'w': stat.S_IWUSR}

# Similarly, by bitwise operations on the st_mode, the relevant
# information can be found out.

change_mode = 0
for char in filemode:
if not mode[char]:
change_mode += ch_mode[char]
if not change_mode:
os.chmod(change_mode)
```

### 5. RequirementsCheckBear

Issue: [coala/coala-bears#1113][1113]

The bear will be focused on Python only since they are most prone for
conflicting requirements.

The implementation is based on the recursive check for the requirements of each
package in requirement.txt file of the project. The requirements are checked
through PyPI's JSON API.

```py
class RequirementsCheckBear(LocalBear):
def collect_requirements():
"""
This will get the requirements through the requirements.txt file and
through requirements folder and pip files.
"""
packages = get_packages()

for package in packages:
requirements = get_requirements(package)
packages.append(r for r in requirements)

return packages

def get_requirements(package_name):
"""
This will get the requirements of a package through the PyPI's JSON API.
"""
url = 'https://pypi.python.org/pypi/%s/json' % (package_name,)

data = requests.get(url).json()

return data['info']['requires_dist']

def run(self, file, filaname,):
packages = get_requirements()
conflicts = detect_conflicts()
if conflicts:
for conflict in conflicts:
message = '{} is conflicting package.'.format(conflict)
yield Result.from_values(origin=self, message=message,)
```

## Issues to be solved

### Pycodestyle - PEP8Bear Integration

Issues covered:

- [coala/coala-bears#1897][1897]
- [coala/projects#551][551]
- [hhatto/autopep8#227][227]

Since the autopep8 upstream is most likely not have any major improvements,
we could have the improvements to the PEP8Bear. Since autopep8 uses Pycodestyle
for error detection, we could integrate the Pycodestyle with PEP8Bear.

Doing this will involve invoking PEP8Bear after running PycodestyleBear.

To implement this, we should be able to pass the selection of the issues
through the `--line-range` and `--select` options available in autopep8.
If multiple similar issues are available in the same line-range then we should
implement the `--column-range` option in autopep8 which will allow us to
pass the appropriate issue to generate a fix for.

This will allow the user to generate a fix for only selected issues.

[project]: https://summerofcode.withgoogle.com/projects/#4866569388163072
[227]: https://github.com/hhatto/autopep8/issues/227
[502]: https://github.com/coala/coala-bears/issues/502
[527]: https://github.com/coala/coala-bears/issues/527
[551]: https://github.com/coala/projects/issues/551
[644]: https://github.com/coala/coala-bears/issues/644
[1113]: https://github.com/coala/coala-bears/issues/1113
[1532]: https://github.com/coala/coala-bears/issues/1532
[1897]: https://github.com/coala/coala-bears/issues/1897
[2092]: https://github.com/coala/coala-bears/issues/2092
[2445]: https://github.com/coala/coala-bears/issues/2445

0 comments on commit a1736da

Please sign in to comment.