Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

modernization of proselint #1361

Closed
wants to merge 249 commits into from
Closed
Changes from 1 commit
Commits
Show all changes
249 commits
Select commit Hold shift + click to select a range
140b3e0
added check for punctuation spacing. Issue #1342
arostkowycz15 Apr 17, 2023
451801f
Added clichés from Nigel Rees' book
mdgjohnny Sep 10, 2023
67be7fc
Tests for Nigel's clichés
mdgjohnny Sep 10, 2023
9db9d31
fix file endings and remove not needed spaces
orgua Dec 27, 2023
8defcc4
raise min py-version to 3.8 and remove future
orgua Dec 27, 2023
237ff5e
update deps
orgua Dec 27, 2023
98ce3d2
py: add commas
orgua Dec 27, 2023
9083fef
clean imports
orgua Dec 27, 2023
623d684
lint code
orgua Dec 27, 2023
f7d62d8
fix programming error
orgua Dec 27, 2023
fe731d0
chore(deps): bump ruby/setup-ruby from 1.95.0 to 1.163.0
dependabot[bot] Dec 27, 2023
e07daf6
chore(deps): bump codecov/codecov-action from 2.1.0 to 3.1.4
dependabot[bot] Dec 27, 2023
6ffbff7
chore(deps): bump actions/setup-python from 2 to 5
dependabot[bot] Dec 27, 2023
24e4032
chore(deps): bump actions/cache from 2 to 3
dependabot[bot] Dec 27, 2023
ac146da
chore(deps-dev): bump pytest from 6.2.5 to 7.4.3
dependabot[bot] Dec 27, 2023
0dc599c
Merge pull request #5 from orgua/dependabot/github_actions/actions/ca…
orgua Dec 27, 2023
d1ec014
Merge pull request #4 from orgua/dependabot/github_actions/actions/se…
orgua Dec 27, 2023
f3ab5cc
Merge pull request #6 from orgua/dependabot/pip/pytest-7.4.3
orgua Dec 27, 2023
791fbd7
chore(deps): bump actions/checkout from 2 to 4
dependabot[bot] Dec 27, 2023
53fe09f
Merge pull request #2 from orgua/dependabot/github_actions/actions/ch…
orgua Dec 27, 2023
f28b461
Merge pull request #1 from orgua/dependabot/github_actions/ruby/setup…
orgua Dec 27, 2023
2ef0a38
Merge pull request #3 from orgua/dependabot/github_actions/codecov/co…
orgua Dec 27, 2023
4aaeea6
do not pin package-versions if not needed
orgua Dec 27, 2023
62cbc93
clean up usage of paths
orgua Dec 27, 2023
873014f
remove legacy-code
orgua Dec 27, 2023
dbae823
avoid shadowing a Python builtins
orgua Dec 27, 2023
1e39e4d
sort imports
orgua Dec 28, 2023
b2ab749
add typehinting
orgua Dec 28, 2023
b24f5b1
format with black-style, ruff
orgua Dec 31, 2023
21947b0
add proper logging capability
orgua Dec 31, 2023
1af69f2
refactor internal paths
orgua Dec 31, 2023
64d4d6d
refactor cache
orgua Dec 31, 2023
147aee1
lint
orgua Dec 31, 2023
f2b91d6
refactor cache
orgua Dec 31, 2023
dd5a0f0
add chromalog and update deps
orgua Dec 31, 2023
54668e0
refactor logger
orgua Dec 31, 2023
c66ef43
various refactoring
orgua Dec 31, 2023
873022e
allow benchmarking & fix bugs
orgua Dec 31, 2023
5d8638a
small bugfixes
orgua Jan 1, 2024
8722878
chore(deps): bump ruby/setup-ruby from 1.163.0 to 1.165.1
dependabot[bot] Jan 1, 2024
6cbd9fe
chore(deps): bump crazy-max/ghaction-github-labeler from 3 to 5
dependabot[bot] Jan 1, 2024
f364082
replace memoiziation, improve code around it
orgua Jan 1, 2024
87f5832
consider age of cache-entries
orgua Jan 1, 2024
52f574e
optimize slowest funcs
orgua Jan 1, 2024
8fea336
remove tests for compiled-file-deletion
orgua Jan 1, 2024
c8368e7
refactor for modularity
orgua Jan 1, 2024
17fd756
last transformations to list comprehensions
orgua Jan 1, 2024
c24211b
Update poetry.lock
orgua Jan 1, 2024
2e468c7
refactor unittests (wip)
orgua Jan 2, 2024
bd5a3d1
remove duplicates
orgua Jan 2, 2024
458aacc
reformat
orgua Jan 2, 2024
625349c
Update config_default.py
orgua Jan 2, 2024
9f73969
refactor unittests
orgua Jan 2, 2024
ada3e7b
move tests for checks in separate folder
orgua Jan 2, 2024
666be5a
allow disabling cache
orgua Jan 2, 2024
3163e3c
small renaming & formatting
orgua Jan 2, 2024
567a517
extending sys.path not needed
orgua Jan 2, 2024
b33bc8f
propagate module-path
orgua Jan 2, 2024
69ffeab
refactor web-scripts
orgua Jan 2, 2024
8930e92
refactor new scripts
orgua Jan 2, 2024
644382a
refactor and reformat scripts
orgua Jan 2, 2024
45f657b
add cache-tests
orgua Jan 2, 2024
dae83c8
add config-tests
orgua Jan 2, 2024
f2dd797
add note to benchmark
orgua Jan 2, 2024
0c6ac6f
add printer for invoke()
orgua Jan 2, 2024
ce47da9
revert renaming txt-file for tests
orgua Jan 2, 2024
af65ec7
update configs in sync to available tests
orgua Jan 2, 2024
4d35420
fix tests
orgua Jan 2, 2024
2055bec
catch missing modules, make ppm-wrapper nicer
orgua Jan 2, 2024
7dc2f58
add exithandler and return number of results as exit-code
orgua Jan 2, 2024
4943d6d
Update test_cache_function.py
orgua Jan 2, 2024
ebef8e0
Update test_config_default.py
orgua Jan 2, 2024
2e5c2f4
workflows make version to string
orgua Jan 2, 2024
12e68ba
Merge pull request #12 from orgua/dependabot/github_actions/crazy-max…
orgua Jan 2, 2024
5f86848
Merge pull request #11 from orgua/dependabot/github_actions/ruby/setu…
orgua Jan 2, 2024
fb72dd6
sort dev-deps, also include pre-commit and remove packages covered by…
orgua Jan 2, 2024
1d69912
Update ci-lint-test.yml
orgua Jan 2, 2024
a67a7f4
Merge branch 'main' of https://github.com/orgua/proselint-modernized
orgua Jan 2, 2024
59d1eec
add multiprocessing for checks
orgua Jan 3, 2024
fb09191
ditch chromalog-package
orgua Jan 3, 2024
1bc6276
parallelize lint()
orgua Jan 3, 2024
e45b917
remove memoizer for checks
orgua Jan 3, 2024
a3b78a1
extend benchmark scripts
orgua Jan 3, 2024
82f671c
extend cache to handle const-properties
orgua Jan 3, 2024
44bfd7e
refactor sexism-check and extend unittests
orgua Jan 3, 2024
edb8b5f
minor fixes
orgua Jan 3, 2024
d2fa1b8
refactor more checks and tests
orgua Jan 3, 2024
4a845b2
fix check_option join & require_padding
orgua Jan 3, 2024
4bfb81a
even out load of checks for extreme cases
orgua Jan 4, 2024
471b241
add pytest-cov dependency
orgua Jan 4, 2024
efeb957
fix check-tests
orgua Jan 4, 2024
956f943
fix test for caching
orgua Jan 4, 2024
b87d194
remove const-cache after benchmarking
orgua Jan 4, 2024
201d7ee
refactor all multiprocessed code into submodule
orgua Jan 4, 2024
769123f
clean up from latest debug session
orgua Jan 4, 2024
7ef28ce
change config
orgua Jan 4, 2024
464f41b
fix unittests
orgua Jan 4, 2024
ccb6c8a
linting
orgua Jan 4, 2024
ccbcf7a
cache is now a singleton
orgua Jan 4, 2024
4cb1813
Update tools.py
orgua Jan 4, 2024
dc4da55
Update test_hyperlinks_on_shell.py
orgua Jan 4, 2024
f4a9fc2
add padding to text - guard against edge-cases
orgua Jan 4, 2024
8a78b9c
rename lint_cache
orgua Jan 4, 2024
f764def
force single-line imports
orgua Jan 4, 2024
7a5e4c3
add global executor and optimize memoizer
orgua Jan 4, 2024
560eabd
optimize check-message
orgua Jan 4, 2024
1de71f7
lint code
orgua Jan 4, 2024
0d87b24
add current ruff-config
orgua Jan 4, 2024
92d72f4
overhaul output-format system
orgua Jan 5, 2024
741639d
store filename in error-list
orgua Jan 5, 2024
7bd74e0
fix bug in memoizer
orgua Jan 5, 2024
87c23a6
improve memoizer
orgua Jan 5, 2024
8a4aa20
fix padding in some checks
orgua Jan 5, 2024
6ef3a1a
improve commercialese
orgua Jan 5, 2024
f3d3657
improve memoizer
orgua Jan 5, 2024
52e445a
Update score.py
orgua Jan 5, 2024
ceb06ee
add weasel words
orgua Jan 5, 2024
e72d054
fix unittests
orgua Jan 5, 2024
2632d2b
speed up scripts
orgua Jan 5, 2024
a3e9af2
move printing out of linter
orgua Jan 5, 2024
e49149e
Merge branch 'pr/1356'
orgua Jan 5, 2024
413cf62
Update test_cliches.py
orgua Jan 5, 2024
5a552be
fix code
orgua Jan 5, 2024
4b09ec2
fix unittests and lint
orgua Jan 5, 2024
ac40919
bump version
orgua Jan 14, 2024
7548c80
Merge branch 'main' into main
Nytelife26 Jan 22, 2024
3fbaaef
ci(lint-test): fix pytest
Nytelife26 Jan 24, 2024
d3213d6
fix file endings and remove not needed spaces
orgua Dec 27, 2023
6eb6949
raise min py-version to 3.8 and remove future
orgua Dec 27, 2023
a04b80b
update deps
orgua Dec 27, 2023
b552b74
py: add commas
orgua Dec 27, 2023
691a145
clean imports
orgua Dec 27, 2023
4c155b1
lint code
orgua Dec 27, 2023
1434a98
fix programming error
orgua Dec 27, 2023
912e965
chore(deps): bump actions/cache from 2 to 3
dependabot[bot] Dec 27, 2023
f14ce3a
chore(deps): bump actions/setup-python from 4 to 5
dependabot[bot] Dec 27, 2023
6d6d995
chore(deps-dev): bump pytest from 6.2.5 to 7.4.3
dependabot[bot] Dec 27, 2023
a0f8bc7
chore(deps): bump actions/checkout from 2 to 4
dependabot[bot] Dec 27, 2023
eefe9f1
chore(deps): bump ruby/setup-ruby from 1.95.0 to 1.163.0
dependabot[bot] Dec 27, 2023
03b91ab
do not pin package-versions if not needed
orgua Dec 27, 2023
75cd2b7
clean up usage of paths
orgua Dec 27, 2023
41ac3bd
remove legacy-code
orgua Dec 27, 2023
a0e43e8
avoid shadowing a Python builtins
orgua Dec 27, 2023
4b1e5c7
sort imports
orgua Dec 28, 2023
3988867
add typehinting
orgua Dec 28, 2023
18d22c0
format with black-style, ruff
orgua Dec 31, 2023
c4fdcfa
add proper logging capability
orgua Dec 31, 2023
391bafb
refactor internal paths
orgua Dec 31, 2023
938f157
refactor cache
orgua Dec 31, 2023
31cc235
lint
orgua Dec 31, 2023
ce94896
refactor cache
orgua Dec 31, 2023
2e74b61
add chromalog and update deps
orgua Dec 31, 2023
fa9f6cd
refactor logger
orgua Dec 31, 2023
a07fa34
various refactoring
orgua Dec 31, 2023
12bd14a
allow benchmarking & fix bugs
orgua Dec 31, 2023
d97811a
small bugfixes
orgua Jan 1, 2024
7e62d8b
replace memoiziation, improve code around it
orgua Jan 1, 2024
d41ca24
consider age of cache-entries
orgua Jan 1, 2024
3b51220
optimize slowest funcs
orgua Jan 1, 2024
fcc7d29
remove tests for compiled-file-deletion
orgua Jan 1, 2024
f0dc9cf
refactor for modularity
orgua Jan 1, 2024
b58f140
last transformations to list comprehensions
orgua Jan 1, 2024
6cffecf
refactor unittests (wip)
orgua Jan 2, 2024
457c832
remove duplicates
orgua Jan 2, 2024
3da41d8
reformat
orgua Jan 2, 2024
74a726c
Update config_default.py
orgua Jan 2, 2024
58b7a42
refactor unittests
orgua Jan 2, 2024
b32d853
move tests for checks in separate folder
orgua Jan 2, 2024
c1bfeea
allow disabling cache
orgua Jan 2, 2024
b8e7048
small renaming & formatting
orgua Jan 2, 2024
870671e
extending sys.path not needed
orgua Jan 2, 2024
fc8a356
propagate module-path
orgua Jan 2, 2024
7bb9d3e
refactor web-scripts
orgua Jan 2, 2024
f7fa09f
refactor new scripts
orgua Jan 2, 2024
4e4a596
refactor and reformat scripts
orgua Jan 2, 2024
9e0307e
add cache-tests
orgua Jan 2, 2024
54631fb
add config-tests
orgua Jan 2, 2024
cb23bb9
add note to benchmark
orgua Jan 2, 2024
05871d9
add printer for invoke()
orgua Jan 2, 2024
ffecc43
revert renaming txt-file for tests
orgua Jan 2, 2024
53dc973
update configs in sync to available tests
orgua Jan 2, 2024
5828288
fix tests
orgua Jan 2, 2024
14beecb
catch missing modules, make ppm-wrapper nicer
orgua Jan 2, 2024
f53c55e
add exithandler and return number of results as exit-code
orgua Jan 2, 2024
a13cc04
Update test_cache_function.py
orgua Jan 2, 2024
c1db241
Update test_config_default.py
orgua Jan 2, 2024
ac334fa
workflows make version to string
orgua Jan 2, 2024
f81bf0b
sort dev-deps, also include pre-commit and remove packages covered by…
orgua Jan 2, 2024
dd86494
Update ci-lint-test.yml
orgua Jan 2, 2024
0e7d620
chore(deps): bump crazy-max/ghaction-github-labeler from 3 to 5
dependabot[bot] Jan 1, 2024
1829d4b
chore(deps): bump ruby/setup-ruby from 1.163.0 to 1.165.1
dependabot[bot] Jan 1, 2024
66ab691
add multiprocessing for checks
orgua Jan 3, 2024
e7d70b2
ditch chromalog-package
orgua Jan 3, 2024
7ef4d60
parallelize lint()
orgua Jan 3, 2024
fd96434
remove memoizer for checks
orgua Jan 3, 2024
1ae3926
extend benchmark scripts
orgua Jan 3, 2024
9d677fe
extend cache to handle const-properties
orgua Jan 3, 2024
cf6384f
refactor sexism-check and extend unittests
orgua Jan 3, 2024
25e1a9d
minor fixes
orgua Jan 3, 2024
98a7980
refactor more checks and tests
orgua Jan 3, 2024
cae7728
fix check_option join & require_padding
orgua Jan 3, 2024
22bf2fc
even out load of checks for extreme cases
orgua Jan 4, 2024
5eab18c
add pytest-cov dependency
orgua Jan 4, 2024
f323c5d
fix check-tests
orgua Jan 4, 2024
1768bb5
fix test for caching
orgua Jan 4, 2024
78a703f
remove const-cache after benchmarking
orgua Jan 4, 2024
c177a65
refactor all multiprocessed code into submodule
orgua Jan 4, 2024
d9cdfd3
clean up from latest debug session
orgua Jan 4, 2024
521b076
change config
orgua Jan 4, 2024
a302e4c
fix unittests
orgua Jan 4, 2024
aaba845
linting
orgua Jan 4, 2024
e3b7b02
cache is now a singleton
orgua Jan 4, 2024
998d0f5
Update tools.py
orgua Jan 4, 2024
6744baf
Update test_hyperlinks_on_shell.py
orgua Jan 4, 2024
e4535c1
add padding to text - guard against edge-cases
orgua Jan 4, 2024
b03c870
rename lint_cache
orgua Jan 4, 2024
23a54f4
force single-line imports
orgua Jan 4, 2024
e294602
add global executor and optimize memoizer
orgua Jan 4, 2024
c4ecb77
optimize check-message
orgua Jan 4, 2024
4c1016e
lint code
orgua Jan 4, 2024
5e2ea1b
add current ruff-config
orgua Jan 4, 2024
3c44e3b
overhaul output-format system
orgua Jan 5, 2024
0a32450
store filename in error-list
orgua Jan 5, 2024
771788d
fix bug in memoizer
orgua Jan 5, 2024
f53a531
improve memoizer
orgua Jan 5, 2024
d7d78f6
fix padding in some checks
orgua Jan 5, 2024
529efbe
improve commercialese
orgua Jan 5, 2024
5ef1d75
improve memoizer
orgua Jan 5, 2024
52da9f3
Update score.py
orgua Jan 5, 2024
94956d5
add weasel words
orgua Jan 5, 2024
7b1a232
fix unittests
orgua Jan 5, 2024
a2c2d48
speed up scripts
orgua Jan 5, 2024
d4bf985
move printing out of linter
orgua Jan 5, 2024
a8af783
Added clichés from Nigel Rees' book
mdgjohnny Sep 10, 2023
185c344
Tests for Nigel's clichés
orgua Jan 5, 2024
c1d67df
added check for punctuation spacing. Issue #1342
arostkowycz15 Apr 17, 2023
a60cc86
fix unittests and lint
orgua Jan 5, 2024
84a40ab
bump version
orgua Jan 14, 2024
b3ebed3
ci(lint-test): fix pytest
Nytelife26 Jan 24, 2024
90d610f
chore: fix merge differences
Nytelife26 Jan 25, 2024
2104094
chore: resolve rebase issues
Nytelife26 Jan 25, 2024
64bb526
chore(deps): add ruff
Nytelife26 Jan 25, 2024
fc5e10a
lint: fix ruff suggestions
Nytelife26 Jan 25, 2024
30f1972
ci(lint-test): lint with ruff
Nytelife26 Jan 25, 2024
cefb7a3
chore: fix lint issues
Nytelife26 Jan 25, 2024
c1cc4db
Merge branch 'main' of https://github.com/orgua/proselint-modernized
orgua Jan 25, 2024
ee7dbf4
Update tests/check.py
orgua Jan 25, 2024
fa9b0cd
implement comments
orgua Jan 25, 2024
eb414d1
Merge branch 'main' of https://github.com/orgua/proselint-modernized
orgua Jan 25, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
store filename in error-list
orgua committed Jan 5, 2024
commit 741639d24891d343746c981635993e8ab38a83a3
14 changes: 9 additions & 5 deletions proselint/checks/__init__.py
Original file line number Diff line number Diff line change
@@ -29,7 +29,7 @@
###############################################################################


def run_checks(_check: Callable, _text: str) -> list:
def run_checks(_check: Callable, _text: str, source: str = "") -> list:
# TODO: frozenset is hashable (list without duplicates) -> check-list, result-lists
# padding
# -> some checks expect words in text and need something around it
@@ -41,14 +41,18 @@ def run_checks(_check: Callable, _text: str) -> list:
(start, end, check_name, message, replacements) = result
(line, column) = get_line_and_column(_text, start)
if not is_quoted(start, _text):
# note:
# - switch to 1based counting by adding +1
Nytelife26 marked this conversation as resolved.
Show resolved Hide resolved
# - for line it cancels out with -1 from padding
errors += [
(
check_name,
message,
line - 1,
column,
start,
end,
source, # can't be Path, unless pickle changes
line, # +1 -1, cancel out
column + 1,
start + 1,
end + 1,
end - start,
"warning",
replacements,
1 change: 1 addition & 0 deletions proselint/config_base.py
Original file line number Diff line number Diff line change
@@ -8,6 +8,7 @@ class Output(IntEnum):
full = default = 0
json = 1
compact = 2
# todo: implement printing to file?

@classmethod
def names(cls) -> list[str]:
48 changes: 22 additions & 26 deletions proselint/tools.py
Original file line number Diff line number Diff line change
@@ -30,8 +30,8 @@
from .memoizer import cache
from .memoizer import memoize_lint

ResultLint: TypeAlias = tuple[str, str, int, int, int, int, int, str, str]
# content: check_name, message, line, column, start, end, length, type, replacement
ResultLint: TypeAlias = tuple[str, str, str, int, int, int, int, int, str, str]
# content: check_name, message, source, line, column, start, end, length, type, replacement


###############################################################################
@@ -149,7 +149,7 @@ def lint(
content: Union[str, IO],
config: Optional[dict] = None,
checks: Optional[list[Callable]] = None,
file_name: Optional[str] = None,
source: str = "",
*,
_exe: Optional[Executor] = None,
) -> list[ResultLint]:
@@ -177,7 +177,7 @@ def lint(
log.debug("[Lint] used outer Executor for parallelization")
# NOTE: ThreadPoolExecutor is only concurrent, but not multi-cpu
# NOTE: .map() is build on .submit(), harder to use here, same speed
futures = [_exe.submit(run_checks, check, _text) for check in checks]
futures = [_exe.submit(run_checks, check, _text, source) for check in checks]
if ret_future:
# this will skip the memoizer
return futures
@@ -236,7 +236,7 @@ def lint_path(
cache.data[key] = _errors
cache.age[key] = cache.ts_now
# todo: include filename in ResultLint? allows to collect all errors
print_errors(_file, _errors, config)
output_errors(_errors, config, _file)
error_num += len(_errors)

duration = time.time() - ts_start
@@ -256,25 +256,25 @@ def errors_to_json(items: list[ResultLint]) -> str:
{
"check": item[0],
"message": item[1],
"file": "todo",
"line": 1 + item[2],
"column": 1 + item[3],
"start": 1 + item[4],
"end": 1 + item[5],
"extent": item[6],
"severity": item[7],
"replacements": item[8],
"source": item[2],
"line": item[3],
"column": item[4],
"start": item[5],
"end": item[6],
"extent": item[7],
"severity": item[8],
"replacements": item[9],
}
for item in items
]

return json.dumps({"status": "success", "data": {"errors": out}}, sort_keys=True)


def print_errors(
filename: Union[Path, str],
def output_errors(
errors: list[ResultLint],
config: dict,
file_path: Optional[Path] = None,
) -> None:
"""Print the errors, resulting from lint, for filename."""
try:
@@ -289,6 +289,7 @@ def print_errors(
(
check,
message,
source,
line,
column,
_, # start,
@@ -298,19 +299,14 @@ def print_errors(
_, # replacements,
) = error

if isinstance(filename, Path):
if isinstance(file_path, Path):
if out_fmt == Output.compact:
filename = filename.name + "debug-WHYNOTHERE"
source = file_path.name
else:
filename = filename.absolute().as_uri()
source = file_path.absolute().as_uri()
# TODO: would be nice to supress "file:///"
# https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda
else:
if out_fmt == Output.compact:
filename = "" # TODO: using cli+compact switches here?!?
else:
filename = str(filename)
elif out_fmt == Output.compact:
source = ""

# note: adding +1 for switching to 1based counting
# todo: why not for json? move to creation
log.info("%s:%d:%d: %s %s", filename, 1 + line, 1 + column, check, message)
log.info("%s:%d:%d: %s %s", source, line, column, check, message)
orgua marked this conversation as resolved.
Show resolved Hide resolved