Improve num_mismatch filter #123

jelmervdl · 2023-10-05T09:13:17Z

Fixes #98.

If you want to play with the regex: https://regex101.com/r/choHwO/1

Questions I have though:

I added a boundary test at the end of the number to skip numbers in text like 30something. But this now also ignores 2nd.
I removed the condition that there had to be numbers on both sides of the bitext to even consider looking at the ratio.

I also want to be able to debug positive matches

jelmervdl added 6 commits October 4, 2023 15:09

Fix imports and types in col.py

714a96b

Be more strict about columns in col.py

0a9f12a

Add more tests for num_mismatch

2942d02

Expand and document the expression used in num_mismatch

268bf04

Don't ignore lines that have only digits on one side

abe136a

Print debug info regardless of whether the line was skipped or not

16decdc

I also want to be able to debug positive matches

jelmervdl marked this pull request as ready for review October 27, 2023 13:37

jelmervdl merged commit 64ecad2 into main Oct 27, 2023
5 checks passed

jelmervdl deleted the num-mismatch branch October 27, 2023 13:38

Provide feedback