use unicode_script crate to check chars are Latin script inside is_english_lingual

[hippietrail]

I noticed is_english_lingual() doesn't check that words are in the Latin script so it would match any 'alphabet' except Chinese, Japanese, and Korean.

Unicode character properties are finer-grained than Unicode blocks. And if new extension blocks are added you don't have to modify the code. I left the other logic in there but some of it may now be redundant.

Note that typst, which I don't know much about but seems to have some link to Harper also uses the unicode_script crate.

[hippietrail]

The build errors are something to do with "manifest keys"??

[elijah-potter]

The build errors are something to do with "manifest keys"??

When you add a dependency, cargo needs to add an entry to the Cargo.lock manifest file. Make sure that file is added to your git stage before you commit changes to dependencies.

Also, would you mind making sure your commits match the practices laid out in our documentation?
Particularly, using Conventional Commits.

[elijah-potter]

Hey @hippietrail, I feel pretty confident merging this. Could you give me a couple examples that replicate the issue so I can add them as test cases?

[hippietrail]

Hey @hippietrail, I feel pretty confident merging this. Could you give me a couple examples that replicate the issue so I can add them as test cases?

These say "This is in Greek/Georgian/Thai" in those languages:

Αυτό είναι στα ελληνικά.
ეს ქართულად.
นี่มันภาษาไทย

This is English with misstakes.

Without the change looks like this:

With it it's like this:

[hippietrail]

I should note two things:

• This does not handle the rare case of words that have letters from multiple scripts. I believe those will always get split at script-change boundaries. In English my guess is this will pop up from time to time with Greek letters.
• This means all Latin extended letters can be included in English words, such as Polish, Icelandic, obsolete letters, etc. But the spellchecker will flag them so not a problem.