chore(deps): update module github.com/pemistahl/lingua-go to v1.4.0

[renovate]

This PR contains the following updates:

Package	Change	Age	Adoption	Passing	Confidence
github.com/pemistahl/lingua-go	v1.0.5 -> v1.4.0

---

Release Notes

pemistahl/lingua-go (github.com/pemistahl/lingua-go)

v1.4.0: Lingua 1.4.0

Compare Source

Features

• The new functions GetIsoCode639_1FromValue() and GetIsoCode639_3FromValue() have been introduced to return the proper IsoCode639_1 and IsoCode639_3 for a given name string. (#​44)

Changes

• The functions GetLanguageFromIsoCode639_1() and GetLanguageFromIsoCode639_3() now correctly return Unknown instead of -1 if a language cannot be found for the given iso code. (#​44)

Bug Fixes

• The method LanguageDetector.DetectMultipleLanguagesOf() returned wrong values for start and end indices for texts consisting of only a single word. This has been fixed. (#​43)

v1.3.4: Lingua 1.3.4

Compare Source

Bug Fixes

• When trying to detect multiple languages in a text consisting of only a single word, a panic occurred. This has been fixed. (#​41)

v1.3.3: Lingua 1.3.3

Compare Source

Bug Fixes

• For long input texts, a panic occurred while computing the confidence values due to an accidental division by zero. This has been fixed. (#​27)

v1.3.2: Lingua 1.3.2

Compare Source

Improvements

• After applying some internal optimizations, language detection is now faster, at least between 20% and 30%, approximately. For long input texts, the speed improvement is greater than for short input texts.

v1.3.1: Lingua 1.3.1

Compare Source

Bug Fixes

• For long input texts, an error occurred while computing the confidence values due to numerical underflow when converting probabilities. This has been fixed.

v1.3.0: Lingua 1.3.0

Compare Source

Improvements

• The min-max normalization method for the confidence values has been replaced with applying the softmax function. This gives more realistic probabilities. (#​25)

v1.2.2: Lingua 1.2.2

Compare Source

Bug Fixes

• Under certain circumstances, calling the method LanguageDetector.DetectMultipleLanguagesOf() caused an index error. This has been fixed.

v1.2.1: Lingua 1.2.1

Compare Source

Bug Fixes

• A misconfiguration in a go.mod file caused errors when trying to download the library via the go get command. This has been fixed. Thanks to @​BenStigsen for the pointer. (#​23)

v1.2.0: Lingua 1.2.0

Compare Source

Features

• The new method LanguageDetector.DetectMultipleLanguagesOf() has been introduced. It allows to detect multiple languages in mixed-language text. (#​9)

v1.1.1: Lingua 1.1.1

Compare Source

Documentation

• Some documentation mistakes have been fixed and missing information has been added.

v1.1.0: Lingua 1.1.0

Compare Source

Features

• The new method LanguageDetectorBuilder.WithLowAccuracyMode() has been introduced. By activating it, detection accuracy for short text is reduced in favor of a smaller memory footprint and faster detection performance. (#​17)

• The new method LanguageDetector.ComputeLanguageConfidence() has been introduced. It allows to retrieve the confidence value for one specific language only, given the input text. (#​19)

Improvements

• The computation of the confidence values has been revised and the min-max normalization algorithm is now applied to the values, making them better comparable by behaving more like real probabilities. (#​16)

• The language models are now serialized as protocol buffers instead of json. Thanks to this change, they are now loaded into memory twice as fast as before. (#​22)

Bug Fixes

• The unigram counts in the statistics engine were not retrieved correctly. This has been fixed, producing more correct detection results. (#​14)

Compatibility

• The lowest supported Go version is 1.18 now. Older versions are no longer compatible with this library.

Miscellaneous

• The library now has a fresh and colorful new logo. Why? Well, why not? (-:

---

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

---

• If you want to rebase/retry this PR, check this box

---

This PR was generated by Mend Renovate. View the repository job log.