Adds dependency on sacremoses tokenizer (#183)

* Added dependency on sacremoses * [CodeBuild] * [CodeBuild] * [CodeBuild]
exasol · Jan 30, 2024 · 9231e61 · 9231e61
1 parent 53f485c
commit 9231e61
Show file tree

Hide file tree

Showing 3 changed files with 32 additions and 106 deletions.
diff --git a/doc/changes/changes_0.9.0.md b/doc/changes/changes_0.9.0.md
@@ -18,6 +18,7 @@ T.B.D
 ### Refactorings
 
  - #182: Removed torch package index from pyproject.toml
+ - #139: Installs Sacremoses tokenizer
 
 ### Documentation