Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using the "detokenizer" filter rule gives an error #143

Open
bhaddow opened this issue Jan 10, 2024 · 1 comment
Open

Using the "detokenizer" filter rule gives an error #143

bhaddow opened this issue Jan 10, 2024 · 1 comment
Labels
bug Something isn't working component:filter Related to or suggestion for a new filter block

Comments

@bhaddow
Copy link
Contributor

bhaddow commented Jan 10, 2024

How to reproduce:

Install the requirements-all.txt

Load a corpus, and navigate to filters

Attempt to add the "detokenizer" rule.

The following error is produced:

Usage: sacremoses [OPTIONS] COMMAND [ARGS]...
Try 'sacremoses -h' for help.

Error: No such option: -l
Error: subprocess exited with status code 2
@bhaddow bhaddow changed the title Using the "detokenizer" rule gives an error Using the "detokenizer" filter rule gives an error Jan 10, 2024
@jelmervdl jelmervdl added bug Something isn't working component:filter Related to or suggestion for a new filter block labels Jan 12, 2024
@jelmervdl
Copy link
Collaborator

That's so weird, all sacremoses commands should be able to accept an -l option.

Maybe it has something to do with python dependencies, where the sacremoses command is installed by another package that's incompatible.

Additionally, (but more relevant for another issue) we should have a test suite that installs requirements-all.txt to see whether everything still works in that environment.

Additionally additionally, integration tests for individual filters would be nice, but quite a bit of (easy but boring) work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working component:filter Related to or suggestion for a new filter block
Projects
None yet
Development

No branches or pull requests

2 participants