Request for translate.py to error when requested model checkpoint is unavailable

[bhartmoore]

With the update to remove OpenNMT, silnlp no longer throws an error if the requested fine-tuned model checkpoint is unavailable. Instead, it outputs a well-formatted inference from the base model, which will contain garbage in most cases because the target language is not available in the base model. Because we now clean up models after just 30 days, this situation may arise more frequently. It would be helpful if translate.py could throw an error or warning when the requested model checkpoint is not found. Even having the ClearML job fail would be preferable to having it complete from the base model for an unknown language.

[isaac091]

It does already throw a warning, WARNING - Model has no checkpoints. Using base model.. However, it's easy to miss as just one line of text, so I think it would make sense to have it fail completely.

Sometimes for development tasks it's nice to be able to run the script without needing a trained model, but we could add a "base" or "0" checkpoint option for that.

[bhartmoore]

Thanks, Isaac! Looking back, I see that warning in the log from drafts I ran that hit this issue. I'm open to whatever solution makes sense for both development and user support.