Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add --reset_learning_rate option to lstmtraining #3470

Merged
merged 1 commit into from
Jun 28, 2021

Conversation

nagadomi
Copy link
Contributor

When the --reset_learning_rate option is specified, it resets the learning rate stored in each layer of the network loaded with --continue_from to the value specified by the --learning_rate option.
If checkpoint is available, it does nothing.

background:

Currently, the -learning_rate option specified when finetuning with the lstmtraining --continue_from ... command is ignored. Instead, the learning rate stored in the pretrained model will be used. You can check its value with the combine_tessdata -l command.
The learning rate stored in the pretrained model is different for each language, and the behavior of finetuning is also different.
So, if the user has experience in machine learning, it is useful in some situations to be able to manually override the learning rate.

# finetuning with a low learning rate
lstmtraining --learning_rate 0.0001 --reset_learning_rate --continue_from ...

# training from a high learning rate using the pretraiend model as initial parameters
lstmtraining --learning_rate 0.001 --reset_learning_rate --continue_from ...

# learning rate decay manually
lstmtraining --learning_rate 0.0003 --reset_learning_rate --model_output ./lr1 --continue_from ...
lstmtraining --learning_rate 0.0002 --reset_learning_rate --model_output ./lr2 --continue_from ./lr1_checkpoint ...
lstmtraining --learning_rate 0.0001 --reset_learning_rate --model_output ./lr3 --continue_from ./lr2_checkpoint ...
lstmtraining --learning_rate 0.00001 --reset_learning_rate --model_output ./final --continue_from ./lr3_checkpoint ...

When the --reset_learning_rate option is specified,
it resets the learning rate stored in each layer of the network
loaded with --continue_from to the value specified by the --learning_rate option.
If checkpoint is available, it does nothing.
@egorpugin egorpugin merged commit ff1062d into tesseract-ocr:master Jun 28, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants