Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Show layer specified learning rates with combine_tessdata -l #3468

Merged
merged 1 commit into from
Jun 26, 2021

Conversation

nagadomi
Copy link
Contributor

combine_tessdata -l ... command shows the initial value of the learning rate, but not the layer (final)learning rates that are actually used in finetuning (lstmtraining with -continue_from option).
This PR will add layer learning rates to the output of combine_tessdata -l command.

Example output:

% ./combine_tessdata -l ../tessdata_best/eng.traineddata
LSTM: network=[1,36,0,1Ct3,3,16Mp3,3Lfys64Lfx96Lrx96Lfx512O1c1], int_mode=0, recoding=1, iteration=814100, sample_iteration=814136, null_char=110, learning_rate=0.001, momentum=0.5, adam_beta=0.999
Layer Learning Rates: :0(Input)=0.001, :1:0(Convolve)=0.001, :1:1(ConvNL)=0.0005, :2(Maxpool)=0.001, :3:0(Lfys64)=0.0005, :4(Lfx96)=0.0005, :5:0(Lrx96)=0.0005, :6(Lfx512)=0.0005, :7(Output)=0.0005
% ./combine_tessdata -l ../tessdata_best/jpn_vert.traineddata              
LSTM: network=[1,48,0,1Ct3,3,16Mp3,3Lfys64Lfx96Lrx96Lfx512O1c1], int_mode=0, recoding=1, iteration=1874000, sample_iteration=1878622, null_char=414, learning_rate=0.001, momentum=0.5, adam_beta=0.999
Layer Learning Rates: :0(Input)=0.001, :1:0(Convolve)=0.001, :1:1(ConvNL)=0.000125, :2(Maxpool)=0.001, :3:0(Lfys64)=0.000125, :4(Lfx96)=0.000125, :5:0(Lrx96)=0.000125, :6(Lfx512)=0.000125, :7(Output)=0.000125

In this output, we can see that the learning rate of jpn_vert has been decayed to 0.000125.
This information may be useful for debugging when finetuning does not work very well, or for determining the learning rate when doing a full training.

@egorpugin egorpugin merged commit b2fa77f into tesseract-ocr:master Jun 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants