Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Inconsistent results with trainer.test and Inference.infer under different batch sizes. #178

Closed
xinghai-sun opened this issue Jul 26, 2017 · 1 comment
Closed

Inconsistent results with trainer.test and Inference.infer under different batch sizes. #178

xinghai-sun opened this issue Jul 26, 2017 · 1 comment
Assignees
@xinghai-sun @qingqing01

Comments

@xinghai-sun
Copy link
Contributor

xinghai-sun commented Jul 26, 2017 •
edited
Loading

  1. Inconsistent inference results with Inference.infer under different batch sizes.
  2. Small inference difference between trainer.test and Inference.infer.

Experiments are run with DeepSpeech2 on Paddle.
@xinghai-sun
Copy link
Contributor Author

xinghai-sun commented Jul 26, 2017 •
edited
Loading

We concluded with the following two reasons which might have caused the inconsistency:

  1. Due to the different padding sizes with different batch sizes. (It is verified by inspecting the output data.)
  2. Due to the different float point precision: trainer.test computes the average loss under float32 within c++ codes, while inference.infer does the averaging under float64 in Python codes.

@xinghai-sun xinghai-sun mentioned this issue Jul 26, 2017
Closed
17 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@xinghai-sun xinghai-sun

@qingqing01 qingqing01

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@xinghai-sun @qingqing01