On the Accuracy of Division Calculation #2093
Answered
by
frankfliu
huaizhe2012
asked this question in
Q&A
-
|
Hello, My data precision is eight decimal places, but after division, the precision becomes four. What can I do to keep its precision，thanks. |
Beta Was this translation helpful? Give feedback.
Answered by
frankfliu
Oct 20, 2022
Replies: 1 comment 1 reply
-
|
This is just display issue. DJL trim the trailing 0s when print the NDArray. You can dump the NDArray using |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
This is really a display issue，Thank you very much |
Beta Was this translation helpful? Give feedback.
Answer selected by
huaizhe2012
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This is just display issue. DJL trim the trailing 0s when print the NDArray.
You can dump the NDArray using
NDArray.toNumber()to verify the precision.