Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rounding error with f64::from_str_radix #198

Closed
TatriX opened this issue Oct 13, 2020 · 1 comment · Fixed by #201
Closed

Rounding error with f64::from_str_radix #198

TatriX opened this issue Oct 13, 2020 · 1 comment · Fixed by #201

Comments

@TatriX
Copy link

TatriX commented Oct 13, 2020

Consider the following program:

use num::Num;

fn main() {
    let y = "2.89".parse::<f64>();
    println!("{:?}", y);
    let x = f64::from_str_radix("2.89", 10);
    println!("{:?}", x);
}

It prints:

Ok(2.89)
Ok(2.8899999999999997)

Am I missing anything here or results should be equal?

@cuviper
Copy link
Member

cuviper commented Oct 14, 2020

Well, 2.89 is not a number that can be precisely represented in binary floating point. If you force it to print more digits like `"{:.30?}", you'll get:

Ok(2.890000000000000124344978758018)
Ok(2.889999999999999680255768907955)

We can also print to_bits():

0100000000000111000111101011100001010001111010111000010100011111
0100000000000111000111101011100001010001111010111000010100011110

So parse did indeed choose a closer approximation, and yes this look like a rounding error.

We could special case radix=10 to use parse, but it would be nicer to improve this for all bases.

@cuviper cuviper transferred this issue from rust-num/num Jan 6, 2021
@bors bors bot closed this as completed in 9c8f20e Mar 2, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants