Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reject near-boundary and NaN values for Float-to-Int conversions #11230

Merged

Conversation

HertzDevil
Copy link
Contributor

@HertzDevil HertzDevil commented Sep 18, 2021

Fixes #10421. Fixes #11105.

2147483648_f32.to_i32     # raises OverflowError
Float32::NAN.to_i32       # raises OverflowError
Float32::INFINITY.to_u128 # raises OverflowError

None of the above expressions currently raise.

@HertzDevil HertzDevil added kind:bug A bug in the code. Does not apply to documentation, specs, etc. topic:compiler:codegen topic:stdlib:numeric labels Sep 18, 2021
@HertzDevil
Copy link
Contributor Author

It seems more of compiler-rt's methods need to be ported manually if we want 128-bit integer support on Windows...

@HertzDevil HertzDevil changed the title Be more strict with float-to-integer conversions Reject near-boundary and NaN values for Float-to-Int conversions Sep 19, 2021
@HertzDevil HertzDevil marked this pull request as draft September 19, 2021 05:56
@HertzDevil HertzDevil marked this pull request as ready for review September 19, 2021 09:13
spec/compiler/codegen/arithmetics_spec.cr Outdated Show resolved Hide resolved
src/compiler/crystal/codegen/primitives.cr Outdated Show resolved Hide resolved
begin
{{float_type}}::NAN.{{method}}
results |= 1
rescue OverflowError
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm having a hard time to understand why trying to convert a NaN to an int would be an overflow. It's not out of bounds like an overflow, a NaN just can't be represented as an integer (makes no sense), and reporting an overflow feels confusing.

Ruby raises a FloatDomainError (NaN) exception for example.

In plain Crystal I'd merely raise ArgumentError.new("NaN") if nan?.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe OverflowError could be interpreted more generically to also include conversion errors where the value can't be represented in the target type.

But that's probably not a good idea because it would apply to many other conversion errors as well. So yeah, ArgumentError seems like a good idea.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no OverflowError in Ruby; instead they call it RangeError, and then make FloatDomainError a subclass of it. That means Ruby code that catches overflow errors will also catch domain errors:

begin
  (0.0 / 0.0).to_i
rescue RangeError
  puts 1 # => 1
end

I think we should copy this or simply rename OverflowError to RangeError.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still thinking about the best way to approach this. For performance reasons (for the compiler) it seems we should mimic OverflowError:

# :nodoc:
fun __crystal_raise_nan : NoReturn
  raise ArgumentError.new("not-a-number")
end

including making it a "well-known function".

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO overflow and conversion failure are the same class of error. I would use the same error class for both. Currently, we have OverflowError. Maybe that name doesn't fit very well with conversion failure. But I'd still suggest to use that. We can see to maybe rename OverflowError to a better name that also describes conversion failures (MathError? ArithmeticError?). It seems easier to go with the error type we already have instead of using a different one and then later introducing a new concept to reunite them. Alternatively, we could introduce a new error type as a parent for OverflowError (and maybe merger OverflowError into that later).

ArgumentError doesn't seem to fit for this.

@straight-shoota straight-shoota added this to the 1.3.0 milestone Dec 6, 2021
@HertzDevil HertzDevil marked this pull request as draft December 8, 2021 08:07
@HertzDevil HertzDevil marked this pull request as ready for review December 8, 2021 08:55
@straight-shoota straight-shoota removed this from the 1.3.0 milestone Dec 8, 2021
Comment on lines +38 to +52
it "raises overflow if equal to Int::MAX (#11105)" do
# these examples hold because the integer would be rounded _up_ to the
# nearest representable float

expect_raises(OverflowError) { Float32.new!(Int32::MAX).to_i32 }
expect_raises(OverflowError) { Float32.new!(UInt32::MAX).to_u32 }
expect_raises(OverflowError) { Float32.new!(Int64::MAX).to_i64 }
expect_raises(OverflowError) { Float32.new!(UInt64::MAX).to_u64 }
expect_raises(OverflowError) { Float32.new!(Int128::MAX).to_i128 }

expect_raises(OverflowError) { Float64.new!(Int64::MAX).to_i64 }
expect_raises(OverflowError) { Float64.new!(UInt64::MAX).to_u64 }
expect_raises(OverflowError) { Float64.new!(Int128::MAX).to_i128 }
expect_raises(OverflowError) { Float64.new!(UInt128::MAX).to_u128 }
end
Copy link
Member

@oprypin oprypin Dec 16, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This spec's definition is kinda circular.
Also would be nice to test that it doesn't overflow right below the cutoff.
I'm not sure if my suggestion is better, let me know what you think.

Suggested change
it "raises overflow if equal to Int::MAX (#11105)" do
# these examples hold because the integer would be rounded _up_ to the
# nearest representable float
expect_raises(OverflowError) { Float32.new!(Int32::MAX).to_i32 }
expect_raises(OverflowError) { Float32.new!(UInt32::MAX).to_u32 }
expect_raises(OverflowError) { Float32.new!(Int64::MAX).to_i64 }
expect_raises(OverflowError) { Float32.new!(UInt64::MAX).to_u64 }
expect_raises(OverflowError) { Float32.new!(Int128::MAX).to_i128 }
expect_raises(OverflowError) { Float64.new!(Int64::MAX).to_i64 }
expect_raises(OverflowError) { Float64.new!(UInt64::MAX).to_u64 }
expect_raises(OverflowError) { Float64.new!(Int128::MAX).to_i128 }
expect_raises(OverflowError) { Float64.new!(UInt128::MAX).to_u128 }
end
describe "handles overflow near Int::MAX (#11105)" do
describe "for Float32" do
it "doesn't overflow below" do
(n = 2.1474835e+9_f32).to_i32.should eq(n)
(n = 4.294967e+9_f32).to_u32.should eq(n)
(n = 9.2233715e+18_f32).to_i64.should eq(n)
(n = 1.8446743e+19_f32).to_u64.should eq(n)
(n = 1.7014117e+38_f32).to_i128.should eq(n)
(n = Float32::MAX).to_u128.should eq(n)
end
it "overflows above" do
expect_raises(OverflowError) { 2.1474836e+9_f32.to_i32 }
expect_raises(OverflowError) { 4.2949673e+9_f32.to_u32 }
expect_raises(OverflowError) { 9.223372e+18_f32.to_i64 }
expect_raises(OverflowError) { 1.8446744e+19_f32.to_u64 }
expect_raises(OverflowError) { 1.7014118e+38_f32.to_i128 }
end
end
describe "for Float64" do
it "doesn't overflow below" do
(n = 9.223372036854775e+18).to_i64.should eq(n)
(n = 1.844674407370955e+19).to_u64.should eq(n)
(n = 1.7014118346046921e+38).to_i128.should eq(n)
(n = 3.4028236692093843e+38).to_u128.should eq(n)
end
it "overflows above" do
expect_raises(OverflowError) { 9.223372036854776e+18.to_i64 }
expect_raises(OverflowError) { 1.8446744073709552e+19.to_u64 }
expect_raises(OverflowError) { 1.7014118346046923e+38.to_i128 }
expect_raises(OverflowError) { 3.402823669209385e+38.to_u128 }
end
end
end

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you resolved one of my two concerns. But, I do think the idea of writing as exact number literals should be seriously considered. Because, if both the Float32.new! part and the .to_u64 part break at the same time, it's still possible for this spec to pass. I will not push very hard on this, but curious to see some feedback.

@straight-shoota straight-shoota added this to the 1.3.0 milestone Dec 20, 2021
@straight-shoota straight-shoota merged commit 2297d89 into crystal-lang:master Dec 21, 2021
@HertzDevil HertzDevil deleted the bug/float-to-int-overflow branch December 21, 2021 13:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind:bug A bug in the code. Does not apply to documentation, specs, etc. topic:compiler:codegen topic:stdlib:numeric
Projects
None yet
4 participants