Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ruff] Implement check for Decimal called with a float literal (RUF032) #12909

Merged
merged 20 commits into from
Aug 19, 2024

Conversation

kbaskett248
Copy link
Contributor

@kbaskett248 kbaskett248 commented Aug 15, 2024

Summary

This PR adds a new check for a Decimal call with a float argument. As described in #11840, this is a problem when precision is important, which is usually why Decimal is used in the first place. To pull an example by @mayanyax from the issue:

>>> Decimal(0.3) == Decimal(0.1) + Decimal(0.1) + Decimal(0.1)
False
>>> Decimal('0.3') == Decimal('0.1') + Decimal('0.1') + Decimal('0.1')
True

The check supports positive and negative floats. It also includes a fix that wraps the float in quotes.

Test Plan

Test cases were added for the new rule.

Copy link
Contributor

github-actions bot commented Aug 15, 2024

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

ℹ️ ecosystem check detected linter changes. (+25 -0 violations, +0 -0 fixes in 4 projects; 50 projects unchanged)

RasaHQ/rasa (+14 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview

+ tests/core/test_utils.py:41:29: RUF032 `Decimal()` called with float literal argument
+ tests/core/test_utils.py:49:28: RUF032 `Decimal()` called with float literal argument
+ tests/core/test_utils.py:49:52: RUF032 `Decimal()` called with float literal argument
+ tests/core/test_utils.py:49:77: RUF032 `Decimal()` called with float literal argument
+ tests/core/test_utils.py:55:52: RUF032 `Decimal()` called with float literal argument
+ tests/core/test_utils.py:55:76: RUF032 `Decimal()` called with float literal argument
+ tests/core/test_utils.py:56:40: RUF032 `Decimal()` called with float literal argument
+ tests/core/test_utils.py:71:18: RUF032 `Decimal()` called with float literal argument
+ tests/core/test_utils.py:77:19: RUF032 `Decimal()` called with float literal argument
+ tests/core/test_utils.py:77:33: RUF032 `Decimal()` called with float literal argument
+ tests/core/test_utils.py:77:48: RUF032 `Decimal()` called with float literal argument
+ tests/core/test_utils.py:80:43: RUF032 `Decimal()` called with float literal argument
+ tests/core/test_utils.py:80:57: RUF032 `Decimal()` called with float literal argument
+ tests/core/test_utils.py:80:82: RUF032 `Decimal()` called with float literal argument

apache/airflow (+3 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview --select ALL

+ tests/providers/amazon/aws/transfers/test_dynamodb_to_s3.py:94:21: RUF032 `Decimal()` called with float literal argument
+ tests/providers/google/cloud/transfers/test_cassandra_to_gcs.py:132:27: RUF032 `Decimal()` called with float literal argument
+ tests/www/views/test_views_trigger_dag.py:104:28: RUF032 `Decimal()` called with float literal argument

ibis-project/ibis (+1 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview

+ ibis/expr/datatypes/tests/test_value.py:48:26: RUF032 `Decimal()` called with float literal argument

pandas-dev/pandas (+7 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview

+ pandas/tests/dtypes/cast/test_downcast.py:36:39: RUF032 `Decimal()` called with float literal argument
+ pandas/tests/dtypes/cast/test_downcast.py:38:39: RUF032 `Decimal()` called with float literal argument
+ pandas/tests/dtypes/test_missing.py:326:21: RUF032 `Decimal()` called with float literal argument
+ pandas/tests/tools/test_to_numeric.py:195:40: RUF032 `Decimal()` called with float literal argument
+ pandas/tests/tools/test_to_numeric.py:210:31: RUF032 `Decimal()` called with float literal argument
+ pandas/tests/tools/test_to_numeric.py:210:60: RUF032 `Decimal()` called with float literal argument
+ pandas/tests/tools/test_to_numeric.py:213:37: RUF032 `Decimal()` called with float literal argument

Changes by rule (1 rules affected)

code total + violation - violation + fix - fix
RUF032 25 25 0 0 0

@MichaReiser MichaReiser added rule Implementing or modifying a lint rule preview Related to preview mode features labels Aug 16, 2024
Copy link
Member

@MichaReiser MichaReiser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice. I'd love to get some input from @AlexWaygood but this looks great


fn fix_float_literal(range: TextRange, float_literal: &str) -> Fix {
let content = format!("\"{float_literal}\"");
Fix::unsafe_edit(Edit::range_replacement(content, range))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's your reasoning for making it an unsafe edit. Is it because it changes the precision of the values and, therefore, could result in runtime changes?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think an unsafe edit is appropriate here... in most cases, you'll be changing the underlying value of the Decimal object being constructed. There might be code elsewhere that implicitly depended on the previous, "wrong" value that was caused by constructing the Decimal instance from the float, so it's entirely possible that fixing this error could unexpectedly cause your code to break.

@kbaskett248, it would be great if you could add one or two sentences to the docs explaining why this is an unsafe fix, like how we do with other rules.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honestly it was mostly unfamiliarity with the apis, but @AlexWaygood brings up a good point.

Copy link

codspeed-hq bot commented Aug 16, 2024

CodSpeed Performance Report

Merging #12909 will not alter performance

Comparing kbaskett248:rule/float-literal-decimal (9a8fd54) with main (65de8f2)

Summary

✅ 32 untouched benchmarks

@AlexWaygood
Copy link
Member

Merging #12909 will degrade performances by 5.27%

You can ignore that, it's been super flaky recently

@kbaskett248
Copy link
Contributor Author

Thanks for your help with this @AlexWaygood and @MichaReiser! Any other updates you'd like me to make?

Comment on lines 79 to 81
fn fix_float_literal(range: TextRange, float_literal: &str) -> Fix {
let content = format!("\"{float_literal}\"");
Fix::unsafe_edit(Edit::range_replacement(content, range))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One last thing. Here we should use the quote style that the user is generally using for the rest of their file, which won't necessarily be double quotes if they're not using autoformatting. We can do this with the checker.stylist():

Suggested change
fn fix_float_literal(range: TextRange, float_literal: &str) -> Fix {
let content = format!("\"{float_literal}\"");
Fix::unsafe_edit(Edit::range_replacement(content, range))
fn fix_float_literal(range: TextRange, float_literal: &str, stylist: &Stylist) -> Fix {
let quote = stylist.quote();
let content = format!("{quote}{float_literal}{quote}");
Fix::unsafe_edit(Edit::range_replacement(content, range))

You'll just want to pass checker.stylist() into this function at the callsite, and you'll want to add use ruff_python_codegen::Stylist; to the top of the file so that you can use it here in the type signature

@MichaReiser MichaReiser enabled auto-merge (squash) August 19, 2024 09:12
@AlexWaygood AlexWaygood force-pushed the rule/float-literal-decimal branch from a4964d9 to 9a8fd54 Compare August 19, 2024 09:17
@MichaReiser MichaReiser merged commit f4c8c7e into astral-sh:main Aug 19, 2024
17 checks passed
@kbaskett248
Copy link
Contributor Author

Thank you both for your help getting this in!

@AlexWaygood
Copy link
Member

Thank you both for your help getting this in!

Thanks for the great contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
preview Related to preview mode features rule Implementing or modifying a lint rule
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants