-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[flake8-type-checking
] Support auto-quoting when annotations contain quotes
#11811
Changes from 16 commits
65a38e9
c8df851
d97e2bd
8ca1f5c
29e0d4a
1b58828
14f51e3
e8c0f90
c6ef3db
75f4407
3a253f6
bf363e2
f27ae89
2b58cbb
fbf0bb1
5079a8c
f7e19b4
26081a6
5c4b4df
d4602a2
114edd0
ff973d4
2c10896
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,7 @@ | ||
use anyhow::Result; | ||
use ast::str::Quote; | ||
use ast::visitor::source_order; | ||
use ruff_python_ast::visitor::source_order::SourceOrderVisitor; | ||
use std::cmp::Reverse; | ||
|
||
use ruff_diagnostics::Edit; | ||
|
@@ -9,7 +12,6 @@ use ruff_python_codegen::{Generator, Stylist}; | |
use ruff_python_semantic::{ | ||
analyze, Binding, BindingKind, Modules, NodeId, ResolvedReference, ScopeKind, SemanticModel, | ||
}; | ||
use ruff_source_file::Locator; | ||
use ruff_text_size::Ranged; | ||
|
||
use crate::rules::flake8_type_checking::settings::Settings; | ||
|
@@ -221,9 +223,7 @@ pub(crate) fn is_singledispatch_implementation( | |
pub(crate) fn quote_annotation( | ||
Glyphack marked this conversation as resolved.
Show resolved
Hide resolved
|
||
node_id: NodeId, | ||
semantic: &SemanticModel, | ||
locator: &Locator, | ||
stylist: &Stylist, | ||
generator: Generator, | ||
) -> Result<Edit> { | ||
let expr = semantic.expression(node_id).expect("Expression not found"); | ||
if let Some(parent_id) = semantic.parent_expression_id(node_id) { | ||
|
@@ -233,51 +233,42 @@ pub(crate) fn quote_annotation( | |
// If we're quoting the value of a subscript, we need to quote the entire | ||
// expression. For example, when quoting `DataFrame` in `DataFrame[int]`, we | ||
// should generate `"DataFrame[int]"`. | ||
return quote_annotation(parent_id, semantic, locator, stylist, generator); | ||
return quote_annotation(parent_id, semantic, stylist); | ||
} | ||
} | ||
Some(Expr::Attribute(parent)) => { | ||
if expr == parent.value.as_ref() { | ||
// If we're quoting the value of an attribute, we need to quote the entire | ||
// expression. For example, when quoting `DataFrame` in `pd.DataFrame`, we | ||
// should generate `"pd.DataFrame"`. | ||
return quote_annotation(parent_id, semantic, locator, stylist, generator); | ||
return quote_annotation(parent_id, semantic, stylist); | ||
} | ||
} | ||
Some(Expr::Call(parent)) => { | ||
if expr == parent.func.as_ref() { | ||
// If we're quoting the function of a call, we need to quote the entire | ||
// expression. For example, when quoting `DataFrame` in `DataFrame()`, we | ||
// should generate `"DataFrame()"`. | ||
return quote_annotation(parent_id, semantic, locator, stylist, generator); | ||
return quote_annotation(parent_id, semantic, stylist); | ||
} | ||
} | ||
Some(Expr::BinOp(parent)) => { | ||
if parent.op.is_bit_or() { | ||
// If we're quoting the left or right side of a binary operation, we need to | ||
// quote the entire expression. For example, when quoting `DataFrame` in | ||
// `DataFrame | Series`, we should generate `"DataFrame | Series"`. | ||
return quote_annotation(parent_id, semantic, locator, stylist, generator); | ||
return quote_annotation(parent_id, semantic, stylist); | ||
} | ||
} | ||
_ => {} | ||
} | ||
} | ||
|
||
// If the annotation already contains a quote, avoid attempting to re-quote it. For example: | ||
// ```python | ||
// from typing import Literal | ||
// | ||
// Set[Literal["Foo"]] | ||
// ``` | ||
let text = locator.slice(expr); | ||
if text.contains('\'') || text.contains('"') { | ||
return Err(anyhow::anyhow!("Annotation already contains a quote")); | ||
} | ||
|
||
// Quote the entire expression. | ||
let quote = stylist.quote(); | ||
let annotation = generator.expr(expr); | ||
let mut quote_annotation = QuoteAnnotation::new(stylist); | ||
quote_annotation.visit_expr(expr); | ||
|
||
let annotation = quote_annotation.annotation; | ||
|
||
Ok(Edit::range_replacement( | ||
format!("{quote}{annotation}{quote}"), | ||
|
@@ -304,3 +295,105 @@ pub(crate) fn filter_contained(edits: Vec<Edit>) -> Vec<Edit> { | |
} | ||
filtered | ||
} | ||
|
||
#[derive(Copy, PartialEq, Clone)] | ||
enum State { | ||
Literal, | ||
AnnotatedFirst, | ||
AnnotatedRest, | ||
Other, | ||
} | ||
|
||
pub(crate) struct QuoteAnnotation<'a> { | ||
state: Vec<State>, | ||
stylist: &'a Stylist<'a>, | ||
annotation: String, | ||
final_quote_type: Quote, | ||
} | ||
Glyphack marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
impl<'a> QuoteAnnotation<'a> { | ||
pub(crate) fn new(stylist: &'a Stylist<'a>) -> Self { | ||
let final_quote_type = stylist.quote(); | ||
Self { | ||
state: vec![], | ||
stylist, | ||
annotation: String::new(), | ||
final_quote_type, | ||
} | ||
} | ||
} | ||
|
||
impl<'a> source_order::SourceOrderVisitor<'a> for QuoteAnnotation<'a> { | ||
fn visit_expr(&mut self, expr: &'a Expr) { | ||
let generator = Generator::from(self.stylist); | ||
match expr { | ||
Expr::Subscript(ast::ExprSubscript { value, slice, .. }) => { | ||
if let Some(name) = value.as_name_expr() { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should we only look for There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Correct. Is it enough if we rely on an attribute with value of "typing" and attr of "Optional" or "Literal", etc.? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I added another case for attributes. |
||
let value = generator.expr(value); | ||
self.annotation.push_str(&value); | ||
self.annotation.push('['); | ||
match name.id.as_str() { | ||
"Literal" => self.state.push(State::Literal), | ||
"Annotated" => self.state.push(State::AnnotatedFirst), | ||
_ => self.state.push(State::Other), | ||
} | ||
|
||
self.visit_expr(slice); | ||
self.state.pop(); | ||
self.annotation.push(']'); | ||
} | ||
} | ||
Expr::Tuple(ast::ExprTuple { elts, .. }) => { | ||
let Some(first_elm) = elts.first() else { | ||
return; | ||
}; | ||
self.visit_expr(first_elm); | ||
if self.state.last().copied() == Some(State::AnnotatedFirst) { | ||
self.state.push(State::AnnotatedRest); | ||
} | ||
for elm in elts.iter().skip(1) { | ||
self.annotation.push_str(", "); | ||
self.visit_expr(elm); | ||
} | ||
self.state.pop(); | ||
} | ||
Expr::BinOp(ast::ExprBinOp { | ||
left, op, right, .. | ||
}) => { | ||
Glyphack marked this conversation as resolved.
Show resolved
Hide resolved
|
||
self.visit_expr(left); | ||
self.annotation.push(' '); | ||
self.annotation.push_str(op.as_str()); | ||
self.annotation.push(' '); | ||
self.visit_expr(right); | ||
} | ||
Expr::BoolOp(ast::ExprBoolOp { op, values, .. }) => { | ||
for (i, value) in values.iter().enumerate() { | ||
if i > 0 { | ||
self.annotation.push(' '); | ||
self.annotation.push_str(op.as_str()); | ||
} | ||
self.visit_expr(value); | ||
} | ||
} | ||
_ => { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There was a branch for There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You are correct. I was not very familiar with syntax in type annotation but I don't see anything with a boolean operation. |
||
let source = match self.state.last().copied() { | ||
Some(State::Literal | State::AnnotatedRest) => { | ||
let mut source = generator.expr(expr); | ||
source = source.replace( | ||
self.final_quote_type.as_char(), | ||
&self.final_quote_type.opposite().as_char().to_string(), | ||
); | ||
source | ||
} | ||
None | Some(State::AnnotatedFirst | State::Other) => { | ||
let mut source = generator.expr(expr); | ||
source = source.replace(self.final_quote_type.as_char(), ""); | ||
source = source.replace(self.final_quote_type.opposite().as_char(), ""); | ||
source | ||
} | ||
}; | ||
self.annotation.push_str(&source); | ||
} | ||
} | ||
} | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,24 +1,22 @@ | ||
--- | ||
source: crates/ruff_linter/src/rules/flake8_type_checking/mod.rs | ||
--- | ||
quote.py:64:28: TCH004 [*] Quote references to `pandas.DataFrame`. Import is in a type-checking block. | ||
quote.py:67:28: TCH004 [*] Quote references to `pandas.DataFrame`. Import is in a type-checking block. | ||
| | ||
63 | if TYPE_CHECKING: | ||
64 | from pandas import DataFrame | ||
66 | if TYPE_CHECKING: | ||
67 | from pandas import DataFrame | ||
| ^^^^^^^^^ TCH004 | ||
65 | | ||
66 | def func(value: DataFrame): | ||
68 | | ||
69 | def func(value: DataFrame): | ||
| | ||
= help: Quote references | ||
|
||
ℹ Unsafe fix | ||
63 63 | if TYPE_CHECKING: | ||
64 64 | from pandas import DataFrame | ||
65 65 | | ||
66 |- def func(value: DataFrame): | ||
66 |+ def func(value: "DataFrame"): | ||
67 67 | ... | ||
66 66 | if TYPE_CHECKING: | ||
67 67 | from pandas import DataFrame | ||
68 68 | | ||
69 69 | | ||
|
||
|
||
69 |- def func(value: DataFrame): | ||
69 |+ def func(value: "DataFrame"): | ||
70 70 | ... | ||
71 71 | | ||
72 72 | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think I know why the test is failing within the current
MAX_ITERATIONS
value. All of the different test cases are within the same function here so when some of them tries to add theif TYPE_CHECKING
block along with theTYPE_CHECKING
import, the edits collide and thus it won't be applied which will then require another iteration. We should split the test cases, grouping them based on similarity, in similar way as done in the top half of the file.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You are right. I think I can fix that easily. There's just another problem I encountered. Aside from the new tests I've written the old ones also have this problem that each time the type annotation is changed a new import is added and the duplicate imports I think are removed in the end. So now the file fails again.
It can be reproduced by just duplicating one of the functions multiple times in master:
My guess is that two rules are conflicting here. One is moving the import to type checking block but since now that import is in type checking block another function later in the quote.py is using that import and it is resolved when the symbol is looked up here
I'm not sure if I'm right because the import should be resolved to the import inside the function itself. It's my guess. I will investigate this later after resolving other issues.
For now I'm going to create a new file that will avoid this issue. Moving my new test cases to a file works fine.