-
Notifications
You must be signed in to change notification settings - Fork 3.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Compiler V2] Check recursive struct definitions #12257
Conversation
⏱️ 27h 25m total CI duration on this PR
🚨 4 jobs on the last run were significantly faster/slower than expected
|
6082975
to
ed9e378
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #12257 +/- ##
========================================
Coverage 63.9% 64.0%
========================================
Files 816 817 +1
Lines 180914 181062 +148
========================================
+ Hits 115779 115933 +154
+ Misses 65135 65129 -6 ☔ View full report in Codecov by Sentry. |
4 │ │ } | ||
│ ╰─────^ | ||
│ | ||
= S contains T... |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You could actually add locations to where T is defined. Check out error_with_hints
e.g. in the reference_saftey_processor how to do it. The current approach is also good, but in case you have not seen this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
see #12257 (comment)
|
||
/// Gets the display name of the struct type | ||
/// Requires: the type is a struct type | ||
fn get_struct_type_name(&self, ty: &Type) -> String { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You should use TypeDisplayContext
for this, then you do not need those functions, and it prints the instantiation. For now it would be a fully qualified name, but the name will become relative in the local module once my error message PR lands.
For this, you construct a q: QualifiedInstId<StructId>
, then call q.to_type().display(context)
. To construct the context, you'd put the reverse_struct_table and the type parameter names of the type in question into the context.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
&self, | ||
path: &Vec<Type>, | ||
this_struct_id: QualifiedId<StructId> | ||
) -> Vec<String> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Return Vec<(Loc, String)>
and stick this into error_with_hints
. It would be nice if Loc would be that of the field which contributes to the recursion.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I now included the name of the field in the error messages. I didn't use error_with_hints
because the trace is not shown in order. See this commit 6c83893 where I used error_with_hints
let child_name = self.get_struct_type_name(&path[i + 1]); | ||
loop_notes.push(format!("{} contains {}...", parent_name, child_name)); | ||
} | ||
loop_notes.push(format!("{} contains {}, which forms a loop.", self.get_struct_type_name(path.last().unwrap()), self.get_struct_display_name_simple(this_struct_id))); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please align with other error messages. Our convention is to put programming symbols into backticks, hence: "\backquote{}\backquote contains..."
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note that Wolfgang means literal backquotes:
"`{}` contains"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
// checks if `ty` occurs in `path` | ||
for parent in path.iter() { | ||
if let Type::Struct(mid, sid, _) = parent { | ||
let parent_id = QualifiedId { module_id: *mid, id: *sid }; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
mid.qualified(*sid)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
if parent_id == this_struct_id { | ||
if checking == this_struct_id { | ||
let loop_notes = self.gen_error_msg_for_fields_loop(path, this_struct_id); | ||
self.error_with_notes(loc_checking, &format!("recursive definition {}", self.get_struct_display_name_simple(this_struct_id)), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why isn't that formatted correctly?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
return false; | ||
} | ||
} | ||
} else { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If thats unreachable, the idiomatic way would be
let Struct(...) = parent else { panic!("unexpected") };
...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
let child_name = self.get_struct_type_name(&path[i + 1]); | ||
loop_notes.push(format!("{} contains {}...", parent_name, child_name)); | ||
} | ||
loop_notes.push(format!("{} contains {}, which forms a loop.", self.get_struct_type_name(path.last().unwrap()), self.get_struct_display_name_simple(this_struct_id))); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note that Wolfgang means literal backquotes:
"`{}` contains"
|
||
/// Checks for recursive definition of structs and adds diagnostics | ||
pub fn check_resursive_struct(&self, struct_entry: &StructEntry) { | ||
let params = (0..struct_entry.type_params.len()).map(|i| Type::TypeParameter(i as u16)).collect_vec(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This would be a bit more readable if you pull out the len calculation:
let num_params = struct_entry.type_params.len()
let params = (0..num_params).map(|i| Type::Parameter(i as u16)).collect_vec();
but I guess that's still pretty ugly. Oh, well..
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
refactored a bit
edcffce
to
b474938
Compare
All comments addressed. PTAL @wrwg @brmataptos |
33 │ │ } | ||
│ ╰─────^ | ||
│ | ||
= `type_param::S` contains field `f: type_param::G<type_param::S>`... |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is this not displayed correctly?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think Teng already has a fix for this. Just remember to come back to here once his fix landed, can yours land in the meantime
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice error messages now, thanks for trying out the other way to display errors. Yes, those hints are ordered in source order, which maybe confusing for the cycle problem.
33 │ │ } | ||
│ ╰─────^ | ||
│ | ||
= `type_param::S` contains field `f: type_param::G<type_param::S>`... |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think Teng already has a fix for this. Just remember to come back to here once his fix landed, can yours land in the meantime
} | ||
|
||
/// Checks for recursive definition of structs and adds diagnostics | ||
pub fn check_resursive_struct(&self, struct_entry: &StructEntry) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
pub fn check_resursive_struct(&self, struct_entry: &StructEntry) { | |
pub fn check_recursive_struct(&self, struct_entry: &StructEntry) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
)); | ||
} | ||
loop_notes.push(format!( | ||
"`{}` contains field `{}: {}`, which forms a loop.", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we please change "loop" to "cycle"? (here and in all the func/var names).
IMO "loop" in this context seems very weird.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
} | ||
loop_notes.push(format!( | ||
"`{}` contains field `{}: {}`, which forms a loop.", | ||
self.get_struct_type_name(&path.last().unwrap().2, this_struct_id), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use expect
instead of unwrap
.
Could you please add a note in the function doc that path
is expected to be non-empty, else this function will panic?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
&self, | ||
path: &Vec<(Loc, Symbol, Type)>, | ||
this_struct_id: QualifiedId<StructId>, | ||
_loc: Loc, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we have _loc
as a parameter?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We may want to use it in the error messages.
loop_notes | ||
} | ||
|
||
/// Checks if a field type occurs in its access path, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please format the comments to have reasonable number of columns.
This line ends at 58, the next line ends at 132.
Rust fmt does not introduce line breaks in comments, so unfortunately you cannot rely on that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
{ | ||
path.push((ty_loc.clone(), *field_name, ty.clone())); | ||
let field_ty_instantiated = field_ty_uninstantiated.instantiate(insts); | ||
// short-curcuit upon first recursive occurence found |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
// short-curcuit upon first recursive occurence found | |
// short-circuit upon first recursive occurrence found |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, we are not doing what this comment claims. It is because the return values are flipped.
Please add this test case (or similar), which demonstrates the issue:
struct X {
f: Y,
g: Y,
}
struct Y {
f: X,
g: X
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
), | ||
loop_notes, | ||
); | ||
return true; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this match the function doc on return value?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
outdated
} | ||
} | ||
} else { | ||
panic!("field access path contains non-struct") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
panic!("field access path contains non-struct") | |
unreachable!("field access path contains non-struct") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
outdated
self.error_with_notes( | ||
loc_checking, | ||
&format!( | ||
"recursive definition {}", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"recursive definition {}", | |
"recursive definition of struct `{}`", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Or something similar.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I change the name to "cyclic data" as in V1, since the problem is not always recursive definition
} | ||
} | ||
if let Some(fields) = &this_struct_entry.fields { | ||
// check for descendant fields recursively |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we consider the code:
struct X {
a: Y,
b: Y,
c: Y,
..
z: Y
}
where Y
is a struct with several other fields (including possibly other structs, but none of them recursive), then we will explore a multiplicative number of paths (most of them unnecessarily)?
That is, if X
has 10 fields that are Y
s, and Y
has 10 fields that are Z
s (and Z
is just a wrapper struct), then we recursively call this function 100 times, where as we only have to explore path X
-any-field-> Y
-any-field-> Z
to show there is no recursive struct?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
solved in latest commit
@fEst1ck Also, note that "Aptos Move Test for Compiler v2" is failing (but it is not a required test), because it is taking too much time - it should finish with much less time. Please investigate before merging in (the performance issue I pointed out may be related, may be not). |
ff19aac
to
7c486de
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have minor comments - please address them before merging.
LGTM otherwise - I thought this rewrite was much easier to reason about, and is certainly more performant than the previous version. Nice work!
// Copyright © Aptos Foundation | ||
// SPDX-License-Identifier: Apache-2.0 | ||
|
||
//! Implements an AST pass that checks any struct `S` cannot contain descendant of type `S`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Mention if there are any prerequisites (other AST passes that must be run before it, cross module cyclic struct references have already been flagged as errors, etc.) or any side-effects, including if there are none. This would allow us to re-order passes if need be.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
//! Implements an AST pass that checks any struct `S` cannot contain descendant of type `S`. | |
//! Implements an AST pass that checks any struct `m::S` cannot contain descendant of type `m::S`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(either this, or mention that this check is only done within each module)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
checked: &mut BTreeSet<StructId>, | ||
) { | ||
// check for cyclic dependencies | ||
for (i, (_ancestor_loc, _ancecsotr_name, ancestor)) in path.iter().enumerate() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
for (i, (_ancestor_loc, _ancecsotr_name, ancestor)) in path.iter().enumerate() { | |
for (i, (_ancestor_loc, _ancestor_name, ancestor)) in path.iter().enumerate() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
} | ||
}, | ||
Type::Primitive(_) | Type::TypeParameter(_) => {}, | ||
_ => panic!("invalid field type"), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
_ => panic!("invalid field type"), | |
_ => unreachable!("invalid field type"), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
path.push((loc.clone(), field_name, struct_id)); | ||
match field_env.get_type() { | ||
Type::Struct(field_mod_id, field_struct_id, insts) => { | ||
// make sure the field struct has been checked |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
// make sure the field struct has been checked | |
// make sure the field struct has not been checked already |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
} | ||
// check the type parameters of the fields cannot contain the struct we are checking | ||
if insts.iter().any(|ty| { | ||
self.ty_contains_struct(path, &ty, loc.clone(), struct_id, checked) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
self.ty_contains_struct(path, &ty, loc.clone(), struct_id, checked) | |
self.ty_contains_struct(path, ty, loc.clone(), struct_id, checked) |
fn report_invalid_field(&self, struct_env: &StructEnv, field_env: &FieldEnv) { | ||
let struct_name = self.get_struct_name(struct_env.get_id()); | ||
let note = format!( | ||
"invalid field {} of {} containing {} itself", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"invalid field {} of {} containing {} itself", | |
"invalid field `{}` of `{}` containing `{}` itself", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
@@ -0,0 +1,25 @@ | |||
address 0x42 { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: having test file names like recursive_structs
and recursive_struct
(where both contains multiple recursive structs) can make them difficult to maintain or refer to. More descriptive names would be ideal, but even numbering them recursive_structs_[1,2]
would be better than existing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
From the PR description:
We already say this in the move book, see: https://aptos.dev/move/book/structs-and-resources/#defining-structs. Did you mean something else? |
Also found this https://aptos.dev/move/book/generics/#recursive-structs |
1a6837b
to
3932ffb
Compare
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
✅ Forge suite
|
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
✅ Forge suite
|
Description
This PR adds check for recursive struct definitions in the model builder. That is, no struct
S
contains a descendant field of typeS
. Seechecking/typing/recursive_struct.move
for examples.TODO: add that we can't have recursive structs in the move book.
Fixes #12119
Test Plan
checking/typing/recursive_struct.move