Handle erroneous nodes in open rewrite #4412
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
rewrite-java-test/src/test/java/org/openrewrite/java/JavaParserTest.java
Outdated
Show resolved
Hide resolved
rewrite-java-test/src/test/java/org/openrewrite/java/JavaParserTest.java
Outdated
Show resolved
Hide resolved
rewrite-java-test/src/test/java/org/openrewrite/java/JavaParserTest.java
Outdated
Show resolved
Hide resolved
rewrite-java-test/src/test/java/org/openrewrite/java/JavaParserTest.java
Outdated
Show resolved
Hide resolved
|
Thanks for the continued work on this @vudayani ! I must say it's great to see this come in, also taking into account the out-of-bounds-delivered raw feedback. :) I've fixed a minor conflict with imports on main, and added two missing license headers. Let us know if/when you consider this ready for review, or whether there's additional work you still plan to do. |
|
I think this PR is ready for review and more feedback... Likely , there will be more work to do. |
rewrite-java-test/src/test/java/org/openrewrite/java/JavaParserTest.java
Outdated
Show resolved
Hide resolved
rewrite-java/src/main/java/org/openrewrite/java/search/FindCompileErrors.java
Outdated
Show resolved
Hide resolved
rewrite-java/src/main/java/org/openrewrite/java/table/CompileErrors.java
Outdated
Show resolved
Hide resolved
timtebeek
reviewed
Sep 21, 2024
rewrite-java-17/src/main/java/org/openrewrite/java/isolated/ReloadableJava17ParserVisitor.java
Show resolved
Hide resolved
|
Thanks for the continued work on this @vudayani ! We're on to a downstream failure right now it seems. `class org.openrewrite.java.tree.J$Erroneous cannot be cast to class org.openrewrite.java.tree.Expression` in `org.openrewrite.java.format.SpacesVisitor`. |
…rTest.java Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
…rTest.java Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
vudayani
force-pushed
the
handle-erroneous-nodes
branch
from
d6a680a to
611d18b
Compare
|
Thanks for your patience @timtebeek |
|
@timtebeek It is ready for review and all tests seem to pass locally. Thanks again for your patience, and sorry for the delay in getting back on this. |
|
Thanks a lot for all the hard work here @vudayani ! I've tagged Knut for help with review as I'm traveling, and I know Jonathan has been quite busy these past couple of weeks. I hope we can get this in before Wednesday's release, but that'll depend on their feedback as well. |
timtebeek
reviewed
Oct 6, 2024
rewrite-java/src/main/java/org/openrewrite/java/table/CompileErrors.java
Show resolved
Hide resolved
|
Where are we with this @timtebeek ? What do you think? Is there anything specific missing before this can be merged? Would be awesome to finally push this forward into main... :-) |
Thanks for the reminder @martinlippert ; I'd already tagged folks for additional review, but I've now also raised this on an internal channel, as I understand you're eager to get this in for your purposes, as are we to see this finalized. |
sambsnyd
approved these changes
Nov 21, 2024
rewrite-java/src/main/java/org/openrewrite/java/search/FindCompileErrors.java
Outdated
Show resolved
Hide resolved
| rightPadded = (JRightPadded<J2>) JRightPadded.build(getErroneous(List.of(rightPadded))); | ||
| } | ||
| if (endPos(t) == cursor && rightPadded.getElement() instanceof J.Erroneous) { | ||
| cursor++; |
There was a problem hiding this comment.
This part is unclear to me. Why advance the cursor a single character? Can we add a comment explaining this? Is there a specific test case which would demonstrate this?
There was a problem hiding this comment.
I'd have to dig into this to recall why this was done. Hopefully a test would fail if I comment this out. Will get back on this.
There was a problem hiding this comment.
Some suggestions could not be made:
- rewrite-groovy/src/test/java/org/openrewrite/groovy/tree/BinaryTest.java
- lines 209-209
- rewrite-groovy/src/test/java/org/openrewrite/groovy/tree/ClassDeclarationTest.java
- lines 297-297
- lines 311-311
- rewrite-groovy/src/test/java/org/openrewrite/groovy/tree/CompilationUnitTest.java
- lines 132-132
- rewrite-groovy/src/test/java/org/openrewrite/groovy/tree/MethodDeclarationTest.java
- lines 189-189
- rewrite-groovy/src/test/java/org/openrewrite/groovy/tree/VariableDeclarationsTest.java
- lines 190-190
|
Thanks a lot @sambsnyd for taking a look, providing feedback, and for your approval here. |
timtebeek
approved these changes
Dec 19, 2024
There was a problem hiding this comment.
Thanks for the work done here @vudayani , @BoykoAlex and all the others involved. Appreciate you working through the feedback, and the patience shown as we figured out how to best fit this in. Great to see that you took it upon yourselves to add this feature.
This was referenced Dec 19, 2024
amishra-u
pushed a commit
to amishra-u/rewrite
that referenced
this pull request
Dec 28, 2024
* Handle erroneous nodes in a tree * Add visitErroneous to all java parser visitors * Override the visitVariable to handle erroneous identifier names set by JavacParser * retain name and suffix for erroneous varDecl * override the visitVariable to handle error identifiers in all java parser visitors * Remove sysout * Update rewrite-java-test/src/test/java/org/openrewrite/java/JavaParserTest.java Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update rewrite-java-test/src/test/java/org/openrewrite/java/JavaParserTest.java Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * handle errors in method params, variable declarations, fix tests * Add missing license headers * fix compilation error * fix compilation error in Java8ParserVisitor * Apply code suggestions from bot * fix cases for statementDelim * fix block statement template generator to handle adding semicolon * fix ChangeStaticFieldToMethod recipe * Record compiler errors from erroneous LST nodes * Adjustments for comments * Java 17 parser adjustment alos in 8, 11 and 21 * Add `FindCompileErrorsTest` & move away from deprecated `print()` --------- Co-authored-by: Jonathan Schnéider <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Tim te Beek <[email protected]> Co-authored-by: aboyko <[email protected]>
timtebeek
added a commit
that referenced
this pull request
Dec 31, 2024
* Add lombok support for java-11 * Handle erroneous nodes in open rewrite (#4412) * Handle erroneous nodes in a tree * Add visitErroneous to all java parser visitors * Override the visitVariable to handle erroneous identifier names set by JavacParser * retain name and suffix for erroneous varDecl * override the visitVariable to handle error identifiers in all java parser visitors * Remove sysout * Update rewrite-java-test/src/test/java/org/openrewrite/java/JavaParserTest.java Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update rewrite-java-test/src/test/java/org/openrewrite/java/JavaParserTest.java Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * handle errors in method params, variable declarations, fix tests * Add missing license headers * fix compilation error * fix compilation error in Java8ParserVisitor * Apply code suggestions from bot * fix cases for statementDelim * fix block statement template generator to handle adding semicolon * fix ChangeStaticFieldToMethod recipe * Record compiler errors from erroneous LST nodes * Adjustments for comments * Java 17 parser adjustment alos in 8, 11 and 21 * Add `FindCompileErrorsTest` & move away from deprecated `print()` --------- Co-authored-by: Jonathan Schnéider <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Tim te Beek <[email protected]> Co-authored-by: aboyko <[email protected]> * Make Groovy Parser correctly handle nested parenthesis (#4801) * WIP * Format * Format * Move grabbing of whitespace and resetting cursor to where it is actually required * Extra check is not required * Use toString * Add `emptyListLiteralWithParentheses` test * Add `insideFourParenthesesAndEnters` test * Move list tests all to ListTest * Add `emptyMapLiteralWithParentheses` * Review feedback and fix new testcases * Add `attributeWithParentheses` * Improve AttributeTest * Improve AttributeTest * Improve AttributeTest * Improve AttributeTest * Improve AttributeTest * Review fix new testcases * Revert edit to testcase * Add and fix testcase with newline * Add JavaDoc and move logic regarding whitespace and resetting cursor --------- Co-authored-by: lingenj <[email protected]> * suppress javax.json (#4804) * suppress javax.json * Update suppressions.xml * Refactor SpringReference (#4805) * Separating and clearer naming * Add license header * Review feedback * refactor: Update Gradle wrapper (#4808) Use this link to re-run the recipe: https://app.moderne.io/recipes/org.openrewrite.gradle.UpdateGradleWrapper?organizationId=T3BlblJld3JpdGU%3D#defaults=W3sibmFtZSI6ImFkZElmTWlzc2luZyIsInZhbHVlIjoiRmFsc2UifV0= Co-authored-by: Moderne <[email protected]> * Add recipe to remove Gradle Enterprise and Develocity (#4809) * Add recipe to remove Gradle Enterprise and Develocity * Remove left over java plugin * Add a UsesType precondition to ReplaceConstant * Allow file scheme in `RemoteArchive` to simplify testing (#4791) * Allow file scheme in `RemoteArchive` to simplify testing While it might look a bit controversial, the file scheme can also point to a remote (for instance a mounted network share) file. By allowing the `file://` scheme we can use `RemoteArchive` for those files. As a useful side effect, this makes testing RemoteArchive handling a lot easier. * fix test * Update rewrite-core/src/test/java/org/openrewrite/remote/RemoteArchiveTest.java Co-authored-by: Sam Snyder <[email protected]> --------- Co-authored-by: Sam Snyder <[email protected]> * Try alternative way of determining parenthesis level for `BinaryExpression` when AST doesn't provide `_INSIDE_PARENTHESES_LEVEL` flag (#4807) * Add a `isClassAvailable` method to the ReflectionUtils (#4810) * Add a `isClassAvailable` method to the ReflectionUtils * Add a `isClassAvailable` method to the ReflectionUtils * Add a `isClassAvailable` method to the ReflectionUtils * Update rewrite.yml to enforce CompareEnumsWithEqualityOperator * Correctly map generic return and parameter types in `JavaReflectionTypeMapping` (#4812) * Polish formatting * Add more scenarios to JavaTypeGoat for simply typed fields and methods that return exceptions. * Support mapping of generic thrown exception types (#4813) * refactor: Enum values should be compared with "==" (#4811) Use this link to re-run the recipe: https://app.moderne.io/recipes/org.openrewrite.staticanalysis.CompareEnumsWithEqualityOperator?organizationId=T3BlblJld3JpdGU%3D Co-authored-by: Moderne <[email protected]> * Keep the names of generic type variables defined by methods. (#4814) * Make the same performance improvement to parameter names allocations that we previously made to Java 17/21 in #3345. * Fix Java reflection mapping of generic typed fields. (#4815) * Revert parenthesis changes (#4818) * Revert "Try alternative way of determining parenthesis level for `BinaryExpression` when AST doesn't provide `_INSIDE_PARENTHESES_LEVEL` flag (#4807)" This reverts commit e59e48b. * Revert "Make Groovy Parser correctly handle nested parenthesis (#4801)" This reverts commit 91a031a. * JavaTemplate bug when inserting `final var` into for-each (#4806) * JavaTemplate bug when inserting `final var` into for-each * Split variable declarations when they contain stop comment * Reduce accidental changes between Java 11 and 17 parsers * Add missing import --------- Co-authored-by: Udayani Vaka <[email protected]> Co-authored-by: Jonathan Schnéider <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Tim te Beek <[email protected]> Co-authored-by: aboyko <[email protected]> Co-authored-by: Laurens Westerlaken <[email protected]> Co-authored-by: lingenj <[email protected]> Co-authored-by: Peter Streef <[email protected]> Co-authored-by: Shannon Pamperl <[email protected]> Co-authored-by: Moderne <[email protected]> Co-authored-by: Sam Snyder <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What's changed?
The PR introduces handling for erroneous nodes in OpenRewrite. Currently, when there is an error in the LST, the parser may attempt to recover by ignoring or removing the erroneous node to produce a valid LST. The changes in this PR address this by adding a mechanism to handle these erroneous nodes instead of ignoring them, ensuring that they are preserved and correctly processed.
What's your motivation?
The motivation behind this PR is to handle scenarios where errors in the source code could lead to discrepancies in the computed results. When any recipe is executed, the edits are computed on a parsed LST without the error statement. By handling erroneous nodes, we ensure that the error lines are retained and reflected correctly in the LST, leading to more accurate edits and transformations.
Anything in particular you'd like reviewers to focus on?
Anyone you would like to review specifically?
Have you considered any alternatives or workarounds?
Any additional context
This update is crucial for ensuring consistency in scenarios where error lines could affect the outcome of recipes and transformations.
Checklist