Allowing for trailing dollars and percent signs in String literals

[mccartney]

What's changed?

Changing the HCL lexing/parsing grammar to allow for $ and % signs at the end of String literals.

What's your motivation?

Until now it hasn't been allowed, which I think is a bug as described in:

• fixes HCL Parser does not accept percent sign (%) in String literals #4601
• fixes HCL Parser fails with a trailing dollar sign in String literal #4613

Checklist

• I've added unit tests to cover both positive and negative cases (some already exist)

[mccartney]

Hmm. Now I see the autogenerated Java files by ANTLR observe removed copyright/license notices. Not sure how to restore them (other than manual crafting them). Looking into it.

[timtebeek]

./gradlew licenseFormat

ought to help; although the bot also should chime in soon.

[github-actions]

Some suggestions could not be made:

• rewrite-hcl/src/main/java/org/openrewrite/hcl/internal/grammar/HCLParserBaseListener.java
  • lines 1-0
• rewrite-hcl/src/main/java/org/openrewrite/hcl/internal/grammar/HCLParserBaseVisitor.java
  • lines 1-0
• rewrite-hcl/src/main/java/org/openrewrite/hcl/internal/grammar/HCLParserListener.java
  • lines 1-0
• rewrite-hcl/src/main/java/org/openrewrite/hcl/internal/grammar/HCLParserVisitor.java
  • lines 1-0
• rewrite-hcl/src/main/java/org/openrewrite/hcl/internal/grammar/JsonPathParserBaseListener.java
  • lines 1-0
• rewrite-hcl/src/main/java/org/openrewrite/hcl/internal/grammar/JsonPathParserBaseVisitor.java
  • lines 1-0
• rewrite-hcl/src/main/java/org/openrewrite/hcl/internal/grammar/JsonPathParserListener.java
  • lines 1-0
• rewrite-hcl/src/main/java/org/openrewrite/hcl/internal/grammar/JsonPathParserVisitor.java
  • lines 1-0

[mccartney]

./gradlew licenseFormat

Thanks! Didn't know.
I ran that in fe73bc4 and closed all the bot suggestions.

[mccartney]

I need help in deciding what to do with the next batch of bot suggestions here.
Valid or not, but I am not sure if we want to apply bot suggestions against generated code. This seems counterproductive to me, but maybe this is what the project wants to enforce for all code.

Which BTW, to me is a sign this code shouldn't be checked-in to VCS.

[timtebeek]

Thanks a lot for the fixes here @mccartney ; I agree it feels odd to apply recipe fixes to generated code, but that's what we've come to do (for now) to clear out issues on code such that the bot only ever sees net-new issues on PRs. Since the changes are automated I think it's fine for now; longer term we can look at excluding those files from recipe changes.

[timtebeek]

On the subject of the code changes here; would you mind educating me a bit on what you've changed and the process behind regenerating those parsers? 😅 Would help me review & verify, as the changes look good already but I'd like the process to be repeatable for the next one.

[mccartney]

would you mind educating me a bit on what you've changed

Sure:

• The only file I have edited in this PR myself is the .g4 one. The input to ANTLR.
• Then I temporarily edited rewrite-hcl/build.gradle file to mention org.antlr:antlr4 instead of org.antlr:antlr4-runtime (turns out to be prerequisite to running generateAntlrSources in current state of things)
• Then I ran ./gradlew rewrite-hcl:generateAntlrSources, so that ANTLR generates rewrite-hcl/src/main/java/org/openrewrite/hcl/internal/grammar.
• Then I added a new test of escapingTheDollarSign to ensure the $${something} case is properly handled.

[mccartney]

I'd like the process to be repeatable for the next one.

IMHO the best way for this process be repeatable and self-documented is

• Plug ANTLR parser generation to Gradle, avoid storing generated code in the project #4626

[timtebeek]

Great to see these issues resolved so quickly after you've reported them here. Thanks for the helpful pointers as well on a part I hadn't seen much of before. 🙏🏻