-
Notifications
You must be signed in to change notification settings - Fork 3.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Provide Javascript port of TokenStreamRewriter #3560
Provide Javascript port of TokenStreamRewriter #3560
Conversation
9d078cd
to
0d0987f
Compare
18dbf92
to
8f82b2b
Compare
cf4d564
to
fba190e
Compare
@@ -148,7 +152,7 @@ export default class BufferedTokenStream extends TokenStream { | |||
return n; | |||
} | |||
|
|||
// Get all tokens from start..stop inclusively/// | |||
// Get all tokens from start..stop inclusively/// |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Automatic code formatting kicked in on this file.
@eirikur-grid sorry if it's taken us so long to look at this |
This week is going to be quite hectic for me. I'll take a look shortly after Christmas. |
9e5cb39
to
bb0a42b
Compare
@ericvergnaud I've rebased and added the type definitions file. |
@eirikur-grid we're about to release, and this is a good time to finalize things.
If you can do that rapidly enough, this PR will make it into the next release |
You caught me at a very busy time, hence the late reply. I should be able to get around to this before the end of this week. |
HeyToo bad, version 4.12 just went out Please rebase and we’ll ship this in the next releaseEricEnvoyé de mon iPhoneLe 19 févr. 2023 à 23:13, Eiríkur Fannar Torfason ***@***.***> a écrit :
@eirikur-grid we're about to release, and this is a good time to finalize things. Could you:
* squash your commits and gpg-sign off the squashed commit (this replaces the contributors.txt strategy)
* move your changes in index.js to index.node.js and index.web.js
* add an export in index.d.ts
* move your spec/* files under spec/rewriter/* to clarify their purpose
* not mandatory but a .ts test would also be welcome (to validate the ts api)
If you can do that rapidly enough, this PR will make it into the next release
You caught me at a very busy time, hence the late reply. I should be able to get around to this before the end of this week.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>
|
Signed-off-by: Eiríkur Fannar Torfason <[email protected]>
e51de80
to
c9995f9
Compare
@ericvergnaud I believe I've addressed all remarks. |
@@ -0,0 +1,16 @@ | |||
ANTLR_VERSION = 4.12.0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we have a small problem here
The purpose if the CI is not to test against the latest official version, rather against the version being tested.
So the makefile should use the tool from the local maven repo, not from the antlr web site.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, how do I do that? Are there any examples within the repo that I can follow? I can see that the Java tests are capable of generating lexers/parsers from a grammar at run-time but what about other languages?
When I started working on this, I found Python tests using generated code (runtime/Python2/tests/mocks/TestLexer.py) that has been commited to the repo, so I assumed this was OK.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think my contribution ends here. Over a year ago, I found myself needing a JavaScript version of the TokenStreamRewriter. That is no longer the case, so I have no vested interest in putting more effort into this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All generic grammar tests generate parsers at CI time, but they are run from java.
Every GitHub action runs the "Build ANTLR with Maven" step, which builds the tool and registers it in the local maven repo, which should be found at ~/m2/repository/org/antlr/antlr4/${version}/antlr4-${version}-complete.jar.
You are correct in observing that many target-specific tests do not follow the expected policy, but let's not make things worse :-)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry to hear that, and apologies for taking so long to consider this PR. I hope your interest will grow again soon. This is useful work for the community.
@@ -0,0 +1,37 @@ | |||
// Generated from abc.g4 by ANTLR 4.12.0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this needs to be regenerated during tests or it will fail with a version mismatch
@@ -0,0 +1,50 @@ | |||
// Generated from calc.g4 by ANTLR 4.12.0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same here
@parrt ready to merge, I suggest we deal with the versioning issue at a global level since it affects all targets |
Re "derived from antlr4ts", license appears to be "The ANTLR Project" so we're good there: https://github.com/tunnelvisionlabs/antlr4ts/blob/master/LICENSE |
undefined Signed-off-by: Jim.Idle <[email protected]>
What
Add a JavaScript port of the TokenStreamRewriter that includes most, but not all, of the methods from the Java implementation.
Why
So that JavaScript developers can benefit from having it to implement all sorts of neat things (refactorings, translation etc.)
Scope
Included
insertAfter
insertBefore
replace
delete
getText
getProgram
getTokenStream
Omitted
The following methods were omitted because
a) I couldn't find any tests for them and
b) their utility is not immediately obvious to me.
rollback
deleteProgram
getLastRewriteTokenIndex
If someone needs them then we can obviously add support for them in a subsequent PR.
How
By
TokenStreamRewriter
classCredit to @sharwell for having ported the TokenStreamRewriter and the corresponding unit tests to TypeScript (see https://github.com/tunnelvisionlabs/antlr4ts). That has made creating a JavaScript port relatively straight-forward.
Drive-bys
TokenStreamRewriter
..gitignore
to exclude autogenerated files that don't need to be placed under source control.npm run lint
targetCloses #1767