Typescript target #4027

ericvergnaud · 2022-12-18T22:05:44Z

Candidate PR for the typescript target.

When using variables to compare (like in if clause) the variable shouldn't be quoted. More details can be found at the link below: https://cmake.org/cmake/help/latest/command/if.html#variable-expansion Signed-off-by: HS <[email protected]> Signed-off-by: Eric Vergnaud <[email protected]>

Signed-off-by: Eric Vergnaud <[email protected]>

Signed-off-by: nicksxs <[email protected]> Signed-off-by: Eric Vergnaud <[email protected]>

Signed-off-by: Terence Parr <[email protected]> Signed-off-by: Eric Vergnaud <[email protected]>

This reverts commit 1df58f7. Signed-off-by: Eric Vergnaud <[email protected]>

Signed-off-by: Terence Parr <[email protected]> Signed-off-by: Eric Vergnaud <[email protected]>

Add static library distribution in SPM Signed-off-by: Hell_Ghost <[email protected]> Signed-off-by: Eric Vergnaud <[email protected]>

Signed-off-by: Hell_Ghost [email protected] Signed-off-by: Hell_Ghost <[email protected]> Signed-off-by: Eric Vergnaud <[email protected]>

cpp builds using actions/ccache. Builds are more reliable (avoids the archive.apache server which intermittently reports timeouts) and also significantly improves the overall builds times (down from 46 mins to 28 mins). The slowest part of the build now is the Windows+cpp builds because there is no reliable cache implementation yet. MacOS+cpp (65% cache hit) is also relatively slow compared to Ubuntu+cpp (99% cache hit). Signed-off-by: HS <[email protected]> Signed-off-by: Terence Parr <[email protected]> # Conflicts: # .github/workflows/hosted.yml Signed-off-by: Eric Vergnaud <[email protected]>

Signed-off-by: Terence Parr <[email protected]> Signed-off-by: Eric Vergnaud <[email protected]>

Codex- · 2022-12-19T01:55:24Z

The types in this PR are not 'generated', they are hand coded

Apologies for the confusion, by generated I'm referring to the output generated by antlr when processing a grammar.

Currently, we use the types package and then use typescript to infer and generate .d.ts files for the Parser, Listener, and Lexer that are generated for the Javascript runtime.

KvanTTT · 2022-12-19T12:25:06Z

...rg/antlr/v4/test/runtime/descriptors/LeftRecursion/MultipleAlternativesWithCommonLabel_1.txt

-  : e '*' e     {$v = <Cast("BinaryContext","$ctx"):ContextMember({<Production("e")>(0)}, {<Result("v")>})> * <Cast("BinaryContext","$ctx"):ContextMember({<Production("e")>(1)}, {<Result("v")>})>;}  # binary
-  | e '+' e     {$v = <Cast("BinaryContext","$ctx"):ContextMember({<Production("e")>(0)}, {<Result("v")>})> + <Cast("BinaryContext","$ctx"):ContextMember({<Production("e")>(1)}, {<Result("v")>})>;}  # binary
+  : e '*' e     {$v = <Cast("BinaryContext","$ctx"):SubContextLocal({<Production("e")>(0)}, {<Result("v")>})> * <Cast("BinaryContext","$ctx"):SubContextLocal({<Production("e")>(1)}, {<Result("v")>})>;}  # binary
+  | e '+' e     {$v = <Cast("BinaryContext","$ctx"):SubContextLocal({<Production("e")>(0)}, {<Result("v")>})> + <Cast("BinaryContext","$ctx"):SubContextLocal({<Production("e")>(1)}, {<Result("v")>})>;}  # binary


Sorry, but I don't understand why the changes related to test data are placed into the current pull request related to TypeScript. In my opinion they should be moved into another pull request.

They are part of this PR because they were required to make the typescript target tests runnable.
They don’t add any value in themselves, it's simply a technical change to enable correct code generation of typescript tests.

In other words, they're not worth a PR unless we integrate the TypeScript target, so it makes sense to have them with this PR

KvanTTT · 2022-12-19T12:26:11Z

runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/CSharp.test.stg

+ContextListFunction(ctx, rule) ::= "<ctx>.<rule>()"
 StringType() ::= "String"
-ContextMember(ctx, subctx, member) ::= "<ctx>.<subctx>.<member>"
+ContextMember(ctx, member) ::= "<ctx>.<member>"
+SubContextLocal(ctx, subctx, local) ::= "<ctx>.<subctx>.<local>"
+SubContextMember(ctx, subctx, member) ::= "<ctx>.<subctx>.<member>"


The changes related to test templates also should be extracted into another pull request.

See previous comment

@ericvergnaud It would be nice to see the upgrade to the test specificity in a separate PR. Want me to pull those changes out into a PR we merge before your TypeScript target PR?

parrt · 2022-12-19T18:21:45Z

@parrt looks good to me, all tests pass for all targets on latest dev

yay! I would like to see the test updates in a separate PR. happy to do that for you if you want. Then we can merge both.

thanks for such a huge contribution!

ericvergnaud · 2022-12-19T18:50:45Z

yes please proceed to split the PR you might split as follows actually: - test templates and descriptors - FileUtils.java - VisitorFile and ListenerFile (fixes incomplete PR #3878) - all the rest except maybe StructDecl, which I’m noticing now, there could be an incorrect merge with StructDecl, I’m 100% positive that I haven’t touched that stuff, so not sure why it shows up in the merge… also not sure how the go tests pass if that merge is incorrect

…

Le 19 déc. 2022 à 19:21, Terence Parr ***@***.***> a écrit : @parrt <https://github.com/parrt> looks good to me, all tests pass for all targets on latest dev yay! I would like to see the test updates in a separate PR. happy to do that for you if you want. Then we can merge both. thanks for such a huge contribution! — Reply to this email directly, view it on GitHub <#4027 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZNQJAFM5X4CDUE4NNQADTWOCRUHANCNFSM6AAAAAATCXSV6E>. You are receiving this because you were mentioned.

KvanTTT · 2022-12-19T19:03:48Z

all the rest except maybe StructDecl, which I’m noticing now, there could be an incorrect merge with StructDecl, I’m 100% positive that I haven’t touched that stuff, so not sure why it shows up in the merge… also not sure how the go tests pass if that merge is incorrect

I highly recommend using interactive rebase in IntelliJ IDEA to filter out unrelated or incorrect changes and to prepare clear history.

parrt · 2022-12-19T19:52:57Z

Acknowledged @ericvergnaud @KvanTTT. I'll see if intellij can help me tease this apart.

parrt · 2022-12-19T23:21:35Z

tool/pom.xml

@@ -24,7 +24,7 @@
 		<dependency>
 			<groupId>org.antlr</groupId>
 			<artifactId>antlr4-runtime</artifactId>
-			<version>${project.version}</version>
+			<version>4.11.1</version>


Wouldn't it be better to use the project.version not 4.11.1?

certainly, I don't remember making this change but it wasn't PR'd intentionnally

no doubt I did something dumb like that...

Still I wonder why the rebase or merge from dev into your branch...i'm noticing a few things. ALMOST done with new PR for your test updates.

parrt · 2022-12-19T23:29:16Z

all the rest except maybe StructDecl, which I’m noticing now, there could be an incorrect merge with StructDecl, I’m 100% positive that I haven’t touched that stuff, so not sure why it shows up in the merge… also not sure how the go tests pass if that merge is incorrect

yeah, weird. maybe it needed a rebase or something if you branched from dev earlier than these changes? Hmm...i'll have to be careful pulling this apart. I think looking at all changes and pulling those out manually is easier than merging interactively with ~180 commits haha.

parrt · 2022-12-19T23:34:33Z

@ericvergnaud yeah, it looks like you did a refactor/cleanup that whacked a previous changes in "make it easy to break" 974606e

Hmm...i wonder if i can do a rebase to make that merge properly....i wonder why a conflict didn't appear?

…rebase after we merge this into dev. All tests pass locally (didn't check python2 actually but python3 works). Signed-off-by: Terence Parr <[email protected]>

parrt · 2022-12-20T00:56:44Z

Ok, @ericvergnaud @KvanTTT I have two PRs that should be clean:

Nervous something is borked up here. Some merge conflicts got overridden. I think we should VERY carefully rebuild a PR for the TS target. Maybe get a clean branch from head of dev. Then copy all changed files from previous PR branch on top, which SHOULD highlight merge conflicts and be fresher :)

ericvergnaud · 2022-12-20T06:59:41Z

Agreed. Seems we’re missing a PR for VisitiorFile/ListenerFile ? ((fixes incomplete PR #3878)) Then I’ll rebase and submit a new PR

…

Le 20 déc. 2022 à 02:31, Terence Parr ***@***.***> a écrit : Ok, @ericvergnaud <https://github.com/ericvergnaud> @KvanTTT <https://github.com/KvanTTT> I have two PRs that should be clean: #4029 <#4029> #4030 <#4030> Nervous something is borked up here. Some merge conflicts got overridden. I think we should VERY carefully rebuild a PR for the TS target. Maybe get a clean branch from head of dev. Then copy all changed files from previous PR branch on top, which SHOULD highlight merge conflicts and be fresher :) — Reply to this email directly, view it on GitHub <#4027 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZNQJGYJ3QE27M7RFLXGIDWOED6BANCNFSM6AAAAAATCXSV6E>. You are receiving this because you were mentioned.

tool/src/org/antlr/v4/codegen/model/ListenerFile.java

# Conflicts: # runtime-testsuite/test/org/antlr/v4/test/runtime/FileUtils.java # tool/src/org/antlr/v4/codegen/model/ListenerFile.java

ericvergnaud · 2022-12-20T18:23:11Z

@parrt thanks for the intermediate PRs. I've cleaned up the suspicious changes, all changes left are valid. Xing fingers that the build succeeds. If it does you can squash and merge.

parrt · 2022-12-20T18:30:54Z

If it does you can squash and merge.

acknowledged.

ericvergnaud · 2022-12-20T22:18:01Z

@parrt All green! :-)

parrt · 2022-12-21T01:22:35Z

I'll reset 4.11.2 -> 4.12.0 in milestones. thanks, @ericvergnaud great job!

hs-apotell and others added 30 commits September 9, 2022 17:09

initial commit

de0efeb

Signed-off-by: Eric Vergnaud <[email protected]>

renamed for clarity

e611332

Signed-off-by: Eric Vergnaud <[email protected]>

renamed for clarity

e8f59d6

Signed-off-by: Eric Vergnaud <[email protected]>

able to locate antlr4 runtime using ts-node, missing types

a4aa1fb

Signed-off-by: Eric Vergnaud <[email protected]>

progressing

6915202

Signed-off-by: Eric Vergnaud <[email protected]>

able to 'run' a test. It fails but it compiles and resolves!

c0fd00f

Signed-off-by: Eric Vergnaud <[email protected]>

reflect refactored runtime

b4f9e93

Signed-off-by: Eric Vergnaud <[email protected]>

able to run RecursiveLexerRuleRefWithWildcardPlus_1 test locally

971645f

Signed-off-by: Eric Vergnaud <[email protected]>

passes LexerExec tests in IntelliJ

4c80ae1

Signed-off-by: Eric Vergnaud <[email protected]>

make ATN private

16b121a

Signed-off-by: Eric Vergnaud <[email protected]>

ignore same tests as JavaScript

009107c

Signed-off-by: Eric Vergnaud <[email protected]>

compiles Parser and Lexer bu local run fails

66144f8

Signed-off-by: Eric Vergnaud <[email protected]>

ParserExec.TokenOffset test successful in IntelliJ !

484cc5a

Signed-off-by: Eric Vergnaud <[email protected]>

Progressing, passing 131 of 348 tests

e34c664

Signed-off-by: Eric Vergnaud <[email protected]>

pass 327 tests out of 348

9aca326

Signed-off-by: Eric Vergnaud <[email protected]>

more successful tests

a9b058f

Signed-off-by: Eric Vergnaud <[email protected]>

333 successful tests out of 348

dcff0b0

Signed-off-by: Eric Vergnaud <[email protected]>

all tests pass except 7 caused by antlr#3868

06fd2d9

Signed-off-by: Eric Vergnaud <[email protected]>

update getting-started doc

0a5f10e

Signed-off-by: nicksxs <[email protected]> Signed-off-by: Eric Vergnaud <[email protected]>

add blank github action file for hosted CI

291d974

Signed-off-by: Terence Parr <[email protected]> Signed-off-by: Eric Vergnaud <[email protected]>

update getting started document to say java 11

d7c6681

Signed-off-by: Terence Parr <[email protected]> Signed-off-by: Eric Vergnaud <[email protected]>

Revert "update getting started document to say java 11"

1fab453

This reverts commit 1df58f7. Signed-off-by: Eric Vergnaud <[email protected]>

add C# book code links

ac470cb

Signed-off-by: Terence Parr <[email protected]> Signed-off-by: Eric Vergnaud <[email protected]>

Add Jim/Ken to readme

3fb98a3

Signed-off-by: Terence Parr <[email protected]> Signed-off-by: Eric Vergnaud <[email protected]>

Update Swift Package to support static library

20d6f35

Add static library distribution in SPM Signed-off-by: Hell_Ghost <[email protected]> Signed-off-by: Eric Vergnaud <[email protected]>

Update Package.swift

0d8116e

Signed-off-by: Hell_Ghost [email protected] Signed-off-by: Hell_Ghost <[email protected]> Signed-off-by: Eric Vergnaud <[email protected]>

use snap to install go 1.19

778bcc3

Signed-off-by: Terence Parr <[email protected]> Signed-off-by: Eric Vergnaud <[email protected]>

grr...install snap

7298b4d

Signed-off-by: Terence Parr <[email protected]> Signed-off-by: Eric Vergnaud <[email protected]>

This was referenced Dec 18, 2022

Provide Javascript port of TokenStreamRewriter #3560

Merged

Build development and production ES5 bundles for the JavaScript runtime #3065

Closed

Migrate JavaScript runtime to TypeScript #1286

Closed

KvanTTT suggested changes Dec 19, 2022

View reviewed changes

parrt reviewed Dec 19, 2022

View reviewed changes

parrt mentioned this pull request Dec 20, 2022

Extract unit test updates by @ericvergnaud needed for TypeScript #4029

Merged

parrt mentioned this pull request Dec 20, 2022

Extract FileUtils updates by @ericvergnaud #4030

Merged

parrt reviewed Dec 20, 2022

View reviewed changes

tool/src/org/antlr/v4/codegen/model/ListenerFile.java Outdated Show resolved Hide resolved

ericvergnaud added 2 commits December 20, 2022 18:43

Merge branch 'dev' into typescript-target

bed7b5d

# Conflicts: # runtime-testsuite/test/org/antlr/v4/test/runtime/FileUtils.java # tool/src/org/antlr/v4/codegen/model/ListenerFile.java

rollback unwanted changes

c13d89e

parrt added target:typescript code-gen comp:runtime labels Dec 21, 2022

parrt added this to the 4.11.2 milestone Dec 21, 2022

parrt merged commit 2c75e64 into antlr:dev Dec 21, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Typescript target #4027

Typescript target #4027

ericvergnaud commented Dec 18, 2022

Codex- commented Dec 19, 2022

KvanTTT Dec 19, 2022

ericvergnaud Dec 19, 2022

ericvergnaud Dec 19, 2022

KvanTTT Dec 19, 2022 •

edited

Loading

ericvergnaud Dec 19, 2022

parrt Dec 19, 2022

parrt commented Dec 19, 2022

ericvergnaud commented Dec 19, 2022 via email

KvanTTT commented Dec 19, 2022

parrt commented Dec 19, 2022

parrt Dec 19, 2022

ericvergnaud Dec 20, 2022

parrt Dec 20, 2022

parrt commented Dec 19, 2022

parrt commented Dec 19, 2022

parrt commented Dec 20, 2022

ericvergnaud commented Dec 20, 2022 via email

ericvergnaud commented Dec 20, 2022

parrt commented Dec 20, 2022

ericvergnaud commented Dec 20, 2022

parrt commented Dec 21, 2022

Typescript target #4027

Typescript target #4027

Conversation

ericvergnaud commented Dec 18, 2022

Codex- commented Dec 19, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

KvanTTT Dec 19, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

parrt commented Dec 19, 2022

ericvergnaud commented Dec 19, 2022 via email

KvanTTT commented Dec 19, 2022

parrt commented Dec 19, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

parrt commented Dec 19, 2022

parrt commented Dec 19, 2022

parrt commented Dec 20, 2022

ericvergnaud commented Dec 20, 2022 via email

ericvergnaud commented Dec 20, 2022

parrt commented Dec 20, 2022

ericvergnaud commented Dec 20, 2022

parrt commented Dec 21, 2022

KvanTTT Dec 19, 2022 •

edited

Loading