Doesn't seem to update with treesitter playground

[expipiplus1]

Firstly: It's fantastic to see this working with Haskell!

I notice that with treesitter playground things don't seem to be updating (and I assume that this indicates a deeper problem)

Here's a recording of me editing a C file and a Haskell file, demonstrating the difference in experience:

https://asciinema.org/a/J38cw04yWrtdxFYfjWPwZXAvd

I'm using neovim master (as of today) and treesitter 0.19.5

[tek]

yeah, I haven't managed to get it to work properly in neovim yet. For me it produces lots of errors and has problems with newlines. Do you get any errors? Haven't tried it in a month, I'll investigate with the latest masters.

[expipiplus1]

No errors or anything like that I'm afraid. I'm happy to do any debugging, just let me know how!

[tek]

that's great, maybe these problems were fixed recently.
When you edit a file containing

import Foo.Bar
import Foo.Bar

is the second line highlighted? or if you don't have highlighting, what's the tree displayed by playground?

[expipiplus1]

Seems to be highlighting things correctly, not sure if that's tree sitter or vim though. Playground is certainly not correct though

Edit: Seems to be vim nothing changes if I TSBufToggle highlight. Editing a C file, I do see differences when I toggle highlighting.

[expipiplus1]

On a file with more imports, it seems to ERROR on some of the imports:

[tek]

hm! it definitely looks better than previously. I will investigate.

[expipiplus1]

Thanks!

…

On Thu, 27 May 2021, 11:53 pm Torsten Schmits, ***@***.***> wrote: hm! it definitely looks better than previously. I will investigate. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#35 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGRJXF6W4DRPSVJY4PLSJTTPZTF5ANCNFSM45T2LRNQ> .

[tek]

so I gave it a try and it still suffers from the same problem I saw before, that when e.g. two import statements are in successive lines, the second one is skipped.

Then I added some code below the imports, and none of that was highlighted/shown in the playground, until I deleted one of the imports, which triggered an update. So that might be similar behaviour to what you described.

In any case, my guess is that this is still a problem with nvim, I think we'll need to wait for it to be stable, since the parsing itself works fine. Unfortunately, I haven't figured out how to reproduce these issues with the tree-sitter cli.

[expipiplus1]

Sounds good to me! Is it worth opening an issue on neovim? (I don't think I understand what's going on well enough to do that myself however)

…

On Sat, 29 May 2021, 3:17 am Torsten Schmits, ***@***.***> wrote: so I gave it a try and it still suffers from the same problem I saw before, that when e.g. two import statements are in successive lines, the second one is skipped. Then I added some code below the imports, and none of that was highlighted/shown in the playground, until I deleted one of the imports, which triggered an update. So that might be similar behaviour to what you described. In any case, my guess is that this is still a problem with nvim, I think we'll need to wait for it to be stable, since the parsing itself works fine. Unfortunately, I haven't figured out how to reproduce these issues with the tree-sitter cli. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#35 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGRJXHWHOXJ3PVYTQABRLDTP7ORHANCNFSM45T2LRNQ> .

[tek]

there's this issue nvim-treesitter/nvim-treesitter#1201 pertaining to a certain error reported by TS while editing which I also experience, and it seems likely to me that it's the cause of what's happening here (the error is "end_col out of range", which fits with the fact that consecutive lines are alternatingly not highlighted, something with the newlines).
So I'm gonna try to come up with a minimal reproducer for that.

I suspect that in order for an issue on neovim to be successfully processed this might be too vague of a situation

[expipiplus1]

How are you seeing these error messages? I seem to get nothing printed from nvim. FWIW I tried with different line endings with no difference.

[tek]

happens after doing a few edits.

I experimented with different newline chars as well in the tests.

[expipiplus1]

Could the error message actually be legit and the parser is returning an out of bounds character?

…

On Sat, 29 May 2021, 8:15 pm Torsten Schmits, ***@***.***> wrote: [image: image] <https://user-images.githubusercontent.com/2632/120069809-1d06c400-c088-11eb-8ad3-221e80864433.png> happens after doing a few edits. I experimented with different newline chars as well in the tests. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#35 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGRJXF6SGOQMQ7CBY2UGV3TQDLEDANCNFSM45T2LRNQ> .

[tek]

I debugged this with the scanner in tests but haven't been able to find anything out.
I'd have to dive into how neovim does the parsing to get a better picture.

Anyway, since this exact error message is the topic of said issue, I hope its fix will improve this.

[theHamsta]

@tek does this problem only happen with nvim or does it also happen with tree-sitter's wasm playground?

[tek]

@theHamsta I haven't seen that! Can you link me?

[theHamsta]

I couldn't find the docs for it. Everytime I look for it I compile it by trial and error

tree-sitter build-wasm # requires emcc or docker
tree-sitter web-ui

[tek]

thanks! I'll investigate the problem with that.

[theHamsta]

It seems to miss the updates also with the web-ui

[tek]

guess that answers #32 (comment) as well, @maxbrunsfeld .

Question is now, how do we create a minimal reproducer from this? Does the web ui provide some additional insights?

[ahlinc]

I couldn't find the docs for it. Everytime I look for it I compile it by trial and error

I usually use tree-sitter CLI built from the master branch and found that there may be problems with starting the local playground due to collision of tree-sitter versions and if to be more concrete due to wasm components collision.

• My tree-sitter version located in ~/.cargo/bin/ which I install there by the cargo install --path cli/ command from tree-sitter's repo.
• Another tree-sitter version may be in a node_modules/.bin/ folder inside of a grammar folder cause it's general recommendation from tree-sitter's documentation and to have access to the node_modules/tree-sitter-cli/dsl.d.ts file what is useful in IDEs.
• Also tree-sitter CLI binary may be located in the global .../node_modules/bin/ folder if some one installed it with npm install -g tree-sitter-cli.

The tree-sitter CLI binary ships by itself another binary tree-sitter.wasm what is its C parser compiled into WASM library and this file is served by tree-sitter web-ui along with a grammar WASM file that is build with tree-sitter build-wasm command.

It seems the reason that local playground may fail to start is that tree-sitter.wasm file and the tree-sitter-<grammar_name>.wasm may be compiled by incompatible emscripten + binaryen versions. Tree-sitter currently uses 2.0.11 emscripten version while in the docker registry there are 2.0.23 version and tree-sitter uses latest version from the docker registry for compilation of the wasm file by the tree-sitter build-wasm command.

I've thought before about proposals:

• To patch tree-sitter that it would use exactly the same version from docker registry as it uses in its CI. But this doesn't protects from issues if someone wants to use locally installed version of Emscripten.
• Add some fuses to tree-sitter CLI and generated WASM files that would protect from such flavor of issues and may be clearly explain what is wrong.
• Another option is to ship with tree-sitter CLI pure C files and compile all of them with grammar files by tree-sitter build-wasm command.

@maxbrunsfeld Could you tell what do you think about above thoughts?

[tinygiant98]

@ahlinc ... Nailed it. The issue I was having in #1155 was resolved by installing emscripten 2.0.11 and rebuilding the parser to wasm. Still have a few issues (such as errors not being displayed in the document), but that's more likely something with the extension and not tree-sitter itself. Thanks!

[theHamsta]

This might be an issue with the playground. It only seems to update when `highlights.scm is present https://nvim-treesitter.zulipchat.com/#narrow/stream/252338-help/topic/Hot.20Reparsing.20with.20Custom.20Parsers/near/244874972

[expipiplus1]

Seems to be fixed now, perhaps due to 9c1fd42?

[theHamsta]

@expipiplus1 No, the bug is that the playground won't update when there are not highlighting queries. highlights.scm has now been added. So it's a bug in our repo not with tree-sitter-haskell, but maybe 9c1fd42 did fix another issue that was preventing to let it work with the web-ui playground

[tek]

FYI my issues disappeared when nvim 0.5 was released.

[tek]

@expipiplus1 can this be closed?

[expipiplus1]

Yes! Thanks!