Linter: buf: Diagnostics are often behind and need a manual :e to refresh diagnostics

[AlxHnr]

FAQ

• I have checked the FAQ and it didn't resolve my problem.

Issues

• I have checked existing issues and there are no issues with the same problem.

Neovim Version

0.7.0

Operating System

Fedora 35

Minimal config

Only change to the provided minimal config:

-        sources = {},
+        sources = {null_ls.builtins.diagnostics.buf},

Steps to reproduce

• Install buf
• nvim --clean -u minimal_init.lua
• Open an empty file like foo.proto. This should display a warning
• Add declarations like syntax = "proto3";
• Try breaking this declaration by setting it to syntax = "proto1298129";
• Undo/Redo your changes

Expected behavior

Diagnostics are not lagging behind and work just as great as other linters, e.g. shellcheck.

Actual behavior

Diagnostics are often behind. Only workaround is to reload the file via :e, or to enter and leave insert mode a few times to force a refresh.

Debug log

[TRACE Thu May 12 20:01:15 2022] ...te/pack/packer/start/null-ls.nvim/lua/null-ls/client.lua:106: starting null-ls client
[TRACE Thu May 12 20:01:15 2022] .../site/pack/packer/start/null-ls.nvim/lua/null-ls/rpc.lua:121: received LSP request for method initialize
[TRACE Thu May 12 20:01:15 2022] .../site/pack/packer/start/null-ls.nvim/lua/null-ls/rpc.lua:146: received LSP notification for method initialized
[TRACE Thu May 12 20:01:15 2022] .../site/pack/packer/start/null-ls.nvim/lua/null-ls/rpc.lua:146: received LSP notification for method textDocument/didOpen
[TRACE Thu May 12 20:01:15 2022] ...ack/packer/start/null-ls.nvim/lua/null-ls/generators.lua:21: running generators for method NULL_LS_DIAGNOSTICS_ON_OPEN
[DEBUG Thu May 12 20:01:15 2022] ...t/null-ls.nvim/lua/null-ls/helpers/generator_factory.lua:344: spawning command "buf" at /home/user with args { "lint", "/tmp/deleteme/foo.proto#include_package_files=true" }
[TRACE Thu May 12 20:01:15 2022] ...t/null-ls.nvim/lua/null-ls/helpers/generator_factory.lua:217: error output: /tmp/deleteme/foo.proto:1:1:Files must have a package defined.
/tmp/deleteme/foo.proto:1:1:Files must have a syntax explicitly specified. If no syntax is specified, the file defaults to "proto2".

[TRACE Thu May 12 20:01:15 2022] ...t/null-ls.nvim/lua/null-ls/helpers/generator_factory.lua:218: output: nil
[TRACE Thu May 12 20:01:15 2022] ...ck/packer/start/null-ls.nvim/lua/null-ls/diagnostics.lua:169: received diagnostics from source 1
[TRACE Thu May 12 20:01:15 2022] ...ck/packer/start/null-ls.nvim/lua/null-ls/diagnostics.lua:170: { {
    bufnr = 3,
    col = 0,
    end_col = 0,
    end_lnum = 1,
    filename = "/tmp/deleteme/foo.proto",
    lnum = 0,
    message = "Files must have a package defined.",
    row = "1",
    severity = 1,
    source = "buf_lint"
  }, {
    bufnr = 3,
    col = 0,
    end_col = 0,
    end_lnum = 1,
    filename = "/tmp/deleteme/foo.proto",
    lnum = 0,
    message = 'Files must have a syntax explicitly specified. If no syntax is specified, the file defaults to "proto2".',
    row = "1",
    severity = 1,
    source = "buf_lint"
  } }
[TRACE Thu May 12 20:01:21 2022] .../site/pack/packer/start/null-ls.nvim/lua/null-ls/rpc.lua:146: received LSP notification for method textDocument/didChange
[TRACE Thu May 12 20:01:21 2022] ...ack/packer/start/null-ls.nvim/lua/null-ls/generators.lua:21: running generators for method NULL_LS_DIAGNOSTICS
[DEBUG Thu May 12 20:01:21 2022] ...t/null-ls.nvim/lua/null-ls/helpers/generator_factory.lua:344: spawning command "buf" at /home/user with args { "lint", "/tmp/deleteme/foo.proto#include_package_files=true" }
[TRACE Thu May 12 20:01:21 2022] ...t/null-ls.nvim/lua/null-ls/helpers/generator_factory.lua:217: error output: /tmp/deleteme/foo.proto:1:1:Files must have a package defined.
/tmp/deleteme/foo.proto:1:1:Files must have a syntax explicitly specified. If no syntax is specified, the file defaults to "proto2".

[TRACE Thu May 12 20:01:21 2022] ...t/null-ls.nvim/lua/null-ls/helpers/generator_factory.lua:218: output: nil
[TRACE Thu May 12 20:01:21 2022] ...ck/packer/start/null-ls.nvim/lua/null-ls/diagnostics.lua:169: received diagnostics from source 1
[TRACE Thu May 12 20:01:21 2022] ...ck/packer/start/null-ls.nvim/lua/null-ls/diagnostics.lua:170: { {
    bufnr = 3,
    col = 0,
    end_col = 0,
    end_lnum = 1,
    filename = "/tmp/deleteme/foo.proto",
    lnum = 0,
    message = "Files must have a package defined.",
    row = "1",
    severity = 1,
    source = "buf_lint"
  }, {
    bufnr = 3,
    col = 0,
    end_col = 0,
    end_lnum = 1,
    filename = "/tmp/deleteme/foo.proto",
    lnum = 0,
    message = 'Files must have a syntax explicitly specified. If no syntax is specified, the file defaults to "proto2".',
    row = "1",
    severity = 1,
    source = "buf_lint"
  } }
[TRACE Thu May 12 20:01:23 2022] .../site/pack/packer/start/null-ls.nvim/lua/null-ls/rpc.lua:146: received LSP notification for method textDocument/didChange
[TRACE Thu May 12 20:01:23 2022] ...ack/packer/start/null-ls.nvim/lua/null-ls/generators.lua:21: running generators for method NULL_LS_DIAGNOSTICS
[DEBUG Thu May 12 20:01:23 2022] ...t/null-ls.nvim/lua/null-ls/helpers/generator_factory.lua:344: spawning command "buf" at /home/user with args { "lint", "/tmp/deleteme/foo.proto#include_package_files=true" }
[TRACE Thu May 12 20:01:23 2022] ...t/null-ls.nvim/lua/null-ls/helpers/generator_factory.lua:217: error output: /tmp/deleteme/foo.proto:1:1:Files must have a package defined.
/tmp/deleteme/foo.proto:1:1:Files must have a syntax explicitly specified. If no syntax is specified, the file defaults to "proto2".

[TRACE Thu May 12 20:01:23 2022] ...t/null-ls.nvim/lua/null-ls/helpers/generator_factory.lua:218: output: nil
[TRACE Thu May 12 20:01:23 2022] ...ck/packer/start/null-ls.nvim/lua/null-ls/diagnostics.lua:169: received diagnostics from source 1
[TRACE Thu May 12 20:01:23 2022] ...ck/packer/start/null-ls.nvim/lua/null-ls/diagnostics.lua:170: { {
    bufnr = 3,
    col = 0,
    end_col = 0,
    end_lnum = 1,
    filename = "/tmp/deleteme/foo.proto",
    lnum = 0,
    message = "Files must have a package defined.",
    row = "1",
    severity = 1,
    source = "buf_lint"
  }, {
    bufnr = 3,
    col = 0,
    end_col = 0,
    end_lnum = 1,
    filename = "/tmp/deleteme/foo.proto",
    lnum = 0,
    message = 'Files must have a syntax explicitly specified. If no syntax is specified, the file defaults to "proto2".',
    row = "1",
    severity = 1,
    source = "buf_lint"
  } }
[TRACE Thu May 12 20:01:27 2022] .../site/pack/packer/start/null-ls.nvim/lua/null-ls/rpc.lua:146: received LSP notification for method textDocument/didSave
[TRACE Thu May 12 20:01:27 2022] ...ack/packer/start/null-ls.nvim/lua/null-ls/generators.lua:21: running generators for method NULL_LS_DIAGNOSTICS_ON_SAVE
[DEBUG Thu May 12 20:01:27 2022] ...ack/packer/start/null-ls.nvim/lua/null-ls/generators.lua:24: no generators available
[TRACE Thu May 12 20:01:28 2022] .../site/pack/packer/start/null-ls.nvim/lua/null-ls/rpc.lua:146: received LSP notification for method textDocument/didClose
[TRACE Thu May 12 20:01:28 2022] .../site/pack/packer/start/null-ls.nvim/lua/null-ls/rpc.lua:146: received LSP notification for method textDocument/didOpen
[TRACE Thu May 12 20:01:28 2022] ...ack/packer/start/null-ls.nvim/lua/null-ls/generators.lua:21: running generators for method NULL_LS_DIAGNOSTICS_ON_OPEN
[DEBUG Thu May 12 20:01:28 2022] ...t/null-ls.nvim/lua/null-ls/helpers/generator_factory.lua:344: spawning command "buf" at /home/user with args { "lint", "/tmp/deleteme/foo.proto#include_package_files=true" }
[TRACE Thu May 12 20:01:28 2022] ...t/null-ls.nvim/lua/null-ls/helpers/generator_factory.lua:217: error output: /tmp/deleteme/foo.proto:1:1:Files must have a package defined.

[TRACE Thu May 12 20:01:28 2022] ...t/null-ls.nvim/lua/null-ls/helpers/generator_factory.lua:218: output: nil
[TRACE Thu May 12 20:01:28 2022] ...ck/packer/start/null-ls.nvim/lua/null-ls/diagnostics.lua:169: received diagnostics from source 1
[TRACE Thu May 12 20:01:28 2022] ...ck/packer/start/null-ls.nvim/lua/null-ls/diagnostics.lua:170: { {
    bufnr = 3,
    col = 0,
    end_col = 0,
    end_lnum = 1,
    filename = "/tmp/deleteme/foo.proto",
    lnum = 0,
    message = "Files must have a package defined.",
    row = "1",
    severity = 1,
    source = "buf_lint"
  } }
[TRACE Thu May 12 20:01:32 2022] .../site/pack/packer/start/null-ls.nvim/lua/null-ls/rpc.lua:146: received LSP notification for method textDocument/didChange
[TRACE Thu May 12 20:01:32 2022] ...ack/packer/start/null-ls.nvim/lua/null-ls/generators.lua:21: running generators for method NULL_LS_DIAGNOSTICS
[DEBUG Thu May 12 20:01:32 2022] ...t/null-ls.nvim/lua/null-ls/helpers/generator_factory.lua:344: spawning command "buf" at /home/user with args { "lint", "/tmp/deleteme/foo.proto#include_package_files=true" }
[TRACE Thu May 12 20:01:32 2022] ...t/null-ls.nvim/lua/null-ls/helpers/generator_factory.lua:217: error output: /tmp/deleteme/foo.proto:1:1:Files must have a package defined.

[TRACE Thu May 12 20:01:32 2022] ...t/null-ls.nvim/lua/null-ls/helpers/generator_factory.lua:218: output: nil
[TRACE Thu May 12 20:01:32 2022] ...ck/packer/start/null-ls.nvim/lua/null-ls/diagnostics.lua:169: received diagnostics from source 1
[TRACE Thu May 12 20:01:32 2022] ...ck/packer/start/null-ls.nvim/lua/null-ls/diagnostics.lua:170: { {
    bufnr = 3,
    col = 0,
    end_col = 0,
    end_lnum = 1,
    filename = "/tmp/deleteme/foo.proto",
    lnum = 0,
    message = "Files must have a package defined.",
    row = "1",
    severity = 1,
    source = "buf_lint"
  } }
[TRACE Thu May 12 20:01:33 2022] .../site/pack/packer/start/null-ls.nvim/lua/null-ls/rpc.lua:146: received LSP notification for method textDocument/didChange
[TRACE Thu May 12 20:01:33 2022] ...ack/packer/start/null-ls.nvim/lua/null-ls/generators.lua:21: running generators for method NULL_LS_DIAGNOSTICS
[DEBUG Thu May 12 20:01:33 2022] ...t/null-ls.nvim/lua/null-ls/helpers/generator_factory.lua:344: spawning command "buf" at /home/user with args { "lint", "/tmp/deleteme/foo.proto#include_package_files=true" }
[TRACE Thu May 12 20:01:33 2022] ...t/null-ls.nvim/lua/null-ls/helpers/generator_factory.lua:217: error output: /tmp/deleteme/foo.proto:1:1:Files must have a package defined.

[TRACE Thu May 12 20:01:33 2022] ...t/null-ls.nvim/lua/null-ls/helpers/generator_factory.lua:218: output: nil
[TRACE Thu May 12 20:01:33 2022] ...ck/packer/start/null-ls.nvim/lua/null-ls/diagnostics.lua:169: received diagnostics from source 1
[TRACE Thu May 12 20:01:33 2022] ...ck/packer/start/null-ls.nvim/lua/null-ls/diagnostics.lua:170: { {
    bufnr = 3,
    col = 0,
    end_col = 0,
    end_lnum = 1,
    filename = "/tmp/deleteme/foo.proto",
    lnum = 0,
    message = "Files must have a package defined.",
    row = "1",
    severity = 1,
    source = "buf_lint"
  } }
[TRACE Thu May 12 20:01:34 2022] .../site/pack/packer/start/null-ls.nvim/lua/null-ls/rpc.lua:146: received LSP notification for method textDocument/didSave
[TRACE Thu May 12 20:01:34 2022] ...ack/packer/start/null-ls.nvim/lua/null-ls/generators.lua:21: running generators for method NULL_LS_DIAGNOSTICS_ON_SAVE
[DEBUG Thu May 12 20:01:34 2022] ...ack/packer/start/null-ls.nvim/lua/null-ls/generators.lua:24: no generators available
[TRACE Thu May 12 20:01:35 2022] .../site/pack/packer/start/null-ls.nvim/lua/null-ls/rpc.lua:146: received LSP notification for method textDocument/didClose
[TRACE Thu May 12 20:01:35 2022] .../site/pack/packer/start/null-ls.nvim/lua/null-ls/rpc.lua:146: received LSP notification for method textDocument/didOpen
[TRACE Thu May 12 20:01:35 2022] ...ack/packer/start/null-ls.nvim/lua/null-ls/generators.lua:21: running generators for method NULL_LS_DIAGNOSTICS_ON_OPEN
[DEBUG Thu May 12 20:01:35 2022] ...t/null-ls.nvim/lua/null-ls/helpers/generator_factory.lua:344: spawning command "buf" at /home/user with args { "lint", "/tmp/deleteme/foo.proto#include_package_files=true" }
[TRACE Thu May 12 20:01:35 2022] ...t/null-ls.nvim/lua/null-ls/helpers/generator_factory.lua:217: error output: /tmp/deleteme/foo.proto:1:10:syntax value must be "proto2" or "proto3"

[TRACE Thu May 12 20:01:35 2022] ...t/null-ls.nvim/lua/null-ls/helpers/generator_factory.lua:218: output: nil
[TRACE Thu May 12 20:01:35 2022] ...ck/packer/start/null-ls.nvim/lua/null-ls/diagnostics.lua:169: received diagnostics from source 1
[TRACE Thu May 12 20:01:35 2022] ...ck/packer/start/null-ls.nvim/lua/null-ls/diagnostics.lua:170: { {
    bufnr = 3,
    col = 9,
    end_col = 0,
    end_lnum = 1,
    filename = "/tmp/deleteme/foo.proto",
    lnum = 0,
    message = 'syntax value must be "proto2" or "proto3"',
    row = "1",
    severity = 1,
    source = "buf_lint"
  } }

Help

No

Implementation help

No response

Requirements

• I have read and followed the instructions above and understand that my issue will be closed if I did not provide the required information.

[jose-elias-alvarez]

This most likely indicates that the source doesn't actually accept stdin and is instead reading the file from the disk. If it doesn't have a way to use stdin, we can instead use a temp file. Both changes should be pretty simple to test and implement, so I'm happy to review a PR.

[AlxHnr]

This problem affects not just diagnostics, but also formatting with buf. vim.lsp.buf.formatting_sync() restores the buffer to a previous state sometimes. This happens very often when formatting on BufWritePre.

[jose-elias-alvarez]

This problem affects not just diagnostics, but also formatting with buf. vim.lsp.buf.formatting_sync() restores the buffer to a previous state sometimes. This happens very often when formatting on BufWritePre.

I suspect the root cause is the same - buf could be formatting the actual file on disk instead of passing formatted output to stdout. We'd also be able to handle this case by using temp files.

Edit: I found this issue on the repository that clarifies that stdin is not currently supported for linting or formatting. It sounds like linting requires additional project context, so we may run into issues when using temp files (which are created outside of the project), but it's worth a shot, and I'd at least expect formatting to work normally when using a temp file. An alternative for diagnostics would be to run only on save, which is what we do for similar cases like golangci-lint.

[AlxHnr]

An alternative for diagnostics would be to run only on save, which is what we do for similar cases like golangci-lint.

This is what ale is doing. They have way more people working on their linter integration, and still decided to lint only when saving. We should just do the same. One day buf will hopefully add proper stdin support. Then we can switch back to continuous linting.

The only remaining question is how to fix the formatter. Especially since we want to support range-based formatting. I don't see any practical solution other than waiting for buf's stdin support. Otherwise it would take a lot of extra logic just for one singe formatter.

[jose-elias-alvarez]

Makes sense to me, let’s try running the linter on save and see how that works (it should be a one-line change). I think it’s worth trying the formatted with a temp file, too - it’s another simple change and should work if no additional context is needed to format (which is what the linked issue implies).

[AlxHnr]

Just push the fix for buf diagnostics. There is nothing to "try out" here. The current situation is already broken and a fix can't make it possibly worse.

[jose-elias-alvarez]

You seem to have an entitlement issue - all you’ve done is complain about how things don’t work the way you expect, and if you can’t even be bothered to put in a PR to fix this, then I don’t think there’s any reason to continue this conversation.