x/tools/gopls: panic in hover on v0.7.2-pre.1

[findleyr]

The panic below was reported by @egonelbre in slack:

The bug appears to be here, where s.End() is out of bounds of the file:
https://cs.opensource.google/go/x/tools/+/master:internal/lsp/source/hover.go;l=344;drc=f09387104b6e0982cb7ded78fee831c96ca49429

gopls: panic: invalid Pos value 6407642 (should be in [6406800, 6407641])
gopls: 
gopls: goroutine 5480 [running]:
gopls: go/token.(*File).Offset(0xc0011045a0, 0x61c5c6)
gopls:  c:/go/src/go/token/position.go:281 +0xeb
gopls: golang.org/x/tools/internal/lsp/source.HoverInfo({0x1703428, 0xc00130ade0}, {0x172d3c0, 0xc000eac100}, {0x1727188, 0xc000b1a240}, {0x1726ef8, 0xc00297e230}, {0x16fdd80, 0xc000704800}, ...)
gopls:  F:/Code/go.googlesource.com/tools/internal/lsp/source/hover.go:344 +0xaa5
gopls: golang.org/x/tools/internal/lsp/source/completion.(*completer).item(_, {_, _}, {{0x1726ef8, 0xc00297e230}, 0x4023364066f85c9d, {0xc002a7e6b8, 0x6}, {0x0, 0x0}, ...})
gopls:  F:/Code/go.googlesource.com/tools/internal/lsp/source/completion/format.go:238 +0x1850
gopls: golang.org/x/tools/internal/lsp/source/completion.(*completer).addCandidate(0xc00200aa00, {0x1703428, 0xc00130ade0}, 0xc0013f8bd8)
gopls:  F:/Code/go.googlesource.com/tools/internal/lsp/source/completion/deep_completion.go:285 +0x3f0
gopls: golang.org/x/tools/internal/lsp/source/completion.(*completer).deepSearch(0xc00200aa00, {0x1703428, 0xc00130ade0})
gopls:  F:/Code/go.googlesource.com/tools/internal/lsp/source/completion/deep_completion.go:169 +0x4a6
gopls: golang.org/x/tools/internal/lsp/source/completion.Completion({0x17033b8, 0xc000109800}, {0x172d3c0, 0xc000eac100}, {0x170f678, 0xc001104360}, {0x10e5358, 0xc0}, {0x0, {0x0, ...}})
gopls:  F:/Code/go.googlesource.com/tools/internal/lsp/source/completion/completion.go:557 +0x100b
gopls: golang.org/x/tools/internal/lsp.(*Server).completion(0xc000448028, {0x17033b8, 0xc000109800}, 0xc00297a050)
gopls:  F:/Code/go.googlesource.com/tools/internal/lsp/completion.go:32 +0x1f0
gopls: golang.org/x/tools/internal/lsp.(*Server).Completion(0xc00066a000, {0x17033b8, 0xc000109800}, 0x1465240)

[egonelbre]

Note, I do not use CRLF line-endings.

[findleyr]

Note, I do not use CRLF line-endings.

Aha, thanks for pointing that out. Updated the issue accordingly.

[gopherbot]

Change https://golang.org/cl/348429 mentions this issue: internal/lsp: check InRange before calling token.Offset

[stamblerre]

@egonelbre: Are you able to reproduce this panic, and if so, do you mind sharing the code and user actions that trigger it? I'm not able to reproduce it.

[egonelbre]

@stamblerre not reliably, but it does happen fairly frequently. I'll see whether I can figure out some pattern to the problem.

[gopherbot]

Change https://golang.org/cl/348530 mentions this issue: internal/lsp: check InRange before calling token.Offset

[egonelbre]

I can reliably reproduce the issue with a single file:

package main

type Example struct

In Sublime Text add a space after struct and it'll crash. Or try to hover over Example.

[egonelbre]

And here's a trace:

[Trace - 17:40:58.454 PM] Sending request 'initialize - (1)'.
Params: {"clientInfo":{"version":"1.10.1","name":"Sublime Text LSP"},"initializationOptions":{},"processId":20032,"workspaceFolders":[{"uri":"file:///F:/Temp/debug/basic","name":"basic"}],"rootUri":"file:///F:/Temp/debug/basic","rootPath":"F:\\Temp\\debug\\basic","capabilities":{"textDocument":{"hover":{"contentFormat":["markdown","plaintext"],"dynamicRegistration":true},"signatureHelp":{"signatureInformation":{"documentationFormat":["markdown","plaintext"],"parameterInformation":{"labelOffsetSupport":true}},"dynamicRegistration":true},"definition":{"dynamicRegistration":true,"linkSupport":true},"completion":{"completionItemKind":{"valueSet":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25]},"completionItem":{"tagSupport":{"valueSet":[1]},"snippetSupport":true,"deprecatedSupport":true,"labelDetailsSupport":true,"documentationFormat":["markdown","plaintext"],"resolveSupport":{"properties":["detail","documentation","additionalTextEdits"]},"insertTextModeSupport":{"valueSet":[2]}},"dynamicRegistration":true,"insertTextMode":2},"selectionRange":{"dynamicRegistration":true},"rename":{"dynamicRegistration":true,"prepareSupport":true},"implementation":{"dynamicRegistration":true,"linkSupport":true},"codeAction":{"resolveSupport":{"properties":["edit"]},"codeActionLiteralSupport":{"codeActionKind":{"valueSet":["quickfix","refactor","refactor.extract","refactor.inline","refactor.rewrite","source.organizeImports"]}},"dynamicRegistration":true,"dataSupport":true},"typeDefinition":{"dynamicRegistration":true,"linkSupport":true},"synchronization":{"didSave":true,"willSave":true,"dynamicRegistration":true,"willSaveWaitUntil":true},"documentSymbol":{"tagSupport":{"valueSet":[1]},"symbolKind":{"valueSet":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26]},"dynamicRegistration":true,"hierarchicalDocumentSymbolSupport":true},"documentHighlight":{"dynamicRegistration":true},"declaration":{"dynamicRegistration":true,"linkSupport":true},"rangeFormatting":{"dynamicRegistration":true},"references":{"dynamicRegistration":true},"formatting":{"dynamicRegistration":true},"codeLens":{"dynamicRegistration":true},"publishDiagnostics":{"versionSupport":true,"tagSupport":{"valueSet":[2,1]},"relatedInformation":true,"dataSupport":true,"codeDescriptionSupport":true},"colorProvider":{"dynamicRegistration":true}},"general":{"markdown":{"parser":"Python-Markdown","version":"3.2.2"},"regularExpressions":{"engine":"ECMAScript"}},"workspace":{"workspaceFolders":true,"symbol":{"tagSupport":{"valueSet":[1]},"symbolKind":{"valueSet":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26]},"dynamicRegistration":true},"configuration":true,"executeCommand":{},"applyEdit":true,"didChangeConfiguration":{"dynamicRegistration":true},"workspaceEdit":{"failureHandling":"abort","documentChanges":true}},"window":{"showMessage":{"messageActionItem":{"additionalPropertiesSupport":true}},"workDoneProgress":true,"showDocument":{"support":true}}}}


[Trace - 17:40:58.455 PM] Received response 'initialize - (1)' in 1ms.
Result: {"capabilities":{"textDocumentSync":{"openClose":true,"change":2,"save":{}},"completionProvider":{"triggerCharacters":["."],"completionItem":{}},"hoverProvider":true,"signatureHelpProvider":{"triggerCharacters":["(",","]},"definitionProvider":true,"typeDefinitionProvider":true,"implementationProvider":true,"referencesProvider":true,"documentHighlightProvider":true,"documentSymbolProvider":true,"codeActionProvider":{"codeActionKinds":["quickfix","refactor.extract","refactor.rewrite","source.fixAll","source.organizeImports"]},"codeLensProvider":{},"documentLinkProvider":{},"workspaceSymbolProvider":true,"documentFormattingProvider":true,"documentOnTypeFormattingProvider":{"firstTriggerCharacter":""},"renameProvider":{"prepareProvider":true},"foldingRangeProvider":true,"executeCommandProvider":{"commands":["gopls.add_dependency","gopls.add_import","gopls.apply_fix","gopls.check_upgrades","gopls.gc_details","gopls.generate","gopls.generate_gopls_mod","gopls.go_get_package","gopls.list_known_packages","gopls.regenerate_cgo","gopls.remove_dependency","gopls.run_tests","gopls.start_debugging","gopls.test","gopls.tidy","gopls.toggle_gc_details","gopls.update_go_sum","gopls.upgrade_dependency","gopls.vendor","gopls.workspace_metadata"]},"callHierarchyProvider":true,"workspace":{"workspaceFolders":{"supported":true,"changeNotifications":"workspace/didChangeWorkspaceFolders"}}},"serverInfo":{"name":"gopls","version":"{\"path\":\"golang.org/x/tools/gopls\",\"version\":\"(devel)\",\"deps\":[{\"path\":\"github.com/BurntSushi/toml\",\"version\":\"v0.4.1\",\"sum\":\"h1:GaI7EiDXDRfa8VshkTj7Fym7ha+y8/XxIgD2okUIjLw=\"},{\"path\":\"github.com/google/go-cmp\",\"version\":\"v0.5.6\",\"sum\":\"h1:BKbKCqvP6I+rmFHt06ZmyQtvB8xAkWdhFyr0ZUNZcxQ=\"},{\"path\":\"github.com/sergi/go-diff\",\"version\":\"v1.1.0\",\"sum\":\"h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0=\"},{\"path\":\"golang.org/x/mod\",\"version\":\"v0.4.2\",\"sum\":\"h1:Gz96sIWK3OalVv/I/qNygP42zyoKp3xptRVCWRFEBvo=\"},{\"path\":\"golang.org/x/sync\",\"version\":\"v0.0.0-20210220032951-036812b2e83c\",\"sum\":\"h1:5KslGYwFpkhGh+Q16bwMP3cOontH8FOep7tGV86Y7SQ=\"},{\"path\":\"golang.org/x/sys\",\"version\":\"v0.0.0-20210809222454-d867a43fc93e\",\"sum\":\"h1:WUoyKPm6nCo1BnNUvPGnFG3T5DUVem42yDJZZ4CNxMA=\"},{\"path\":\"golang.org/x/tools\",\"version\":\"v0.1.5\",\"replace\":{\"path\":\"../\"}},{\"path\":\"golang.org/x/xerrors\",\"version\":\"v0.0.0-20200804184101-5ec99f83aff1\",\"sum\":\"h1:go1bK/D/BFZV2I8cIQd1NKEZ+0owSTG1fDTci4IqFcE=\"},{\"path\":\"honnef.co/go/tools\",\"version\":\"v0.2.0\",\"sum\":\"h1:ws8AfbgTX3oIczLPNPCu5166oBg9ST2vNs0rcht+mDE=\"},{\"path\":\"mvdan.cc/gofumpt\",\"version\":\"v0.1.1\",\"sum\":\"h1:bi/1aS/5W00E2ny5q65w9SnKpWEF/UIOqDYBILpo9rA=\"},{\"path\":\"mvdan.cc/xurls/v2\",\"version\":\"v2.3.0\",\"sum\":\"h1:59Olnbt67UKpxF1EwVBopJvkSUBmgtb468E4GVWIZ1I=\"}]}"}}


[Trace - 17:40:58.456 PM] Sending notification 'initialized'.
Params: {}


[Trace - 17:40:58.457 PM] Received request 'window/workDoneProgress/create - (1)'.
Params: {"token":"5577006791947779410"}


[Trace - 17:40:58.459 PM] Sending notification 'textDocument/didOpen'.
Params: {"textDocument":{"version":0,"text":"package main\n\ntype Example struct","languageId":"go","uri":"file:///F:/Temp/debug/basic/main.go"}}


[Trace - 17:40:58.467 PM] Sending response 'window/workDoneProgress/create - (1)' in 10ms.
Result: 


[Trace - 17:40:58.467 PM] Received notification '$/progress'.
Params: {"token":"5577006791947779410","value":{"kind":"begin","title":"Setting up workspace","message":"Loading packages..."}}


[Trace - 17:40:58.467 PM] Received request 'workspace/configuration - (2)'.
Params: {"items":[{"scopeUri":"file:///F:/Temp/debug/basic","section":"gopls"}]}


[Trace - 17:40:58.470 PM] Sending response 'workspace/configuration - (2)' in 2ms.
Result: [null]


[Trace - 17:40:58.862 PM] Received notification 'window/logMessage'.
Params: {"type":3,"message":"2021/09/09 17:40:58 go env for F:\\Temp\\debug\\basic\n(root F:\\Temp\\debug\\basic)\n(go version go version go1.17 windows/amd64)\n(valid build configuration = false)\n(build flags: [])\nGO111MODULE=\nGOCACHE=Z:\\gocache\nGOMOD=NUL\nGONOPROXY=\nGOMODCACHE=F:\\Go\\pkg\\mod\nGONOSUMDB=\nGOPATH=F:\\Go\nGOPRIVATE=\nGOROOT=c:\\go\nGOFLAGS=\nGOPROXY=https://proxy.golang.org,direct\nGOINSECURE=\nGOSUMDB=sum.golang.org\n\n"}


[Trace - 17:40:59.249 PM] Received notification 'window/logMessage'.
Params: {"type":3,"message":"2021/09/09 17:40:59 go/packages.Load\n\tsnapshot=0\n\tdirectory=F:\\Temp\\debug\\basic\n\tquery=[./ builtin]\n\tpackages=2\n"}


[Trace - 17:40:59.249 PM] Received notification '$/progress'.
Params: {"token":"5577006791947779410","value":{"kind":"end","message":"Finished loading packages."}}


[Trace - 17:40:59.250 PM] Received notification 'textDocument/publishDiagnostics'.
Params: {"uri":"file:///F:/Temp/debug/basic/main.go","diagnostics":[{"range":{"start":{"line":3,"character":0},"end":{"line":3,"character":0}},"severity":1,"source":"syntax","message":"expected ';', found 'EOF'"}]}


[Trace - 17:40:59.252 PM] Received request 'client/registerCapability - (3)'.
Params: {"registrations":[{"id":"workspace/didChangeConfiguration","method":"workspace/didChangeConfiguration"},{"id":"workspace/didChangeWorkspaceFolders","method":"workspace/didChangeWorkspaceFolders"}]}


[Trace - 17:40:59.271 PM] Sending request 'textDocument/codeAction - (2)'.
Params: {"textDocument":{"uri":"file:///F:/Temp/debug/basic/main.go"},"context":{"diagnostics":[]},"range":{"start":{"line":0,"character":0},"end":{"line":0,"character":0}}}


[Trace - 17:40:59.272 PM] Sending response 'client/registerCapability - (3)' in 19ms.
Result: 


[Trace - 17:40:59.273 PM] Received notification 'window/logMessage'.
Params: {"type":1,"message":"2021/09/09 17:40:59 imports fixes: AllImportsFixes: F:\\Temp\\debug\\basic\\main.go:3:20: expected ';', found 'EOF' (and 2 more errors)\n\tfile=\"F:\\\\Temp\\\\debug\\\\basic\\\\main.go\"\n"}


[Trace - 17:40:59.273 PM] Received response 'textDocument/codeAction - (2)' in 1ms.
Result: null


[Trace - 17:41:04.961 PM] Sending request 'textDocument/hover - (3)'.
Params: {"textDocument":{"uri":"file:///F:/Temp/debug/basic/main.go"},"position":{"line":2,"character":9}}


--- crash from sublime text ----

gopls: 2021/09/09 17:40:59 go/packages.Load
	snapshot=0
	directory=F:\Temp\debug\basic
	query=[./ builtin]
	packages=2

gopls: 2021/09/09 17:40:59 imports fixes: AllImportsFixes: F:\Temp\debug\basic\main.go:3:20: expected ';', found 'EOF' (and 2 more errors)
	file="F:\\Temp\\debug\\basic\\main.go"

gopls: panic: invalid Pos value 103 (should be in [69, 102])
gopls: 
gopls: goroutine 277 [running]:
gopls: go/token.(*File).Offset(0xc000606960, 0x53)
gopls: 	c:/go/src/go/token/position.go:281 +0xeb
gopls: golang.org/x/tools/internal/lsp/source.HoverInfo({0xe63460, 0xc000178e10}, {0xe8d3c0, 0xc00031f300}, {0xe87188, 0xc000248300}, {0xe86ef8, 0xc000cfce10}, {0xe5dd80, 0xc000699040}, ...)
gopls: 	F:/Code/go.googlesource.com/tools/internal/lsp/source/hover.go:344 +0xaa5
gopls: golang.org/x/tools/internal/lsp/source.HoverIdentifier({0xe633b8, 0xc0001f60c0}, 0xc0000bda40)
gopls: 	F:/Code/go.googlesource.com/tools/internal/lsp/source/hover.go:101 +0x134
gopls: golang.org/x/tools/internal/lsp/source.Hover({0xe633b8, 0xc0001f60c0}, {0xe8d3c0, 0xc00031f300}, {0xe6f678, 0xc000165560}, {0xbc6240, 0x0})
gopls: 	F:/Code/go.googlesource.com/tools/internal/lsp/source/hover.go:71 +0x8b
gopls: golang.org/x/tools/internal/lsp.(*Server).hover(0xc000498ad8, {0xe633b8, 0xc0001f60c0}, 0xc00079ffb0)
gopls: 	F:/Code/go.googlesource.com/tools/internal/lsp/hover.go:26 +0x267
gopls: golang.org/x/tools/internal/lsp.(*Server).Hover(0xc000039ab0, {0xe633b8, 0xc0001f60c0}, 0xbc6240)
gopls: 	F:/Code/go.googlesource.com/tools/internal/lsp/server_gen.go:128 +0x25
gopls: golang.org/x/tools/internal/lsp/protocol.serverDispatch({0xe633b8, 0xc0001f60c0}, {0xe91540, 0xc000cff200}, 0xc00079ff80, {0xe63620, 0xc0001f6000})
gopls: 	F:/Code/go.googlesource.com/tools/internal/lsp/protocol/tsserver.go:372 +0x12c5
gopls: golang.org/x/tools/internal/lsp/protocol.ServerHandler.func1({0xe633b8, 0xc0001f60c0}, 0xc00079ff80, {0xe63620, 0xc0001f6000})
gopls: 	F:/Code/go.googlesource.com/tools/internal/lsp/protocol/protocol.go:154 +0x90
gopls: golang.org/x/tools/internal/lsp/lsprpc.handshaker.func1({0xe633b8, 0xc0001f60c0}, 0xc00079ff80, {0xe63620, 0xc0001f6000})
gopls: 	F:/Code/go.googlesource.com/tools/internal/lsp/lsprpc/lsprpc.go:506 +0xa7d
gopls: golang.org/x/tools/internal/jsonrpc2.MustReplyHandler.func1({0xe633b8, 0xc0001f60c0}, 0xc000115e78, {0xe63620, 0xc0001f6000})
gopls: 	F:/Code/go.googlesource.com/tools/internal/jsonrpc2/handler.go:35 +0xf6
gopls: golang.org/x/tools/internal/jsonrpc2.AsyncHandler.func1.2()
gopls: 	F:/Code/go.googlesource.com/tools/internal/jsonrpc2/handler.go:103 +0xa3
gopls: created by golang.org/x/tools/internal/jsonrpc2.AsyncHandler.func1
gopls: 	F:/Code/go.googlesource.com/tools/internal/jsonrpc2/handler.go:100 +0x20f

[stamblerre]

Thanks for the repro case! I was able to reproduce and add a test for this.

[gopherbot]

Change https://golang.org/cl/348930 mentions this issue: internal/lsp: remove the TODO and add a test for the hover panic

[findleyr]

Filed #48300 for the idiosyncracy in go/ast and go/parser that contributed to this panic, though the gopls code should have been more robust.

I think we can close this with CL 348930. Thanks again @egonelbre for reporting.

[gopherbot]

Change https://golang.org/cl/348969 mentions this issue: internal/lsp: remove the TODO and add a test for the hover panic