bug: Panic in agent.structsAllocDeploymentStatusToApi

[alexdulin]

Nomad version

Nomad v0.12.2 (ee69b33)

Operating system and Environment details

Linux ip-172-31-63-158 4.15.0-1079-aws #83~16.04.1-Ubuntu SMP Tue Jul 14 15:31:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Issue

Nomad server nodes are occasionally showing panics in their logs. It does not bring down the process, but they are occurring every couple of minutes. This only started after upgrading from 0.12.1 to 0.12.2.

Nomad Server logs (if appropriate)

2020-08-12T18:06:42.737Z [ERROR] http: http: panic serving [REDACTED]:36052: runtime error: invalid memory address or nil pointer dereference
goroutine 230394 [running]:
net/http.(*conn).serve.func1(0xc007c75900)
	net/http/server.go:1800 +0x139
panic(0x2b7e160, 0x533c310)
	runtime/panic.go:975 +0x3e3
github.com/hashicorp/nomad/command/agent.structsAllocDeploymentStatusToApi(...)
	github.com/hashicorp/nomad/command/agent/csi_endpoint.go:420
github.com/hashicorp/nomad/command/agent.structsAllocListStubToApi(0xc00768d080, 0xc0072f7170)
	github.com/hashicorp/nomad/command/agent/csi_endpoint.go:407 +0x2f9
github.com/hashicorp/nomad/command/agent.structsCSIPluginToApi(0xc0067a3c00, 0xc0006511a0)
	github.com/hashicorp/nomad/command/agent/csi_endpoint.go:296 +0x43e
github.com/hashicorp/nomad/command/agent.(*HTTPServer).CSIPluginSpecificRequest(0xc0006aa280, 0x38eac00, 0xc0006511a0, 0xc0061e2f00, 0x4d78e6, 0x5f342fb2, 0x2bf1a085, 0x15d4728535f)
	github.com/hashicorp/nomad/command/agent/csi_endpoint.go:265 +0x351
github.com/hashicorp/nomad/command/agent.(*HTTPServer).wrap.func1(0x38eac00, 0xc0006511a0, 0xc0061e2f00)
	github.com/hashicorp/nomad/command/agent/http.go:448 +0x176
net/http.HandlerFunc.ServeHTTP(0xc000820200, 0x38eac00, 0xc0006511a0, 0xc0061e2f00)
	net/http/server.go:2041 +0x44
net/http.(*ServeMux).ServeHTTP(0xc00011ce40, 0x38eac00, 0xc0006511a0, 0xc0061e2f00)
	net/http/server.go:2416 +0x1a5
github.com/NYTimes/gziphandler.GzipHandlerWithOpts.func1.1(0x38f4c80, 0xc00037b180, 0xc0061e2f00)
	github.com/NYTimes/[email protected]/gzip.go:277 +0x1e6
net/http.HandlerFunc.ServeHTTP(0xc0006bd2f0, 0x38f4c80, 0xc00037b180, 0xc0061e2f00)
	net/http/server.go:2041 +0x44
net/http.serverHandler.ServeHTTP(0xc00064c000, 0x38f4c80, 0xc00037b180, 0xc0061e2f00)
	net/http/server.go:2836 +0xa3
net/http.(*conn).serve(0xc007c75900, 0x3906b80, 0xc004c64880)
	net/http/server.go:1924 +0x86c
created by net/http.(*Server).Serve
	net/http/server.go:2962 +0x35c

[tgross]

Hi @ alexdulin! Sorry about that. I've just landed a patch here: #8659 We're going to evaluate how quickly we can turn around a new patch release.

[alexdulin]

Sounds great. Thanks for finding a fix so quickly!

[github-actions]

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.