Skip to content

Commit

Permalink
helper: reduce size of buffer used by template connections
Browse files Browse the repository at this point in the history
In #12458 we added an in-memory connection buffer so that template runners that
want access to the Nomad API for Service Registration and Variables can
communicate with Nomad without having to create a real HTTP client. The size of
this buffer (1 MiB) was taken directly from its usage in Vault, and each
connection makes 2 such buffers (send and receive). Because each template runner
has its own connection, when there are large numbers of allocations this adds up
to significant memory usage.

The largest Nomad Variable payload is 64KiB, and a small amount of
metadata. Service Registration responses are much smaller, and we don't include
check results in them (as Consul does), so the size is relatively bounded. We
should be able to safely reduce the size of the buffer by a factor of 10 or more
without forcing the template runner to make multiple read calls over the buffer.

Fixes: #18508
  • Loading branch information
tgross committed Sep 15, 2023
1 parent ad4436f commit 921e755
Show file tree
Hide file tree
Showing 2 changed files with 7 additions and 1 deletion.
3 changes: 3 additions & 0 deletions .changelog/18524.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
```release-note:improvement
template: reduce memory usage associated with communicating with the Nomad API
```
5 changes: 4 additions & 1 deletion helper/bufconndialer/bufconndialer.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,10 @@ type BufConnWrapper struct {
// New returns a new BufConnWrapper with a new bufconn.Listener. The wrapper
// provides a dialer for creating connections to the listener.
func New() (net.Listener, *BufConnWrapper) {
ln := bufconn.Listen(1024 * 1024)
// this buffer is sized to accept a maximum-sized Nomad Variable payload
// (64k) with plenty of room to spare for the metadata and envelope, in a
// single read
ln := bufconn.Listen(1024 * 100)
return ln, &BufConnWrapper{listener: ln}
}

Expand Down

0 comments on commit 921e755

Please sign in to comment.