Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding TCP keepalives support for the broker's connection #408

Merged
merged 6 commits into from
Apr 6, 2015

Conversation

epsniff
Copy link
Contributor

@epsniff epsniff commented Apr 3, 2015

Addresses issue #407

Our Google Compute Engine applications have a couple of under utilized topics/partitions that only receive messages once an hour or so. Google Compute Engine's networking automatically kills any connection thats been idle for more than 10mins. This causes the Sarama clients on GCE to return this error read tcp 10.10.10.10:9093: i/o timeout.

From Google Compute Engine's Docs:

Note that a system-wide "hidden" firewall rule is set to disconnect idle TCP connections after 10
minutes. If your instance initiates or accepts long-lived connections with an external host, you can
adjust TCP keep-alive settings to prevent these timeouts from dropping connections. ...

Keepalive Refs:

PS thanks for all the hard work! You guys rock!

@@ -12,6 +12,10 @@ type Config struct {
DialTimeout time.Duration // How long to wait for the initial connection to succeed before timing out and returning an error (default 30s).
ReadTimeout time.Duration // How long to wait for a response before timing out and returning an error (default 30s).
WriteTimeout time.Duration // How long to wait for a transmit to succeed before timing out and returning an error (default 30s).

// KeepAlive specifies the keep-alive period for an active network connection.
// If zero, keep-alives are not enabled.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"not enabled" == "disabled"

Also, please mention that the default is 0 (disabled).

@eapache
Copy link
Contributor

eapache commented Apr 4, 2015

A few nits, but generally LGTM. You're welcome to add a line to the changelog if you wish.

@@ -78,6 +79,7 @@ func TestClientDoesntCachePartitionsForTopicsWithErrors(t *testing.T) {
seedBroker.Returns(metadataResponse)

config := NewConfig()
config.Net.KeepAlive = 12 * time.Millisecond
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure whether this is a useful addition given that we don't actually test it.

@wvanbergen
Copy link
Contributor

LGTM apart from the nitpicks.

@epsniff
Copy link
Contributor Author

epsniff commented Apr 6, 2015

@wvanbergen or @eapache - Does the new config setting need a validator ? https://github.com/Shopify/sarama/blob/master/config.go#L187

The other nits were addressed. :)

@eapache
Copy link
Contributor

eapache commented Apr 6, 2015

Assuming that a setting <0 is incoherent then yes, a validator is important - good catch!

eapache added a commit that referenced this pull request Apr 6, 2015
Adding TCP keepalives support for the broker's connection
@eapache eapache merged commit 9b048b0 into IBM:master Apr 6, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants