Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document / adjust defaults for possible connection (already) closed issue when used on Swarm (due to IPVS) #1110

Open
u1735067 opened this issue Jul 17, 2023 · 1 comment

Comments

@u1735067
Copy link

u1735067 commented Jul 17, 2023

The issue #538 introduced a warning in the README about the Docker Swarm IPVS LB that timeouts TCP connections after 900 secs, which is lower than tcp_keepalive_time, so idle connections might become unavailable (cut by IPVS) while still being needed later, and that warning have since been removed (docker-library/docs@5e28015), probably due to the link being broken.

Could it be possible to reintroduce a warning about this, and/or to propose a default -ctcp_keepalives_idle=870 value (for example; and maybe tcp_keepalives_interval+tcp_keepalives_count)?

The old "success" documentation is visible at https://web.archive.org/web/20200611114911/https://success.docker.com/article/ipvs-connection-timeout-issue.

Possible solutions are:

  • connect to postgres without the LB using tasks.<service_name>
    • client must resolve on each connection in case the IP changed
  • use endpoint_mode: dnsrr to prevent using the LB
    • client must resolve on each connection in case the IP changed
  • use net.ipv4.tcp_keepalive_time: 870 (<900)
  • use -ctcp_keepalives_idle=870 (<900)

Example of all solutions (only one needed):

services:
  postgres:
    command:
      - postgres
      - -ctcp_keepalives_idle=300  # < 900
      # Maybe this too ?
      # - -ctcp_keepalives_interval=30
      # - -ctcp_keepalives_count=5
    sysctls:
      net.ipv4.tcp_keepalive_time: 720  # < 900
    deploy:
      endpoint_mode: dnsrr  # The client should resolve on each connection in case the task (IP) changed

  my_other_service:
    environment:
      POSTGRES_HOST: tasks.postgres  # The client should resolve on each connection in case the task (IP) changed
@tianon
Copy link
Member

tianon commented Dec 14, 2023

Sorry for the delay! 😭

Unfortunately, I think tuning PostgreSQL for use within Swarm is probably out of scope for this repository. 🙈 😞

Maybe we can add back a really small blurb about the problem in the docs, perhaps using this issue as our link instead of that old success article?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants