Feature Request: Configure dynamic port range

[zapman449]

Nomad version

Nomad v0.3.1

Operating system and Environment details

any

Issue

Allow MinDynamicPort and MaxDynamicPort to be specified by the user.

Workaround: compile nomad for yourself, after changing these values in nomad/structs/network.go

[dadgar]

Out of curiosity why do you need that ability? You can also reserve ports on the client that you do not want to be used.

[zapman449]

The reason is to make explaining somethings to a security auditor (PCI, HIPPA, ETC) easier. It's much easier to swallow opening up a smaller range of firewall ports in a highly secure environment than a 40k range of ports.

In the general case, these numbers are reasonable.

[kevincox]

I have another use case for this and a large concern with the current port range.

The use case is quite simple, existing environments may often have a set of ports reserved for this type of thing and adding new ports might be difficult. For example if I have a non-dynamic service that runs on port 10101 I don't want Nomad to try to start a service there, because it will probably fail, or even worse it could succeed if there is a blip in the other service and its port gets stolen. However if my existing environment used ports 20000-30000 for Mesos being able to set this port range for Nomad makes the transition that much easier because I don't have to worry about port conflicts.

Secondly the default port range conflicts with the default Linux ephemeral port range. This means that Nomad might try to assign a port that is free according to its accounting but can be used by any networking performed by other processes.

Both of these issues boil down to the fact that the range of ports that Nomad uses should be considered reserved and must not be allowed to be used by anything else on the server. This is quite a huge requirement to enforce on everyone using your software (especially when the current range is so broad) so I think that this range definitely needs to be configurable.

Side note about existing work: Mesos is awesome in this respect as each slave specifies what set of ports it has available, it is configurable on a per-port level. This is probably overkill but specifying a custom range, and ideally a range by "slave" would be amazing for Nomad.

[dadgar]

@kevincox You can reserve specific ports on the Nomad Clients such that they won't be used to avoid the case you described where a static service already is listening on that port.

https://www.nomadproject.io/docs/agent/config.html#reserved

[kevincox]

Thanks, I didn't see that. That'll work, I'll just reserve 0-19999,30000-65535, then it should be good.

Although I still highly recommend removing the default ephemeral port range and it would be awesome if this setting was fully configurable rather then having a hard coded base with exclusions allowed.

[dadgar]

Closing as there is a work around via exclusions. By having the default range, it makes the scheduling implementation more efficient since we can use a fast, memory efficient bit-map:

nomad/nomad/structs/network.go

Line 111 in 6247fa2

func (idx *NetworkIndex) AddReserved(n *NetworkResource) (collide bool) {

[kevincox]

I don't think the default range has any performance benefit. It would be trivial to allocate a bitmap for the ports specific to the given config.

[prestonp]

I'd like to revisit this because I have a use case where I need to use a range that's below the default hardcoded dynamic port range between port 20000 and 32000.

I propose adding a new block for NetworkResource called dynamic_port_range like this:

job "foo" {
  group "example" {
    task "server" {
      resources {
        network {
          mbits = 200
          port "http" {}
          port "https" {}

          dynamic_port_range {
              min = 4000
              max = 5000
          }
        }
      }
    }
  }
}

[github-actions]

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.