WIP: Docker networking support

[aanand]

This is starting out as a docs-only PR, meant to get early feedback on the direction of Docker's networking support, from the viewpoint of Compose. It's scoped to what we plan to implement in the first stable release.

Your comments are welcomed.

Replaces #1346.

ping @dave-tucker @bfirsh @shykes @mavenugo @lxpollitt @monadic

[nazar-pc]

Does links going to be dropped from Docker itself like you did in docs? Never heard about that.
In my opinion it was quite useful to be strict about who can access whom.
Creating separate network for each app is OK, but few network for app without really any need is overkill I think.

Also you do not currently mention how to provide aliases like db:mysql in links with publishing.

Also

Looking up the name web will return the IP address of one of them, but Docker and Compose do not provide any guarantees about which one

Is quite a big issue, there should be a way to differentiate them, adding suffixes would allow, for instance, load balancer to discover all web servers automatically to do its job.
Or labels might be used here? I think it worth to mention this explicitly as recommended way to do things.

[lxpollitt]

I think this section should focus on providing the flexibility to run multiple apps in a single network. For example, docker compose up -d --net=<network> (and any equivalents of that which might feature directly within the Compose file if we want to support specifying networks in the Compose file).

I don't think support for containers connecting to multiple networks in 1.8 timeframe is realistic. I think there are broad range of libnetwork, network driver and service discovery interaction and implementation issues that need to be resolved to make that work sensibly. I suggest restricting each container to a single network for 1.8.

[squaremo]

Yep. Everything works neatly in the example given, but it's easy to construct a scenario which is not so clear; for instance if there are containers published as foo in both networks, what answer should web get if it asks for foo? (You could answer "just don't do that" I suppose, but it would be better to come up with a model that avoids such an ambiguity).

[monadic]

+1

[aanand]

OK, I'm just going to remove this section for now.

[squaremo]

You may want to say "via the system resolver" or similar here, instead of DNS specifically. Some solutions (e.g., the incumbent) will use /etc/hosts as a mechanism.

[aanand]

Good point. I've reworded this sentence.

[moxiegirl]

@aanand @bfirsh If this is experimental in the next release, do y'all want to create an experimental directory to hold the documentation? Then we can point to this directory from release notes and blogs.

The idea is only released features go in docs.docker.com and experimental feature documentation goes into an experimental directory. That way, "risky" features aren't presented to users who might miss the experimental aspect in the official docs.

@aanand @bfirsh please let me know which way you want to go

[aanand]

Just an update: I've started on an implementation of the networking API in docker-py: docker/docker-py#729

Following conversation with @mavenugo, we're considering renaming "networks" to "scopes" (at the Docker Engine and API level), and removing all talk of networking, since that's not the real developer-level concern (rather, it's service discovery).

I'm going to push an update to the docs to that effect.

[moxiegirl]

@aanand Can you give me a definition of scope in this context? I was looking through the PR and trying to replace "network" with "scope" as used...it wasn't coming out clean. Scope is a synonym for range. And depending on how you use it in context could cause confusion.

[aanand]

@moxiegirl Sure. It's similar to how programming languages define "scope", as in "a context in which a particular name means a particular thing".

I'll ignore Compose for a moment and demonstrate using hypothetical Docker client syntax.

Suppose I have two applications. I create a scope for each, called say app1 and app2.

$ docker service scope create app1
$ docker service scope create app2
$ docker service scope ls
ID             NAME       NETWORK DRIVER  DISCOVERY DRIVER
e722ff412f35   app1       overlay         default
f832ff40dfb7   app2       overlay         default

In each app's scope, there is a set of services, each with a name that's unique within that scope.

A container in that app's scope can publish as a backend for a particular service.

docker run has a --service flag for publishing a container as a backend for a service, creating the service if it doesn't already exist.

$ docker run -d --service app1.db postgres
abc123

$ docker run -d --service app2.db mysql
def456

$ docker service ls
NAME      SCOPE          BACKEND        IP         PORTS
db        app1           abc123         10.42.0.1  5432:5432
db        app2           def456         10.42.0.3  3306:3306

Within the app1 scope, the hostname db resolves to the IP of app1's database container.

$ docker run --service app1.web app1-web-image ping db
PING db (10.42.0.1) 56(84) bytes of data.
64 bytes from db (10.42.0.1): icmp_seq=1 ttl=64 time=997 ms
64 bytes from db (10.42.0.1): icmp_seq=2 ttl=64 time=0.037 ms
^C

Within the app2 scope, the hostname db resolves to the IP of app2's database container.

$ docker run --service app2.web app2-web-image ping db
PING db (10.42.0.3) 56(84) bytes of data.
64 bytes from db (10.42.0.3): icmp_seq=1 ttl=64 time=997 ms
64 bytes from db (10.42.0.3): icmp_seq=2 ttl=64 time=0.037 ms
^C

[moxiegirl]

@aanand So, I'm expecting the work scope to appear in the networking.md file you have provided and it isn't. In the documentation, I'm expecting to see how the scope related to the network as compose understands it, I see in your pseudo command above a NETWORK DRIVER column and no mention of driver in networking.md either... Are you planning on updating network.md?

Also, I'm still waiting to hear if this is "experimental" or going out finished...whatcha thinking?

[mavenugo]

@moxiegirl thats correct. it will appear in networking.md shortly. Also, the word NETWORK DRIVER and NETWORK PLUGINS are used interchangably. To be technically correct, NETWORK DRIVER is the right usage. Plugins provide a way for drivers to be external to the docker daemon process.

the experimental networking features will be part of 1.9 proper release & it is appropriate to have that assumption in compose.

[lxpollitt]

@aanand, @mavenugo: Can you confirm that the scope applies to both service discovery and reachability. i.e. If I bypass service discovery and just try direct IP address access then the scope limits that too. (I think that was the plan for the old network concept, but wanted to double check in case the thinking has moved on for scope.)

[mavenugo]

@lxpollitt thats correct. scope covers both service discovery and service reachability. It is enabled by default and the users will get these features out of the box. I would say that the thinking is converging well in this direction.

[moxiegirl]

@mavenugo I'm sorted thanks. @aanand When you are ready for a docs review let me know.

[shykes]

Alex, yes that is correct. They are still networks, nothing has changed. We
are just experimenting with different words to call them something other
than "network". Honestly people seem pretty confused by the word "scope" so
far so we might switch back. In any case, the underlying concepts have not
changed.

On Wed, Aug 26, 2015 at 2:25 PM, moxiegirl [email protected] wrote:

@mavenugo https://github.com/mavenugo I'm sorted thanks. @aanand
https://github.com/aanand When you are ready for a docs review let me
know.

—
Reply to this email directly or view it on GitHub
#1676 (comment).

[lxpollitt]

@shykes Do you have any feel for whether it is general developers (i.e. users of Docker) or network people (i.e. libnetwork plug-in implementors) getting confused? Or is it just people in general? I have a networking focus in my interests here, but I really don't mind what it is called, so long as its definition is clear and I don't foresee implementation challenges. Given that scope here is being defined as both a limit on service discovery and a limit on reachability (or at least default discoverability and reachability), then it seems perfectly reasonable to me. But people do get hung up on names surprisingly often!

Where it gets trickier from for service discover and networking implementations is around whether a single container can exist in / be connected to multiple scopes and what the expected behavior is in those cases. I don't know what the latest thinking on that is, but that's probably a discussion for another day.

[nazar-pc]

For me personally it is tricky because "scope" is not directly associated with network alone - Docker have FS layer with its own drivers, for instance. Why not just call it "network", no one will be confused about what it actually does.

[aanand]

Attention: this has been updated with the docs for the final planned functionality for Docker 1.9 / Compose 1.5. Please take a second look.

[bfirsh]

Nice!

My only concern is that removing links is a huge change for users. It might be worth sprinkling around some helpful notes. "links are no longer used in Compose 1.5... check out the networking docs to read about the future"

Also - presumably we will force users to update to Docker 1.9 in Compose 1.5? If a user doesn't want to upgrade to 1.9, we should make sure users can still find the Compose 1.4 docs with "links", etc.

[ibuildthecloud]

@mavenugo @aanand Does the section Configure how services are published still apply?

[ibuildthecloud]

How does a compose project use any network driver besides the default driver? Is it possible for a compose project to join an existing network?

[ibuildthecloud]

I'm not fully comfortable with this proposal. First off, I'm glad to see the docker networking UX finally land in 1.9, but I think the jury's still out on how users will use networks. Here's my observations thus far...

Users really want cross host networking. That has been said loud and clear. Beyond that simple request I don't have a clear picture of why users want networks as a first class object from an application perspective. I work with a good amount of users and I rarely get requests where people want VPC-like or OpenStack Neutron like functionality in Docker from an application perspective. What I do get is a lot of requests where people want to plug docker into an existing network fabric. OVS, VLANs, or something crazy. They have a network managed by another team and they just need the Docker containers to play nice with the existing networking model. So you can see that this is more of an administrative function and not a user/developer function. Once Docker 1.9 is out I will be working with a lot of users to address these one off networking requirements.

I would much rather see compose hold off on using networks as a core part of the functionality. I think compose should support net: foo, but beyond that don't change until we understand better how networking will be used. My fear is that if compose makes core assumption of networking it will make it not possible to use compose in certain situations.

For example, I am working with some one now that must plug Docker into OVS in an extremely custom way. The way in which they are doing this, having a single Docker network for their custom set up makes sense, and then use that network as the default. They have no requirement to create multiple "networks" in Docker. Now if I want to deploy two compose templates and both have a container named "foo" the only way I can do that after this proposed change would be to create two networks. But... in the scenario I described they can't create two networks. The only way I can support two network with this OVS approach is if I write a driver that creates "pseudo" networks. Now this just gets really odd.

What compose really needs is service discovery namespacing, not networks. Again, I think compose should just hold off a bit. Lets see how users used Docker networks first.

[dnephin]

I think this proposal supports the situation you describe with publish:. Using publish you could either vary the name using environment variable interpolation (#1765 new in 1.5.0) or by using an override to name the services (#2051, also new in 1.5.0). I believe that would let you run two copies on the same network. The network you join would be based on the project name, so you can configure that as well.

That said, I think it would be good to allow users to customize the default network (if they don't want it to be the project name), and to allow them to completely disable creating a default network. I don't know if those features will be available for the initial release.

[aanand]

A small update: following discussion with @mavenugo and @dave-tucker, we're going to use hostname for configuring the name by which a container joins the network.

How it'll work is: if you've set a hostname in docker-compose.yml, Compose sends that to the Engine, which uses it as the name for the container to join under. If you haven't set a hostname, Compose sets it to the service name for you.

This means that joining under multiple names, or no names at all, will not be supported for v1 - we'll be holding off on those until there's an actual service discovery API.

I've updated the doc:

https://github.com/aanand/fig/blob/3648c2c52bd8fb98f046f29d10640996934901c4/docs/networking.md#configure-how-services-are-published

[ibuildthecloud]

@aanand How does scaling work now. From what I understand you are saying either service name from compose or the hostname will be set to the container name (ie --name). This seems to indicate you can't scale the service anymore.

[aanand]

@ibuildthecloud The container name isn't used anywhere. If you've set hostname in your docker-compose.yml:

web:
  hostname: mywebservice

then Compose will send "Hostname": "mywebservice" when creating each container for web.

If you haven't set a hostname, then Compose will send "Hostname": "web".

In both cases, each container will join the network under that name. If you create multiple containers, they'll join under the same name. If you look up the hostname web inside any container that's connected to the network, you'll get the IP address of one of the web containers back, but precisely which one is left up to Linux. Under the hood, there are simply multiple entries in each container's /etc/hosts.

This is basically the same behaviour that scaling gives you today. It's not very useful, but it's a start - we need a proper load-balancing solution to make it viable.

[pwaller]

@aanand is there a way to specify container scaling in the yml, so that docker-compose up brings up a functional cluster?

[aanand]

@pwaller see #1661 - it's still under discussion.

[mnowster]

The link to the git commit in the requirements isn't working, I think that's why the tests are failing.

The code itself LGTM.

[aanand]

This is now ready to merge, other than the fact that it still depends on docker/docker-py#729 and moby/moby#16645.

It skips the networking-related tests unless running against a 1.9.x daemon - once there's a 1.9.0 RC out, thanks to @dnephin's work in #2069 they should automatically start running.

[albers]

I will add this to bash completion once it is merged.

[dnephin]

Merging as part of #2191