Adds prototype NG driver apis

[BrianBland]

Includes an IPC system for out-of-process drivers so users won't have to recompile the registry to bring their own drivers. Also, drivers can potentially be written in any language with a functioning libchan implementation.

[dmp42]

Relevant: #616 #626

Ping @wking @bacongobbler @noxiouz

[wking]

On Fri, Oct 10, 2014 at 04:41:18PM -0700, Brian Bland wrote:

Includes an IPC system for out-of-process drivers so users won't
have to recompile the registry to bring their own drivers. Also,
drivers can potentially be written in any language with a
functioning libchan implementation.

I'm not up on Go / libchan, so I'm not sure how this is going to scale
with many connecting clients. Do you need a separate coroutine for
each client? If so, it seems like it would be more efficient to have
a single thread using select() or similar event-based processing (but
maybe Go's coroutines have absurdly small memory footprints?).

I'm also missing my independent configs for atomic and streaming
storage 1 and storage-generated checksum names to keep us honest on
the content-addressable side (2 and moby/moby#8093), but if this
is just a proof-of-concept, “look it works in Go”, then maybe it's too
early to be talking about that.

On the content-addressable side, it might be easier to have middleware
that translated between the content-addressable registry-side API and
a key/value storage side API (hashing on the fly and renaming from a
temporary filename after it finished writing) to make it easier to
write storage drivers for key/value backends.

[noxiouz]

I suppose, receiver.Receive supports unpacking into an anonymous struct. It should help to get rid of type conversions.
It looks like:

var response struct {
    Contents []byte
    Error string
}
err = receiver.Receive(&response)
if err != nil {
    return nil, err
}

if p.Error != "" {
   err = errors.New(p.Error)
// err = fmt.Errorf("blabla error %s", p.Error)
}
return p.Contents, err

[BrianBland]

Good call, I could create a request and response object for each method to do the most compile-time checking.

[noxiouz]

Do you need a separate coroutine for each client? If so, it seems like it would be more efficient to have a single thread using select() or similar event-based processing (but maybe Go's coroutines have absurdly small memory footprints?).

It's a common practice in golang. Each goroutine needs ~4KB. Of course, golang has epoll/kqueue under the hood, but we're not allowed to use it directly and have no need, as golang schedules our goroutines.

[BrianBland]

On the content-addressable side, it might be easier to have middleware
that translated between the content-addressable registry-side API and
a key/value storage side API (hashing on the fly and renaming from a
temporary filename after it finished writing) to make it easier to
write storage drivers for key/value backends.

This could be especially valuable in the case of multiple registries pointing at the same storage backend for HA purposes. We can write the layer to layer/<tarsum>.tmp-<some random key> and then only move it to layer/<tarsum> if the sum matches our expectation.

Update: it looks like "move" operations in many (most?) storage systems aside from local filesystems involve full copies followed by deletion of the original file, so this might be a huge performance hit.

[BrianBland]

I'm also missing my independent configs for atomic and streaming storage
... but if this is just a proof-of-concept, “look it works in Go”,
then maybe it's too early to be talking about that.

I'm not yet committed to either approach, but if we decide to make these two interfaces, the IPC client Start() method can ask the driver what interface it is implementing and then return the appropriate wrapper.

[dmp42]

That should disappear entirely. Authorization is not up to the registry to decide, nor the driver.

[dmcgowan]

Instead of creating a new send channel for each request, a single send channel can be created for sending all requests. A new goroutine can still be spawned for each decoded request. Since all channels in libchan are nested, the return channels are still unique even though they were sent through the same channel. This will save a round trip to establish the new channel, but that is likely not as important in the unix socket case.

[bacongobbler]

Driver seems too generic. How about FileStore?

[BrianBland]

Driver is probably needlessly generic. I'm open to naming suggestions.

[noxiouz]

Do we really need panic or it's ok to return error

[noxiouz]

@BrianBland I sent your PR into next-generation branch. Should I have sent PR here instead of your fork?

[BrianBland]

Thanks @noxiouz
Because the next-generation branch is a new orphan branch, I'm reconstructing this PR as a completely new commit with your changes incorporated.