Skip to content

Latest commit

 

History

History
252 lines (177 loc) · 10 KB

Thrift_Hyperbahn.md

File metadata and controls

252 lines (177 loc) · 10 KB
Raw

Set up a Go + Thrift + Hyperbahn Service

The code matching this guide is here.

The TChannel+Thrift integration for Go uses code generated by thrift-gen.

Dependencies

Make sure your GOPATH is set up before following this guide.

You'll need to go get the following:

  • github.com/uber/tchannel-go
  • github.com/uber/tchannel-go/hyperbahn
  • github.com/uber/tchannel-go/thrift
  • github.com/uber/tchannel-go/thrift/thrift-gen

Use Godep to manage dependencies, as the API is still in development and will change.

This example will assume that the service is created in the following directory: $GOPATH/src/github.com/uber/tchannel-go/examples/keyvalue

You should use your own path and update your import paths accordingly.

Thrift service

Create a Thrift file to define your service. For this guide, we'll use:

keyvalue.thrift:

service baseService {
  string HealthCheck()
}

exception KeyNotFound {
  1: string key
}

exception InvalidKey {}

service KeyValue extends baseService {
  // If the key does not start with a letter, InvalidKey is returned.
  // If the key does not exist, KeyNotFound is returned.
  string Get(1: string key) throws (
    1: KeyNotFound notFound
    2: InvalidKey invalidKey)

  // Set returns InvalidKey is an invalid key is sent.
  void Set(1: string key, 2: string value)
}

// Returned when the user is not authorized for the Admin service.
exception NotAuthorized {}

service Admin extends baseService {
  void clearAll() throws (1: NotAuthorized notAuthorized)
}

This Thrift specification defines two services:

  • KeyValue: A simple string key-value store.
  • Admin: Management for the key-value store.

Both of these services inherit baseService and so inherit HealthCheck.

The methods may return exceptions instead of the expected result, which are also defined in the specification.

Once you have defined your service, you should generate the Thrift service and client libraries by running the following:

cd $GOPATH/src/github.com/uber/tchannel-go/examples/keyvalue
thrift-gen --generateThrift --inputFile keyvalue.thrift

This runs the Thrift compiler, and then generates the service and client bindings. You can run the commands manually as well:

# Generate serialization/deserialization logic.
thrift -r --gen go:thrift_import=github.com/uber/tchannel-go/thirdparty/github.com/apache/thrift/lib/go/thrift keyvalue.thrift 

# Generate TChannel service interfaces in the same directory where Thrift generates code.
thrift-gen --inputFile "$THRIFTFILE" --outputFile "THRIFT_FILE_FOLDER/gen-go/thriftName/tchan-keyvalue.go"

Go server

To get the server ready, the following needs to be done:

  1. Create the TChannel which is the network layer protocol.
  2. Create a handler to handle the methods defined in the Thrift definition, and register it with tchannel/thrift.
  3. Create a Hyperbahn client and advertise your service with Hyperbahn.

Create a TChannel

Create a channel using tchannel.NewChannel and listen using Channel.ListenAndServe.

The address passed to Listen should be a remote IP that can be used for incoming connections from other machines. You can use tchannel.ListenIP which uses heuristics to determine a good remote IP.

When creating a channel, you can pass additional options.

Create and register Thrift handler

Create a custom type with methods required by the Thrift generated interface. You can examine this interface by looking in gen-go/keyvalue/tchan-keyvalue.go. For example, the interface for our definition file looks like:

type TChanAdmin interface {
	HealthCheck(ctx thrift.Context) (string, error)
	ClearAll(ctx thrift.Context) error
}

type TChanKeyValue interface {
	Get(ctx thrift.Context, key string) (string, error)
	HealthCheck(ctx thrift.Context) (string, error)
	Set(ctx thrift.Context, key string, value string) error
}

Create an instance of your handler type, and then create a thrift.Server and register your Thrift handler. You can register multiple Thrift services on the same thrift.Server.

Each handler method is run in a new goroutine and so must be thread-safe. Your handler methods can return two types of errors:

  • Errors declared in the Thrift file (e.g. KeyNotFound).
  • Unexpected errors.

If you return an unexpected error, an error frame is sent over Thrift with the message. If there are known error cases, it is better to declare them in the Thrift file and return those explicitly, e.g.:

  if value, ok := map[key]; ok {
    return value, ""
  }
  // Return a Thrift exception if the key is not found.
  return "", &keyvalue.KeyNotFound{Key: key}

Advertise with Hyperbahn

Create a Hyperbahn client using hyperbahn.NewClient which requires a Hyperbahn configuration object that should be loaded from a configuration file for the current environment. You can also pass more options when creating the client.

Call Advertise to advertise the service with Hyperbahn.

Serving

Your service is now serving over Hyperbahn! You can test this by making a call using tcurl:

node tcurl.js -p [HYPERBAHN-HOSTPORT] -t [DIR-TO-THRIFT] keyvalue KeyValue::Set -3 '{"key": "hello", "value": "world"}'
node tcurl.js -p [HYPERBAHN-HOSTPORT] -t [DIR-TO-THRIFT] keyvalue KeyValue::Get -3 '{"key": "hello"}'

Replace [HYPERBAHN-HOSTPORT] with the host:port of a Hyperbahn node, and [DIR-TO-THRIFT] with the directory where the .thrift file is stored.

Your service can now be accessed from any language over Hyperbahn + TChannel!

Go client

Note: The client implementation is still in active development.

To make a client that talks, you need to:

  1. Create a TChannel (or re-use an existing TChannel)
  2. Set up Hyperbahn
  3. Create a Thrift+TChannel client.
  4. Make remote calls using the Thrift client.

Create a TChannel

TChannels are bi-directional and so the client uses the same method as the server code (tchannel.NewChannel) to create a TChannel. You do not need to call ListenAndServe on the channel. Even though the channel does not host a service, a serviceName is required for TChannel. This serviceName should be unique to identify this client.

You can use an existing TChannel which hosts a service to make client calls.

Set up Hyperbahn

Similar to the server code, create a new Hyperbahn client using hyperbahn.NewClient. You do not need to call Advertise, as the client does not have any services to advertise over Hyperbahn.

If you have already set up an existing client for use with a server, then you do not need to do anything further.

Create a Thrift client

The Thrift client has two parts:

  1. The thrift.TChanClient which is configured to hit a specific Hyperbahn service.
  2. A generated client which uses an underlying thrift.TChanClient to call methods for a specific Thrift service.

To create a thrift.TChanClient, use thrift.NewClient. This client can then be used to create a generated client:

thriftClient := thrift.NewClient(ch, "keyvalue", nil)
client := keyvalue.NewTChanKeyValueClient(thriftClient)
adminClient := keyvalue.NewTChanAdminClient(thriftClient)

Make remote calls

Method calls on the client make remote calls over TChannel. E.g.

err := client.Set(ctx, "hello", "world")
val, err := client.Get(ctx, "hello")
// val = "world"

You must pass a context when making method calls which passes the deadline, tracing information, and application headers. A simple root context is:

ctx, cancel := thrift.NewContext(time.Second)

All calls over TChannel are required to have a timeout, and tracing information. NewContext should only be used by edges, all other nodes should pass through the incoming Context. When you pass through a Context, you pass along the deadline, tracing information, and the headers.

Note: Trace spans are automatically generated by TChannel, and the parent is set automatically from the current context's tracing span.

Headers

Thrift + TChannel allows clients to send headers (a list of string key/value pairs) and servers can add response headers to any response.

In Go, headers are attached to a context before a call is made using WithHeaders:

headers := map[string]string{"user": "prashant"}

ctx, cancel := thrift.NewContext(time.Second)
ctx = thrift.WithHeaders(ctx)

The server can read these headers using Headers and can set additional response headers using SetResponseHeaders:

func (h *kvHandler) ClearAll(ctx thrift.Context) {
  headers := ctx.Headers()
  // Application logic
  respHeaders := map[string]string{
    "count": 10,
  }
  ctx.SetResponseHeaders(respHeaders)
}

The client can read the response headers by calling ctx.ResponseHeaders() on the same context that was passed when making the call:

ctx := thrift.WithHeaders(thrift.NewContext(time.Second), headers)
err := adminClient.ClearAll()
// check error
responseHeaders := ctx.ResponseHeaders()

Headers should not be used to pass arguments to the method - the Thrift request/response structs should be used for this.

Limitations & Upcoming Changes

TChannel's peer selection does not yet have a detailed health model for nodes, and selection does not balance load across nodes.

The thrift-gen autogenerated code is new, and may not support all Thrift features (E.g. annotations, includes, multiple files)