diff --git a/README.md b/README.md index ed5caa0ca..daf208d70 100644 --- a/README.md +++ b/README.md @@ -127,7 +127,7 @@ in the large. Go kit is a set of packages and best practices, leveraging years of production experience, and providing a comprehensive, robust, and trustable platform for organizations of any size. -In short, Go kit brings Go to the modern enterprise. +In short, Go kit makes Go a viable choice for business-domain microservices. For more details, see [the motivating blog post](http://peter.bourgon.org/go-kit/) and @@ -144,44 +144,32 @@ See also the ## Non-goals -- Supporting messaging patterns other than RPC (in the initial release) — MPI, pub/sub, CQRS, etc. -- Re-implementing functionality that can be provided by wrapping existing packages -- Having opinions on deployment, orchestration, process supervision, etc. -- Having opinions on configuration passing — flags, env vars, files, etc. +- Supporting messaging patterns other than RPC (for now) — e.g. MPI, pub/sub, CQRS, etc. +- Re-implementing functionality that can be provided by adapting existing software +- Having opinions on operational concerns: deployment, configuration, process supervision, orchestration, etc. ## Contributing -Please see [CONTRIBUTING.md]. Thank you, [contributors]! +Please see [CONTRIBUTING.md][]. Thank you, [contributors][]! [CONTRIBUTING.md]: /CONTRIBUTING.md [contributors]: https://github.com/go-kit/kit/graphs/contributors ## Dependency management -Go kit is a library, designed to be imported into a binary package. Vendoring -is currently the best way for binary package authors to ensure reliable, -reproducible builds. Therefore, we strongly recommend our users use vendoring -for all of their dependencies, including Go kit. To avoid compatibility and -availability issues, Go kit doesn't vendor its own dependencies, and doesn't -recommend use of third-party import proxies. +Go kit is a library, designed to be imported into a binary package. +Vendoring is currently the best way for binary package authors to ensure reliable, reproducible builds. +Therefore, we strongly recommend our users use vendoring for all of their dependencies, including Go kit. +To avoid compatibility and availability issues, Go kit doesn't vendor its own dependencies, and doesn't recommend use of third-party import proxies. -There are several tools which make vendoring easier, including [gb][], -[govendor][], and [godep][]. And Go kit uses a variety of continuous -integration providers to find and fix compatibility problems as soon as they -occur. +There are several tools which make vendoring easier, including [gb][], [glide][], [gvt][], [govendor][], and [vendetta][]. +In addition, Go kit uses a variety of continuous integration providers to find and fix compatibility problems as soon as they occur. [gb]: http://getgb.io +[glide]: https://github.com/Masterminds/glide +[gvt]: https://github.com/FiloSottile/gvt [govendor]: https://github.com/kardianos/govendor -[godep]: https://github.com/tools/godep - -## API stability policy - -The Go kit project depends on code maintained by others. -This includes the Go standard library and sub-repositories and other external libraries. -The Go language and standard library provide stability guarantees, but the other external libraries typically do not. -[The API Stability RFC](https://github.com/go-kit/kit/tree/master/rfc/rfc007-api-stability.md) - proposes a standard policy for package authors to advertise API stability. -The Go kit project prefers to depend on code that abides the API stability policy. +[vendetta]: https://github.com/dpw/vendetta ## Related projects @@ -189,6 +177,7 @@ Projects with a ★ have had particular influence on Go kit's design (or vice-ve ### Service frameworks +- [gizmo](https://github.com/nytimes/gizmo), a microservice toolkit from The New York Times ★ - [go-micro](https://github.com/myodc/go-micro), a microservices client/server library ★ - [gocircuit](https://github.com/gocircuit/circuit), dynamic cloud orchestration - [gotalk](https://github.com/rsms/gotalk), async peer communication protocol & library @@ -230,3 +219,4 @@ Projects with a ★ have had particular influence on Go kit's design (or vice-ve - [Architecting for the Cloud](http://fr.slideshare.net/stonse/architecting-for-the-cloud-using-netflixoss-codemash-workshop-29852233) — Netflix - [Dapper, a Large-Scale Distributed Systems Tracing Infrastructure](http://research.google.com/pubs/pub36356.html) — Google - [Your Server as a Function](http://monkey.org/~marius/funsrv.pdf) (PDF) — Twitter + diff --git a/rfc/README.md b/rfc/README.md deleted file mode 100644 index 87f3a67d9..000000000 --- a/rfc/README.md +++ /dev/null @@ -1,23 +0,0 @@ -# RFCs - -This is a place to hold proposals and design discussions for components of the -Go kit. - -Let's keep the process lightweight. To propose a new RFC, make a pull request -with a new file, roughly in the same format as the others. To make changes to -an existing RFC, make a pull request with your changes, and describe your -motivation in the comments. - -RFCs follow the [Chef RFC workflow](https://github.com/chef/chef-rfc/blob/master/rfc000-rfc-process.md#rfc-review-and-workflow). - -![RFC workflow](http://i.imgur.com/qZvqG2g.png "RFC workflow") - -At this stage of the project, RFC ratification will be done by me (Peter -Bourgon). Once we reach a usable MVP, i.e. semantic version 1.0.0, I hope we'll -transition to a more community-driven governance model. - -For questions and free-form discussion, please use - -- Mailing list: [go-kit](https://groups.google.com/forum/#!forum/go-kit) -- Slack: [gophers.slack.com](https://gophers.slack.com) **#go-kit** ([invite](https://gophersinvite.herokuapp.com/)) - diff --git a/rfc/rfc000-package-server.md b/rfc/rfc000-package-server.md deleted file mode 100644 index 3b049ac10..000000000 --- a/rfc/rfc000-package-server.md +++ /dev/null @@ -1,83 +0,0 @@ ---- -RFC: 000 -Author: Matt Heath -Status: Accepted ---- - -# Package Server - -## Motivation - -[http://peter.bourgon.org/go-kit/#package-server](http://peter.bourgon.org/go-kit/#package-server) - -Package server is probably the biggest and most important component of the toolkit. Ideally, we should be able to write our services as implementations of normal, nominal Go interfaces, and delegate integration with the environment to the server package. The package should encode and enforce conventions for server-side concerns, like health checks, system-wide request tracing, connection management, backpressure and throttling, and so on. For each of those topics, it should provide interfaces for different, pluggable strategies. It should integrate with service discovery, and work equally well over multiple transports. Considerable prior art exists in the form of Finagle, Karyon (Netflix's application service library), and likely many more. - -## Scope - -Sections with a ★ are considered particularly volatile, and may change significantly in the future. - -### Endpoints - -* An endpoint is defined as a handler interface which receives a request and returns a response or an error to the client. -* A server SHALL have one or more endpoints. -* An endpoint SHALL accept a request context and propagate this - allowing access to request-specific values such as the identity of the end user, authorization tokens, and the request's deadline. -* An endpoint SHALL respect request cancellation if or when the request's deadline expires. -* A server MAY expose information about its endpoints, to allow integration with additional tools. - -### Contexts - -* Requests SHALL be executed within a request [context](https://blog.golang.org/context), which the server will pass through the request chain. - -### Throttling & Backpressure - -* A server MAY throttle inbound requests and reject requests from clients based on a number of factors. -* A server MAY respond with either an Out Of Capacity error, or a Rate Limit Exceeded error when rejecting requests. -* A server MAY limit the total number of concurrent requests it can serve. -* A server MAY impose rate limits on specific clients. -* Rate limit behaviour MAY range from minimum request intervals, to time based, or leaky bucket algorithms. -* A server MAY implement a pluggable throttle interface, allowing richer implementations - such as an implementation which shares information across instances of the service. - -### SLAs & SLIs - -* A server MAY report its contractual SLA per endpoint to a discovery system, allowing clients to estimate response time. -* A server MAY expose its actual SLI per endpoint, allowing third-parties to reason about healthiness. - -### Healthchecks ★ - -* A server SHALL accept registration of healthchecks with a defined interface. -* A server MAY register default healthchecks to report the health of built in components of the server. -* A server SHALL register an endpoint which can be queried to obtain healthcheck information and status. -* A server MAY publish these healthcheck statuses via a pluggable transport. - -### Service Discovery - -* A server SHALL register itself with a service discovery mechanism on startup. -* A server SHALL attempt to deregister itself with a service discovery mechanism on shutdown. -* The discovery mechanism SHALL be interchangeable, and satisfy a defined interface, however the mechanism itself is beyond the scope of this RFC. - -### Request Tracing - -* Requests received by the server which are accompanied with tracing information SHALL respect this information and pass this information onto other sub-requests initiated by clients within the server. -* The request tracing mechanism SHALL be interchangeable, and satisfy a defined interface, however the mechanism is beyond the scope of this RFC. - -### Transport - -* A server SHALL receive and respond to requests via a Transport. -* The Transport mechanism SHALL be interchangeable, and satisfy a defined interface, however the mechanism of the transport is beyond the scope of this RFC. - -### Codec ★ - -* A server SHALL encode and decode requests and responses via an interchangeable Codec. -* A server MAY support multiple encodings, and use the appropriate Codec as indicated by the transport. -* A server MAY indicate to the transport the encoding used, allowing clients to easily decode the response. - -## Implementation - -To be defined. - -## Further Reading - -* [Your Server as a Function](http://monkey.org/~marius/funsrv.pdf) - Marius Eriksen -* [Finagle](https://twitter.github.io/finagle/) - Twitter -* [Karyon](https://github.com/Netflix/karyon) - Netflix -* [State of the Art in Microservices](https://www.slideshare.net/adriancockcroft/dockercon-state-of-the-art-in-microservices) - Adrian Cockcroft diff --git a/rfc/rfc001-package-client.md b/rfc/rfc001-package-client.md deleted file mode 100644 index ff93435c9..000000000 --- a/rfc/rfc001-package-client.md +++ /dev/null @@ -1,19 +0,0 @@ ---- -RFC: 001 -Author: Peter Bourgon -Status: Draft ---- - -# package client - -## Motivation - -http://peter.bourgon.org/go-kit/#package-client - -## Scope - -To be defined. - -## Implementation - -To be defined. \ No newline at end of file diff --git a/rfc/rfc002-package-transport.md b/rfc/rfc002-package-transport.md deleted file mode 100644 index 5b6624fa2..000000000 --- a/rfc/rfc002-package-transport.md +++ /dev/null @@ -1,19 +0,0 @@ ---- -RFC: 002 -Author: Peter Bourgon -Status: Draft ---- - -# package transport - -## Motivation - -http://peter.bourgon.org/go-kit/#package-transport - -## Scope - -To be defined. - -## Implementation - -To be defined. \ No newline at end of file diff --git a/rfc/rfc003-package-metrics.md b/rfc/rfc003-package-metrics.md deleted file mode 100644 index fa6c37aa9..000000000 --- a/rfc/rfc003-package-metrics.md +++ /dev/null @@ -1,66 +0,0 @@ ---- -RFC: 003 -Author: Peter Bourgon -Status: Draft ---- - -# package metrics - -## Motivation - -http://peter.bourgon.org/go-kit/#package-metrics - -## Scope - -- Package metrics SHALL implement Gauges, Counters, and Histograms. - -- Each metric type SHALL allow observations with an unlimited number of key/value field pairs, - similar to [package log](https://github.com/go-kit/kit/blob/master/rfc/rfc004-package-log.md). - -- Counter SHALL be an increment-only counter of type uint64. - -- Gauge SHALL be an arbitrarily-settable register of type int64. - -- Histogram SHALL collect observations of type int64. - -- These interfaces SHALL be the primary and exclusive API for metrics. - -- We SHALL provide a variety of implementations of each interface that act as a - bridge to different backends: expvar, Graphite, statsd, Prometheus, etc. - -- Each metric backend MAY provide additional value-add behaviors. For example, - a backend for Histogram may bucket observations according to quantile and - calculate additional, derived statistics. - - -## Implementation - -https://github.com/go-kit/kit/tree/master/metrics - -### Gauge - -```go -type Gauge interface { - With(Field) Gauge - Set(value int64) - Add(delta int64) -} -``` - -### Counter - -```go -type Counter interface { - With(Field) Counter - Add(delta uint64) -} -``` - -### Histogram - -```go -type Histogram interface { - With(Field) Histogram - Observe(int64) -} -``` diff --git a/rfc/rfc004-package-log.md b/rfc/rfc004-package-log.md deleted file mode 100644 index 454f812ab..000000000 --- a/rfc/rfc004-package-log.md +++ /dev/null @@ -1,45 +0,0 @@ ---- -RFC: 004 -Authors: Brian Knox -Status: Accepted ---- - -# package log - -## Motivation - -http://peter.bourgon.org/go-kit/#package-log - -## Scope - -### Key / Value Pairs -* Log SHALL use key / value pairs for structured data. -* Log MAY preserve type safety when expressing logs in a structured format that supports types -* Log SHALL provide a way to set a default k/v set per log context -* Log MAY provide some pre-canned default keys (level, time, etc) for convenience -* Log MAY allow multiple stored logging contexts each with a different set of k/v pairs. -* Log MAY allow per log call adhoc k/v pairs (see Logrus as an example) - -### Transport -* Log SHALL be transport agnostic with pluggable transports. -* Log MAY implement io.MultiWriters allowing broadcast of logs over multiple transports. -* Log MAY use channels instead -* Log MAY provide some built in transports such as syslog and logstash. -* Log MAY use encoding.* above the transport level -* Log MAY provide configurable back pressure handling strategies in the case of blocked Writers - -### Formats -* Log SHALL be format agnostic by providing an interface for log formatting. -* Log MAY provide some built in formatters such as RFC3164, JSON, etc. - -### Levels -* Log MAY provide a set of defined severity levels that can be used (perhaps as a wrapper). -* Log SHALL include severity as a k/v pair and allow setting it through the same mechanism as any other k/v pair -* Log MAY provide wrapper types as a convenience for setting the severity level -* Log MUST NOT intrinsically tie severity levels to program actions - e.g., a call to a specific log level should not call a panic. -* Log MAY allow tying program actions such as panic to a log level. - -## Implementation - -* The initial implementation should be a minimal feature set focused only on the scope of the RFC. -* Additional features and niceties may be added as there is demonstrable proof that the features solve real world problems. diff --git a/rfc/rfc005-service-discovery.md b/rfc/rfc005-service-discovery.md deleted file mode 100644 index 342bed96c..000000000 --- a/rfc/rfc005-service-discovery.md +++ /dev/null @@ -1,19 +0,0 @@ ---- -RFC: 005 -Author: Peter Bourgon -Status: Draft ---- - -# Service discovery - -## Motivation - -http://peter.bourgon.org/go-kit/#service-discovery - -## Scope - -To be defined. - -## Implementation - -To be defined. \ No newline at end of file diff --git a/rfc/rfc006-request-tracing.md b/rfc/rfc006-request-tracing.md deleted file mode 100644 index c79bee4d4..000000000 --- a/rfc/rfc006-request-tracing.md +++ /dev/null @@ -1,33 +0,0 @@ ---- -RFC: 006 -Author: Peter Bourgon -Status: Draft ---- - -# Request tracing - -## Motivation - -[Dapper][]-style request tracing is a necessary introspection tool in any large -distributed system. Gokit services should support request tracing, including -exposition of traces that are compatible with [Zipkin][]. - -[Dapper]: http://research.google.com/pubs/pub36356.html -[Zipkin]: http://itszero.github.io/blog/2014/03/03/introduction-to-twitters-zipkin - -## Scope - -- Request tracing SHALL use Dapper terminology: Trace, Span, Tree, etc. - -- Regardless if request tracing is enabled or disabled, if incoming requests - contain trace IDs, package server MUST transparently forward them to - downstream services. - -- If request tracing is enabled, and incoming requests do not contain trace - IDs, package server SHALL generate relevant IDs and forward them to - downstream services. - -## Implementation - -To be defined. - diff --git a/rfc/rfc007-api-stability.md b/rfc/rfc007-api-stability.md deleted file mode 100644 index eeb399ea2..000000000 --- a/rfc/rfc007-api-stability.md +++ /dev/null @@ -1,97 +0,0 @@ ---- -RFC: 007 -Author: Andrew Gerrand -Status: Accepted ---- - -# API Stability Policy - -## Motivation - -The gokit project depends on code maintained by others. -This includes the Go standard library and sub-repositories -and other external libraries. - -The Go language and standard library provide stability guarantees, but the -other external libraries typically do not. This RFC proposes a standard policy -for package authors to advertise API stability. - -The intention is that the gokit project will require that its dependencies -adhere to the policy, with the greater goal of improving the Go ecosystem -as a whole. - -## Scope - -This policy is for package authors to provide their users with a promise of API -stability. -This document is similar to and inspired by the [Go 1 compatibility -promise](https://golang.org/doc/go1compat). - -An author declaring their package "API Stable" makes the following promise: - -> We will not change the package's exported API in backward incompatible ways. -> Future changes to this package will not break dependent code. - -### Coverage - -The promise of stability includes: - -* The package name, -* Exported type declarations and struct fields (names and types), -* Exported function and method names, parameters, and return values, -* Exported constant names and values, -* Exported variable names and values, -* The documented behavior of all exported code. - -### Exceptions - -* Security. A security issue in the package may come to light whose resolution - requires breaking compatibility. We reserve the right to address such - security issues. - -* Unspecified behavior. Programs that depend on unspecified - behavior may break in future releases. - -* Bugs. If the package has a bug, a program that depends on the buggy behavior - may break if the bug is fixed. We reserve the right to fix such bugs. - -* Struct literals. For the addition of features it may be necessary to add - fields to exported structs in the package API. Code that uses unkeyed struct - literals (such as pkg.T{3, "x"}) to create values of these types would fail - to compile after such a change. However, code that uses keyed literals - (pkg.T{A: 3, B: "x"}) will continue to compile after such a change. We will - update such data structures in a way that allows keyed struct literals to - remain compatible, although unkeyed literals may fail to compile. (There are - also more intricate cases involving nested data structures or interfaces, but - they have the same resolution.) We therefore recommend that composite - literals whose type is defined in a separate package should use the keyed - notation. - -* Dot imports. If a program imports a package using import . "path", additional - names later defined in the imported package may conflict with other names - defined in the program. We do not recommend the use of import . outside of - tests, and using it may cause a program to fail to compile in the future. - -### Breaking compatibility - -Should the author wish to break compatibility by redesigning the API the author -should create a new package with a new import path. - -### Awareness - -This text should be present in a file named STABILITY in the repository root. - -### Enforcement - -Tooling may be devised to check the stability of an API as a package evolves, -similar to the api tool used by the Go core. - -The [vet](https://godoc.org/golang.org/x/tools/cmd/vet) tool will already -detect "untagged" struct literals; that is, struct literals that will break -when new fields are added to the struct. - -## Implementation - -To be defined. - -