Skip to content
This repository has been archived by the owner on Sep 9, 2020. It is now read-only.

Commit

Permalink
docs: Fix up README, polish deduction and intro
Browse files Browse the repository at this point in the history
  • Loading branch information
@sdboyer
sdboyer committed Jan 17, 2018
1 parent afb1cd6 commit 7ae1fda
Show file tree
Hide file tree
Showing 4 changed files with 46 additions and 271 deletions.
236 changes: 4 additions & 232 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,38 +7,15 @@

## Dep

`dep` is a prototype dependency management tool for Go. It requires Go 1.8 or newer to compile.
`dep` is a prototype dependency management tool for Go. It requires Go 1.8 or newer to compile. **`dep` is safe for production use.**

`dep` is the official _experiment_, but not yet the official tool. Check out the [Roadmap](https://github.com/golang/dep/wiki/Roadmap) for more on what this means!

## Current status
For guides and reference materials about `dep`, see [the documentation](https://golang.github.io/dep).

`dep` is safe for production use. That means two things:
## Installation

* Any valid metadata file (`Gopkg.toml` and `Gopkg.lock`) will be readable and considered valid by any future version of `dep`.
* The CLI UI is mostly stable. `dep init` and `dep ensure` are mostly set; `dep status` is likely to change a fair bit, and `dep prune` is [going to be absorbed into `dep ensure`](https://github.com/golang/dep/issues/944).

That said, keep in mind the following:

* `dep init` on an existing project can be a rocky experience - we try to automatically convert from other tools' metadata files, and that process is often complex and murky. Once your project is converted and you're using `dep ensure`, its behavior is quite stable.
* `dep` still has nasty bugs, but in general these are comparable to or fewer than other tools out there.
* `dep` is [pretty slow right now](https://github.com/golang/dep/blob/master/docs/FAQ.md#why-is-dep-slow), especially on the first couple times you run it. Just know that there is a _lot_ of headroom for improvement, and work is actively underway.
* `dep` is still changing rapidly. If you need stability (e.g. for CI), it's best to rely on a released version, not tip.
* `dep`'s exported API interface will continue to change in unpredictable, backwards-incompatible ways until we tag a v1.0.0 release.

## Context

- [The Saga of Go Dependency Management](https://blog.gopheracademy.com/advent-2016/saga-go-dependency-management/)
- Official Google Docs
- [Go Packaging Proposal Process](https://docs.google.com/document/d/18tNd8r5DV0yluCR7tPvkMTsWD_lYcRO7NhpNSDymRr8/edit)
- [User Stories](https://docs.google.com/document/d/1wT8e8wBHMrSRHY4UF_60GCgyWGqvYye4THvaDARPySs/edit)
- [Features](https://docs.google.com/document/d/1JNP6DgSK-c6KqveIhQk-n_HAw3hsZkL-okoleM43NgA/edit)
- [Design Space](https://docs.google.com/document/d/1TpQlQYovCoX9FkpgsoxzdvZplghudHAiQOame30A-v8/edit)
- [Frequently Asked Questions](docs/FAQ.md)

## Setup

Grab the latest binary from the [releases](https://github.com/golang/dep/releases) page.
Unless you are hacking on dep, it is strongly recommended that you use a released version. You can grab the latest binary from the [releases](https://github.com/golang/dep/releases) page.

On macOS you can install or upgrade to the latest released version with Homebrew:

Expand All @@ -53,217 +30,12 @@ If you're interested in hacking on `dep`, you can install via `go get`:
go get -u github.com/golang/dep/cmd/dep
```

To start managing dependencies using dep, run the following from your project's root directory:

```sh
$ dep init
```

This does the following:

1. Look for [existing dependency management files](docs/FAQ.md#what-external-tools-are-supported) to convert
1. Check if your dependencies use dep
1. Identify your dependencies
1. Back up your existing `vendor/` directory (if you have one) to
`_vendor-TIMESTAMP/`
1. Pick the highest compatible version for each dependency
1. Generate [`Gopkg.toml`](docs/Gopkg.toml.md) ("manifest") and `Gopkg.lock` files
1. Install the dependencies in `vendor/`

## Usage

There is one main subcommand you will use: `dep ensure`. `ensure` first checks that `Gopkg.lock` is consistent with `Gopkg.toml` and the `import`s in your code. If any
changes are detected, `dep`'s solver works out a new `Gopkg.lock`. Then, `dep` checks if the contents of `vendor/` are what `Gopkg.lock` (the new one if applicable, else the existing one) says it should be, and rewrites `vendor/` as needed to bring it into line.

In essence, `dep ensure` [works in two phases to keep four buckets of state in sync](https://youtu.be/5LtMb090AZI?t=20m4s):

<img width="463" alt="states-flow" src="https://user-images.githubusercontent.com/21599/29223886-22dd2578-7e96-11e7-8b51-3637b9ddc715.png">


_Note: until we ship [vendor verification](https://github.com/golang/dep/issues/121), we can't efficiently perform the `Gopkg.lock` <-> `vendor/` comparison, so `dep ensure` unconditionally regenerates all of `vendor/` to be safe._

`dep ensure` is safe to run early and often. See the help text for more detailed
usage instructions.

```sh
$ dep help ensure
```

### Installing dependencies

(if your `vendor/` directory isn't [checked in with your code](docs/FAQ.md#should-i-commit-my-vendor-directory))

<!-- may change with https://github.com/golang/dep/pull/489 -->

```sh
$ dep ensure
```

If a dependency already exists in your `vendor/` folder, dep will ensure it
matches the constraints from the manifest. If the dependency is missing from
`vendor/`, the latest version allowed by your manifest will be installed.

### Adding a dependency

```sh
$ dep ensure -add github.com/foo/bar
```

This adds a version constraint to your `Gopkg.toml`, and updates `Gopkg.lock` and `vendor/`. Now, import and use the package in your code! ✨

`dep ensure -add` has some subtle behavior variations depending on the project or package named, and the state of your tree. See `dep ensure -examples` for more information.

### Changing dependencies

If you want to:

* Change the allowed `version`/`branch`/`revision`
* Switch to using a fork

for one or more dependencies, do the following:

1. Manually edit your `Gopkg.toml`.
1. Run

```sh
$ dep ensure
```

### Checking the status of dependencies

Run `dep status` to see the current status of all your dependencies.

```sh
$ dep status
PROJECT CONSTRAINT VERSION REVISION LATEST
github.com/Masterminds/semver branch 2.x branch 2.x 139cc09 c2e7f6c
github.com/Masterminds/vcs ^1.11.0 v1.11.1 3084677 3084677
github.com/armon/go-radix * branch master 4239b77 4239b77
```

On top of that, if you have added new imports to your project or modified `Gopkg.toml` without running `dep ensure` again, `dep status` will tell you there is a mismatch between `Gopkg.lock` and the current status of the project.

```sh
$ dep status
Lock inputs-digest mismatch due to the following packages missing from the lock:
PROJECT MISSING PACKAGES
github.com/Masterminds/goutils [github.com/Masterminds/goutils]
This happens when a new import is added. Run `dep ensure` to install the missing packages.
```

As `dep status` suggests, run `dep ensure` to update your lockfile. Then run `dep status` again, and the lock mismatch should go away.

### Visualizing dependencies

Generate a visual representation of the dependency tree by piping the output of `dep status -dot` to [graphviz](http://www.graphviz.org/).
#### Linux
```
$ sudo apt-get install graphviz
$ dep status -dot | dot -T png | display
```
#### MacOS
```
$ brew install graphviz
$ dep status -dot | dot -T png | open -f -a /Applications/Preview.app
```
#### Windows
```
> choco install graphviz.portable
> dep status -dot | dot -T png -o status.png; start status.png
```
<p align="center"><img src="docs/img/StatusGraph.png"></p>

### Updating dependencies

Updating brings the version of a dependency in `Gopkg.lock` and `vendor/` to the latest version allowed by the constraints in `Gopkg.toml`.

You can update just a targeted subset of dependencies (recommended):

```sh
$ dep ensure -update github.com/some/project github.com/other/project
$ dep ensure -update github.com/another/project
```

Or you can update all your dependencies at once:

```sh
$ dep ensure -update
```

"Latest" means different things depending on the type of constraint in use. If you're depending on a `branch`, `dep` will update to the latest tip of that branch. If you're depending on a `version` using [a semver range](#semantic-versioning), it will update to the latest version in that range.

### Removing dependencies

1. Remove the `import`s and all usage from your code.
1. Remove `[[constraint]]` rules from `Gopkg.toml` (if any).
1. Run

```sh
$ dep ensure
```

### Testing changes to a dependency

Making changes in your `vendor/` directory directly is not recommended, as dep
will overwrite any changes. Instead:

1. Delete the dependency from the `vendor/` directory.

```sh
rm -rf vendor/<dependency>
```

1. Add that dependency to your `GOPATH`, if it isn't already.
```sh
$ go get <dependency>
```
1. Modify the dependency in `$GOPATH/src/<dependency>`.
1. Test, build, etc.
Don't run `dep ensure` until you're done. `dep ensure` will reinstall the
dependency into `vendor/` based on your manifest, as if you were installing from
scratch.
This solution works for short-term use, but for something long-term, take a look
at [virtualgo](https://github.com/GetStream/vg).
To test out code that has been pushed as a new version, or to a branch or fork,
see [changing dependencies](#changing-dependencies).
## Semantic Versioning
`dep ensure` uses an external [semver library](https://github.com/Masterminds/semver) to interpret the version constraints you specify in the manifest. The comparison operators are:
* `=`: equal
* `!=`: not equal
* `>`: greater than
* `<`: less than
* `>=`: greater than or equal to
* `<=`: less than or equal to
* `-`: literal range. Eg: 1.2 - 1.4.5 is equivalent to >= 1.2, <= 1.4.5
* `~`: minor range. Eg: ~1.2.3 is equivalent to >= 1.2.3, < 1.3.0
* `^`: major range. Eg: ^1.2.3 is equivalent to >= 1.2.3, < 2.0.0
* `[xX*]`: wildcard. Eg: 1.2.x is equivalent to >= 1.2.0, < 1.3.0
You might, for example, include a constraint in your manifest that specifies `version = "=2.0.0"` to pin a dependency to version 2.0.0, or constrain to minor releases with: `version = "2.*"`. Refer to the [semver library](https://github.com/Masterminds/semver) documentation for more info.
**Note**: When you specify a version *without an operator*, `dep` automatically uses the `^` operator by default. `dep ensure` will interpret the given version as the min-boundary of a range, for example:
* `1.2.3` becomes the range `>=1.2.3, <2.0.0`
* `0.2.3` becomes the range `>=0.2.3, <0.3.0`
* `0.0.3` becomes the range `>=0.0.3, <0.1.0`
## Feedback

Feedback is greatly appreciated.
At this stage, the maintainers are most interested in feedback centered on the user experience (UX) of the tool.
Do you have workflows that the tool supports well, or doesn't support at all?
Do any of the commands have surprising effects, output, or results?
Please check the existing issues and [FAQ](docs/FAQ.md) to see if your feedback has already been reported.
If not, please file an issue, describing what you did or wanted to do, what you expected to happen, and what actually happened.

## Contributing
Expand Down
41 changes: 32 additions & 9 deletions docs/Gopkg.toml.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -120,21 +120,34 @@ system2-data = "value that is used by another system"



----
## Version rules

Version rules can be used in either `[[constraint]]` or `[[override]]` stanzas. There are three types of version rules - `version`, `branch`, and `revision`. At most one of the three types can be specified.

### `version`

## Version rules
`version` is a property of `constraint`s and `override`s. It is used to specify version constraint of a specific dependency. It can be used to target an arbitrary VCS tag, or a semantic version, or a range of semantic versions.

Version rules can be used in either `[[constraint]]` or `[[override]]` stanzas. There are three types of version rules - `version`, `branch`, and `revision`. At most one of the three types can be specified.
Specifying semantic version ranges can be done using the following operators:

TODOOOOOOOOOOOOO
* `=`: equal
* `!=`: not equal
* `>`: greater than
* `<`: less than
* `>=`: greater than or equal to
* `<=`: less than or equal to
* `-`: literal range. Eg: 1.2 - 1.4.5 is equivalent to >= 1.2, <= 1.4.5
* `~`: minor range. Eg: ~1.2.3 is equivalent to >= 1.2.3, < 1.3.0
* `^`: major range. Eg: ^1.2.3 is equivalent to >= 1.2.3, < 2.0.0
* `[xX*]`: wildcard. Eg: 1.2.x is equivalent to >= 1.2.0, < 1.3.0

### `version`
You might, for example, include a rule that specifies `version = "=2.0.0"` to pin a dependency to version 2.0.0, or constrain to minor releases with: `version = "~2.1.0"`. Refer to the [semver library](https://github.com/Masterminds/semver) documentation for more info.

`version` is a property of `constraint`s and `override`s. It is used to specify version constraint of a specific dependency.
**Note**: When you specify a version *without an operator*, `dep` automatically uses the `^` operator by default. `dep ensure` will interpret the given version as the min-boundary of a range, for example:

Internally, dep uses [Masterminds/semver](https://github.com/Masterminds/semver) to work with semver versioning.
* `1.2.3` becomes the range `>=1.2.3, <2.0.0`
* `0.2.3` becomes the range `>=0.2.3, <0.3.0`
* `0.0.3` becomes the range `>=0.0.3, <0.1.0`

`~` and `=` operators can be used with the versions. When a version is specified without any operator, `dep` automatically adds a caret operator, `^`. The caret operator pins the left-most non-zero digit in the version. For example:
```
Expand All @@ -150,11 +163,21 @@ To pin a version of direct dependency in manifest, prefix the version with `=`.
version = "=0.8.0"
```

[Why is dep ignoring a version constraint in the manifest?](FAQ.md#why-is-dep-ignoring-a-version-constraint-in-the-manifest)
### `branch`

Using a `branch` constraint will cause dep to use the named branch (e.g., `branch = "master"`) for a particular dependency. The revision at the tip of the branch will be recorded into `Gopkg.lock`, and almost always remain the same until a change is requested, via `dep ensure -update`.

In general, you should prefer semantic versions to branches, when a project has made them available.

### `revision`

A `revision` is the underlying immutable identifier - like a git commit SHA1. While it is allowed to constrain to a `revision`, doing so is almost always an antipattern.

Usually, folks are inclined to pin to a revision because they feel it will somehow improve their project's reproducibility. That is not a good reason. `Gopkg.lock` provides reproducibility. Only use `revision` if you have a good reason to believe that _no_ other version of that dependency _could_ work.

# Example

Here's an example of a sample Gopkg.toml with most of the elements
A sample `Gopkg.toml` with most elements present:

```toml
required = ["github.com/user/thing/cmd/thing"]
Expand Down
35 changes: 9 additions & 26 deletions docs/deduction.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,37 +7,20 @@ Deduction is dep's algorithm for looking at an import path and determining the p
- `github.com/golang/dep/gps` -> `github.com/golang/dep`
- `bitbucket.org/foo/bar/baz` -> `bitbucket.org/foo/bar`

The set of hosts supported by static deduction are the same as [those supported by `go get`]().
The set of hosts supported by static deduction are the same as [those supported by `go get`](https://golang.org/cmd/go/#hdr-Remote_import_paths):

If the static logic cannot identify the root for a given import path, the algorithm continues to a dynamic component, where it makes an HTTP(S) request to the import path, and a server is expected to return a response that indicates the root import path. This, again, is equivalent to the behavior of `go get`.
* GitHub
* Bitbucket
* Launchpad
* IBM DevOps Services

In addition, dep also handles [gopkg.in](http://gopkg.in) directly with static deduction because, owing to internal implementation details, it is the easiest way of also attaching filters to adapt the versioning semantics of gopkg.in import paths into dep's versioning model. This turns out fine, as gopkg.in's rules mapping rules are themselves entirely static.

If the static logic cannot identify the root for a given import path, the algorithm continues to a dynamic component: dep makes an HTTP(S) request to the import path, and a server is expected to send back the root import path embedded within the HTML response. Again, this directly emulates the behavior of `go get`.

Import path deduction is applied to all of the following:

* `import` statements found in all `.go` files
* Import paths named in the [`required`](gopkg.toml.md#required) property in `Gopkg.toml`
* `name` properties in both [`[[constraint]]`](Gopkg.toml.md#constraint) and [`[[override]]`](Gopkg.toml.md#override) stanzas in `Gopkg.toml`. This is solely for validation purposes, enforcing that these names correspond strictly to source roots.
* Import paths in the [`required`](gopkg.toml.md#required) list in `Gopkg.toml`
* `name` properties in both [`[[constraint]]`](Gopkg.toml.md#constraint) and [`[[override]]`](Gopkg.toml.md#override) stanzas in `Gopkg.toml`. This is solely for validation purposes, enforcing that these names correspond only to project/source roots.




The results of import path deduction are, in practice, almost entirely fixed; it's easy to imagine why. In the public ecosystem, even dynamic deductions rarely change in practice, as it would either require:

- a `go-get` metadata service to intentionally change its mappings, or
- a `go-get` metadata service to disappear.

`go get` itself is only partially resilient to these cases, but each is potentially catastrophic for a package's retrievability across the ecosystem, sooner rather than later. This steep and abrupt downside makes it nearly impossible for projects only accessible via an unreliable metadata service to ever become popular or widely used in the ecosystem. Thus, in the public ecosystem, we almost only ever see reliable, well-behaved services.










Because deduction has a dynamic component, the deducibility of any given path necessarily cannot be fixed. However, because the

## Static deduction
5 changes: 1 addition & 4 deletions docs/introduction.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,4 @@ Welcome! This is documentation for dep, the "official experiment" dependency man

This site has both guides and reference documents. The guides are practical explanations of how to actually do things with dep, whereas the reference material provides deeper dives on specific topics. Of particular note is the [glossary](#glossary.md) - if you're unfamiliar with terminology used in this documentation, make sure to check there!



* link to guides
* installing dep
If you're trying dep for the first time, check out [Creating a New Project](new-project.md), or [Migrating to dep](migrating.md) if you have an existing Go project that you want to manage with dep.

0 comments on commit 7ae1fda

Please sign in to comment.