diff --git a/doc/_resources/templates/document.html b/doc/_resources/templates/document.html index 2b1e3f6cebd8..f8bd6fcafcb0 100644 --- a/doc/_resources/templates/document.html +++ b/doc/_resources/templates/document.html @@ -116,27 +116,7 @@
  • Usage statistics
  • User surveys
  • Quick links
  • - -
  • Campaigns Beta
  • -
  • - -
  • +
  • Campaigns
  • diff --git a/doc/dev/campaigns_design.md b/doc/dev/campaigns_design.md new file mode 100644 index 000000000000..37c0099dc500 --- /dev/null +++ b/doc/dev/campaigns_design.md @@ -0,0 +1,126 @@ +# Campaigns design doc + +Why are [campaigns](../user/campaigns/index.md) designed the way they are? + +## Principles + +- **Declarative API** (not imperative). You declare your intent, such as "lint files in all repositories with a `package.json` file". The campaign figures out how to achieve your desired state. The external state (of repositories, changesets, code hosts, access tokens, etc.) can change at any time, and temporary errors frequently occur when reading and writing to code hosts. These factors would make an imperative API very cumbersome because each API client would need to handle the complexity of the distributed system. +- **Define a campaign in a file** (not some online API). The source of truth of a campaign's definition is a file that can be stored in version control, reviewed in code review, and re-applied by CI. This is in the same spirit as IaaC (infrastructure as code; e.g., storing your Terraform/Kubernetes/etc. files in Git). We prefer this approach over a (worse) alternative where you define a campaign in a UI with a bunch of text fields, checkboxes, buttons, etc., and need to write a custom API client to import/export the campaign definition. +- **Shareable and portable.** You can share your campaign specs, and it's easy for other people to use them. A campaign spec expresses an intent that's high-level enough to (usually) not be specific to your own particular repositories. You declare and inject configuration and secrets to customize it instead of hard-coding those values. +- **Large-scale.** You can run campaigns across 10,000s of repositories. It might take a while to compute and push everything, and the current implementation might cap out lower than that, but the fundamental design scales well. +- **Accommodates a variety of code hosts and review/merge processes.** Specifically, we don't to limit campaigns to only working for GitHub pull requests. (See [current support list](../user/campaigns/index.md#supported-code-hosts-and-changeset-types).) + +## Comparison to other distributed systems + +Kubernetes is a distributed system with an API that many people are familiar with. Campaigns is also a distributed system. All APIs for distributed systems need to handle a similar set of concerns around robustness, consistency, etc. Here's a comparison showing how these concerns are handled for a Kubernetes [Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) and a Sourcegraph campaign. In some cases, we've found Kubernetes to be a good source of inspiration for the campaigns API, but resembling Kubernetes is **not** an explicit goal. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    + Kubernetes DeploymentSourcegraph campaign
    What underlying thing does this API manage?Pods running on many (possibly unreliable) nodesBranches and changesets on many repositories that can be rate-limited and externally modified (and our authorization can change)
    Spec YAML +
    # File: foo.Deployment.yaml
    +apiVersion: apps/v1
    +kind: Deployment
    +metadata:
    +  name: nginx-deployment
    +spec:
    # Evaluate this to enumerate instances of... + replicas: 2
    +
    # ...this template. + template: + metadata: + labels: + app: nginx + spec: + containers: + - name: nginx + image: nginx:1.14.2 + ports: + - containerPort: 80
    +
    +
    # File: hello-world.campaign.yaml
    +name: hello-world
    +description: Add Hello World to READMEs
    +
    +
    # Evaluate this to enumerate instances of... +on: + - repositoriesMatchingQuery: file:README.md + +steps: + - run: echo Hello | tee -a $(find -name '*.md') + container: alpine:3 +
    +
    # ...this template. +changesetTemplate: + title: Hello World + body: My first campaign! + branch: hello-world + commit: + message: Append Hello to .md files + published: false
    +
    How desired state is computed +
      +
    1. Evaluate replicas, etc. (blue) to determine pod count and other template inputs
    2. +
    3. Instantiate template (pink) once for each pod to produce PodSpecs
    4. +
    +
    +
      +
    1. Evaluate on, steps (blue) to determine list of patches
    2. +
    3. Instantiate changesetTemplate (purple) once for each patch to produce ChangesetSpecs +
    4. +
    +
    Desired state consists of... +
      +
    • DeploymentSpec file (the YAML above)
    • +
    • List of PodSpecs (template instantiations)
    • +
    +
    +
      +
    • CampaignSpec file (the YAML above)
    • +
    • List of ChangesetSpecs (template instantiations)
    • +
    +
    Where is the desired state computed?The deployment controller (part of the Kubernetes cluster) consults the DeploymentSpec and continuously computes the desired state. +

    The Sourcegraph CLI (running on your local machine, not on the Sourcegraph server) consults the campaign spec and computes the desired state when you invoke src campaign apply.

    +

    Difference vs. Kubernetes: A campaign's desired state is computed locally, not on the server. It requires executing arbitrary commands, which is not yet supported by the Sourcegraph server. See campaigns known issue "Campaign steps are run locally...".

    +
    Reconciling desired state vs. actual stateThe "deployment controller" reconciles the resulting PodSpecs against the current actual PodSpecs (and does smart things like rolling deploy).The "campaign controller" (i.e., our backend) reconciles the resulting ChangesetSpecs against the current actual changesets (and does smart things like gradual roll-out/publishing and auto-merging when checks pass).
    + +These docs explain more about Kubernetes' design: + +- [Kubernetes Object Management](https://kubernetes.io/docs/concepts/overview/working-with-objects/object-management/) +- [Kubernetes Controllers](https://kubernetes.io/docs/concepts/architecture/controller/) + - [Desired versus current state](https://kubernetes.io/docs/concepts/architecture/controller/#desired-vs-current) +- [Kubernetes Architecture](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/architecture/architecture.md) +- [Kubernetes General Configuration Tips](https://kubernetes.io/docs/concepts/configuration/overview/#general-configuration-tips) +- [Kubernetes Design and Development Explained](https://thenewstack.io/kubernetes-design-and-development-explained/) diff --git a/doc/dev/campaigns_development.md b/doc/dev/campaigns_development.md index 91992a365667..eda7553a2007 100644 --- a/doc/dev/campaigns_development.md +++ b/doc/dev/campaigns_development.md @@ -4,22 +4,25 @@ Before diving into the technical part of campaigns, make sure to read up on what campaigns are, what they're not and what we want them to be. -1. Start by looking at the product page for [code change management](https://about.sourcegraph.com/product/code-change-management) -1. Read through the first page of the [campaigns documentation](https://docs.sourcegraph.com/user/campaigns/) **IMPORTANT:** Watch the video! +1. Start by looking at the [campaigns description on about.sourcegraph.com](https://about.sourcegraph.com). +1. Read through the [campaigns documentation](https://docs.sourcegraph.com/user/campaigns). + +## [Campaigns design doc](campaigns_design.md) + +See "[Campaigns design doc](campaigns_design.md)". ## Starting up your environment -1. Run `./enterprise/dev/start.sh` — Wait until all repositories are cloned. -2. Follow the [campaigns "Getting started" guide](../user/campaigns/getting_started.md) to setup campaigns. -3. Create [your first campaign](../user/campaigns/creating_campaign_from_patches.md). **Remember:** If you create a campaign, you're opening real PRs on GitHub. Make sure only [testing repositories](#github-testing-account) are affected. If you create a large campaign, it takes a while to preview/create but also helps a lot with finding bugs/errors, etc. +1. Run `./enterprise/dev/start.sh` and wait until all repositories are cloned. +1. Create [your first campaign](../user/campaigns/hello_world_campaign.md). **Remember:** If you create a campaign, you're opening real PRs on GitHub. Make sure only [testing repositories](#github-testing-account) are affected. If you create a large campaign, it takes a while to preview/create but also helps a lot with finding bugs/errors, etc. ## Glossary -The code campaigns feature introduces a lot of new names, GraphQL queries and mutations and database tables. This section tries to explain the most common names and provide a mapping between the GraphQL types and their internal counterpart in the Go backend. +The campaigns feature introduces a lot of new names, GraphQL queries and mutations and database tables. This section tries to explain the most common names and provide a mapping between the GraphQL types and their internal counterpart in the Go backend. | GraphQL type | Go type | Database table | Description | | ------------------- | -------------------- | -------------------| ----------- | -| `Campaign` | `campaigns.Campaign` | `campaigns` | A campaign is a collection of changesets on code hosts. The central entity. | +| `Campaign` | `campaigns.Campaign` | `campaigns` | A campaign is a collection of changesets. The central entity. | | `ExternalChangeset` | `campaigns.Changeset` | `changesets` | Changeset is the unified name for pull requests/merge requests/etc. on code hosts. | | `PatchSet` | `campaigns.PatchSet` | `patch_sets` | A patch set is a collection of patches that will be applied by creating and publishing a campaign. A campaign *has one* patch set. | | `Patch` | `campaigns.Patch` | `patches` | A patch for a repository that *can* be turned into a changeset on a code host. It belongs to a patch set, which has multiple patches, one per repository. | diff --git a/doc/user/campaigns/CampaignSpec.schema.json b/doc/user/campaigns/CampaignSpec.schema.json new file mode 100644 index 000000000000..abdc239a65b3 --- /dev/null +++ b/doc/user/campaigns/CampaignSpec.schema.json @@ -0,0 +1,118 @@ +{ + "$schema": "http://json-schema.org/draft-07/schema#", + "title": "CampaignSpec", + "description": "A campaign specification, which describes the campaign and what kinds of changes to make (or what existing changesets to track).", + "type": "object", + "additionalProperties": false, + "required": ["name"], + "properties": { + "name": { + "type": "string", + "description": "The name of the campaign, which is unique among all campaigns in the namespace. A campaign's name is case-preserving.", + "pattern": "^[\\w.-]+$" + }, + "description": { + "type": "string", + "description": "The description of the campaign." + }, + "on": { + "type": "array", + "description": "The set of repositories (and branches) to run the campaign on, specified as a list of search queries (that match repositories) and/or specific repositories.", + "items": { + "title": "OnQueryOrRepository", + "oneOf": [ + { + "title": "OnQuery", + "type": "object", + "description": "A Sourcegraph search query that matches a set of repositories (and branches). Each matched repository branch is added to the list of repositories that the campaign will be run on.", + "additionalProperties": false, + "required": ["repositoriesMatchingQuery"], + "properties": { + "repositoriesMatchingQuery": { + "type": "string", + "description": "A Sourcegraph search query that matches a set of repositories (and branches). If the query matches files, symbols, or some other object inside a repository, the object's repository is included.", + "examples": ["file:README.md"] + } + } + }, + { + "title": "OnRepository", + "type": "object", + "description": "A specific repository (and branch) that is added to the list of repositories that the campaign will be run on.", + "additionalProperties": false, + "required": ["repository"], + "properties": { + "repository": { + "type": "string", + "description": "The name of the repository (as it is known to Sourcegraph).", + "examples": ["github.com/foo/bar"] + }, + "branch": { + "type": "string", + "description": "The branch on the repository to propose changes to. If unset, the repository's default branch is used." + } + } + } + ] + } + }, + "steps": { + "type": "array", + "description": "The sequence of commands to run (for each repository branch matched in the `on` property) to produce the campaign's changes.", + "items": { + "title": "Step", + "type": "object", + "description": "A command to run (as part of a sequence) in a repository branch to produce the campaign's changes.", + "additionalProperties": false, + "required": ["run", "container"], + "properties": { + "run": { + "type": "string", + "description": "The shell command to run in the container. It can also be a multi-line shell script. The working directory is the root directory of the repository checkout." + }, + "container": { + "type": "string", + "description": "The Docker image used to launch the Docker container in which the shell command is run.", + "examples": ["alpine:3"] + }, + "env": { + "type": "object", + "description": "Environment variables to set in the environment when running this command." + } + } + } + }, + "changesetTemplate": { + "type": "object", + "description": "A template describing how to create (and update) changesets with the file changes produced by the command steps.", + "additionalProperties": false, + "required": ["title", "branch", "commit", "published"], + "properties": { + "title": { "type": "string", "description": "The title of the changeset." }, + "body": { "type": "string", "description": "The body (description) of the changeset." }, + "branch": { + "type": "string", + "description": "The name of the Git branch to create or update on each repository with the changes." + }, + "commit": { + "title": "GitCommitDescription", + "type": "object", + "description": "The Git commit to create with the changes.", + "additionalProperties": false, + "required": ["message"], + "properties": { + "message": { + "type": "string", + "description": "The Git commit message." + } + } + }, + "published": { + "type": "boolean", + "description": "Whether to publish the changeset. An unpublished changeset can be previewed on Sourcegraph by any person who can view the campaign, but its commit, branch, and pull request aren't created on the code host. A published changeset results in a commit, branch, and pull request being created on the code host.", + "$comment": "TODO(sqs): Come up with a way to specify that only a subset of changesets should be published. For example, making `published` an array with some include/exclude syntax items." + } + } + } + } +} diff --git a/doc/user/campaigns/actions.md b/doc/user/campaigns/actions.md deleted file mode 100644 index cf2afe2c7dbb..000000000000 --- a/doc/user/campaigns/actions.md +++ /dev/null @@ -1,237 +0,0 @@ -# Actions - -## What are actions? - -The `src` CLI offers the ability to _execute actions_ with the `src action exec` command to **produce a set of patches**, one per repository. - -These **patches can then be turned into _changesets_ (pull requests)** on the code hosts on which the repositories are hosted by creating a campaign (see "[Creating a campaign from patches](./creating_campaign_from_patches.md)"). - -An _action_ is made up of two things: - -- A Sourcegraph search query (the `scopeQuery`). -- A series of steps to be executed in each repository yielded by the search query. - -Here is an example definition of an action: - -```json -{ - "scopeQuery": "lang:go gopkg.in\/inconshreveable\/log15.v2", - "steps": [ - { - "type": "docker", - "image": "comby/comby", - "args": [ - "-in-place", - "import (:[before]\"gopkg.in/inconshreveable/log15.v2\":[after])", - "import (:[before]\"github.com/inconshreveable/log15\":[after])", - ".go", - "-matcher", - ".go", - "-d", "/work", - "-exclude-dir", ".,vendor" - ] - }, - { - "type": "command", - "args": ["goimports", "-w", "."] - } - ] -} -``` - -This action uses [Comby](https://comby.dev) to update a Go import path. - -The `"scopeQuery"` yields every repository in which the old import path is mentioned in a Go file. - -The first step, of type `"docker"`, executes the Docker image `comby/comby` (with each repository mounted under `/work`) to rewrite the import path from `gopkg.in/inconshreveable/log15.v2` to `github.com/inconshreveable/log15`. - -The second step, a `"command"`, then runs [`goimports`](https://pkg.go.dev/golang.org/x/tools/cmd/goimports) to ensure that updating the import paths worked and that the code is correctly formatted. - -Since it is a `"command"`, the `goimports` step doesn't use a Docker container, but instead runs the `goimports` executable in a temporary directory on the machine on which the `src` CLI is executed. - -## Requirements - -To execute actions with the `src` CLI the following is required: - -- `src` CLI, setup to point to your Sourcegraph instance. -- [git](https://git-scm.com/). -- [Docker](https://www.docker.com/), if you want to execute Docker containers. - -## Defining an action - -An action definition is a JSON file ([JSON schema for actions](https://raw.githubusercontent.com/sourcegraph/src-cli/master/schema/actions.schema.json)) and needs to specify: - -- `"scopeQuery"` - a Sourcegraph search query to generate a list of repositories over which to run the action. -- `"steps"` - a list of action steps to execute in each repository. - -A single step can either be a of type `"command"`, which means the step is executed on the machine on which `src actions exec` is executed, or it can be of type `"docker"` which then runs a container in which the repository is mounted. - -To create a new, empty action definition you can use the following helper command: - -``` -$ src actions create -``` - -That will create a new `action.json` file in the current directory (see `src actions create -h` for how specify a filename) with a reference to the [JSON schema](https://raw.githubusercontent.com/sourcegraph/src-cli/master/schema/actions.schema.json). - -### The scope query - -The scope query, specified as `"scopeQuery"` in an action definition, is a [Sourcegraph search](../search/index.md) query that's executed to yield a list of repositories in which to execute an action. - -It doesn't have to use `type:repo` to only search for repositories, because `src action exec` will construct a _unique list of repositories_ associated with each search result. If the scope query, for example, yields eight search results in three different repositories, `src action exec` will execute the action in these three repositories. - -Examples: - -- `lang:go fmt.Sprintf` returns repositories in which `fmt.Sprintf` appears in a Go file. -- `lang:go fmt.Sprintf repo:github.com/my-org/[a-c]` returns repositories in which `fmt.Sprintf` appears in a Go file, if they are part of the `my-org` GitHub organization and their name begins with `a`, `b` or `c`. -- `repohasfile:yarn.lock` returns repositories in which a file called `yarn.lock` exists. - -See "[Search query syntax](../search/queries.md)" and "[Search examples](../search/examples.md)" for more information. - -If you want to see which repositories are yielded by a `"scopeQuery"` in an action definition without execution the action, use `src action scope-query`: - -``` -$ src action scope-query -f my-action-definition.json -``` - -This will return a list of repositories. - -Since action definitions are JSON files and require the `"scopeQuery"` to be escaped, it often helps to use `src action scope-query` in combination with the `-v` flag to see exactly which query is sent to the Sourcegraph instance: - -``` -$ src -v action scope-query -f my-action-definition.json -``` - -### Docker steps - -A Docker step specification requires three attributes: `"type"`, `"image"` and `"args"`. - -The `"type"` is `"docker"`. - -`"image"` is the Docker image that is executed. - -`"args"` is a list of arguments to be passed to `docker run`. - -Here is an example Docker step in an action definition: - -```json -{ - "type": "docker", - "image": "alpine:3", - "args": ["sh", "-c", "find /work -iname '*.txt' -type f | xargs -n 1 sed -i s/this/that/g"] -} -``` - -This step will effectively execute the following for each repository yielded by the scope query: - -``` -docker run -it --rm --workdir /work --mount type=bind,source=,target=/work -- alpine:3 \ - sh -c 'find /work -iname '*.txt' -type f | xargs -n 1 sed -i s/this/that/g' -``` - -Note the `` placeholder: when executing an action, each repository yielded by the scope query is extracted into a temporary directory, which gets reused for all steps in an action. - -That temporary directory is then mounted into each Docker container under `/work`. That's why the `find` command in the example searches in `/work`. - -If you need more directories (outside of the repository) to be persisted across multiple `"docker"` steps, you can use the `"cacheDirs"` property of a step definition. Example: - -```json -{ - "scopeQuery": "lang:go repo:sourcegraph/sourcegraph$", - "steps": [ - { - "type": "docker", - "image": "alpine:3", - "args": ["sh", "-c", "echo 'hello from cache 1' > /cache1/hello.txt && echo 'hello from cache 2' > /cache2/hello.txt"], - "cacheDirs": ["/cache1", "/cache2"] - }, - { - "type": "docker", - "image": "alpine:3", - "args": ["sh", "-c", "cp /cache1/hello.txt /work/hello_cache1.txt && cp /cache2/hello.txt /work/hello_cache2.txt"], - "cacheDirs": ["/cache1", "/cache2"] - } - ] -} -``` - -Note the `cacheDirs` properties used by each step. For every entry in `cacheDirs`, `src action exec` will create a temporary directory that persists across all steps and which is then mounted under the specified name into the container. - -This can be used, for example, to cache package manager installations across multiple steps. - -### Command steps - -A command step specification requires two attributes: `"type"` and `"args"`. - -The `"type"` is `"command"`. - -`"args"` is a list of consisting of the command to be executed and its arguments. - -Example: - -```json -{ - "type": "command", - "args": ["sed", "-i", "", "s/this/that/g README.md"] -} -``` - -This will execute `sed -i '' s/this/that/g README.md` _on the machine on which `src action exec` is being executed_. There are no containers involved. - -The current working directory for each `"command"` step is the root of each repository (extracted into a temporary directory on the machine on which `src action exec` is executed) yielded by the scope query. - -## Executing - -After creating an action definition and saving it to a file, it can be executed by running `src action exec`: - -``` -src action exec -f my-action-definition.json -``` - -What this does is the following: - -1. Send the `"scopeQuery"` to the configured Sourcegraph instance and turn the results into a unique list of repositories. -1. Download a ZIP archive of each repository in that list and, _on the machine on which `src action exec` is executed_, extract it to a local temporary directory in `/tmp`. -1. Execute the action for each repository in parallel, with each step in an action being executed sequentially on the same copy of the repository. -1. Produce a patch for each repository by creating _a diff between a fresh copy of the repository and the directory in which the action ran_. - -Run `src action exec -h` to get a complete overview of which flags are accepted by `src action exec`, but noteworthy are: - -* `-keep-logs` causes `src action exec` to redirect STDOUT/STDERR of each step to a log file and not clean that up after execution has finished. -* `-j` specifies the number of parallel jobs, where one job is the action being executed in a single repository. -* `-clear-cache` clears the cache before executing an action. -* `-create-patchset` creates a patchset out of the produced patches, which can then be used to create a campaign. - -If you are experimenting with actions and want to get more insight, pass the `-v` flag to `src` to see the what each step prints to STDOUT and STDOUT: - -``` -src -v action exec -f my-action-definition.json -``` - -### Where to run `src action exec` - -The steps of an action are executed on the machine where the `src` CLI is installed and executed. - -For most usecases that involve a lot of repositories and action steps that require a lot of resources, we recommend that `src` CLI should be run on a Linux machine with considerable CPU, RAM, and network bandwidth to reduce the execution time. Putting this machine in the same network as your Sourcegraph instance will also improve performance. - -Another factor affecting execution time is the number of jobs executed in parallel, which is by default the number of cores on the machine. This can be adjusted using the `-j` parameter. - -## Creating patchsets - -In order to create a patchset out of the patches produced by executing an action, pipe the output to `src campaign patchset create-from-patches`: - -``` -src actions exec -f my-action-definition.json | src campaign patchset create-from-patches -``` - -Or pass the `-create-patchset` flag directly to `src action exec`: - -``` -src actions exec -f my-action-definition.json -create-patchset -``` - -If the action failed to execute in one of the repositories, `src action exec` will ask for confirmation to create a patchset anyway, if `-create-patchset` is given. In order to _always create a patchset_, without asking for confirmation, use the `-force-create-patchset` flag: - -``` -src actions exec -f my-action-definition.json -force-create-patchset -``` diff --git a/doc/user/campaigns/campaigns-icon.svg b/doc/user/campaigns/campaigns-icon.svg new file mode 100644 index 000000000000..87a54595f808 --- /dev/null +++ b/doc/user/campaigns/campaigns-icon.svg @@ -0,0 +1,5 @@ + + + + + diff --git a/doc/user/campaigns/configuration.md b/doc/user/campaigns/configuration.md deleted file mode 100644 index 81fa5b7ad1f8..000000000000 --- a/doc/user/campaigns/configuration.md +++ /dev/null @@ -1,29 +0,0 @@ -# Configuration - -## Enabling campaigns - -In order to use campaigns, a site-admin of your Sourcegraph instance must enable it in the site configuration settings e.g. `sourcegraph.example.com/site-admin/configuration` - -```json -{ - "experimentalFeatures": { - "automation": "enabled" - } -} -``` - -## Read-access for non-site-admins - -Without any further configuration, campaigns are **only accessible to site admins.** If you want to grant read-only access to non-site-admins, use the following site configuration setting: - -```json -{ - "campaigns.readAccess.enabled": true -} -``` - -## Code host configuration - -When using campaigns with repositories hosted on GitHub, make sure that the GitHub connection configured in Sourcegraph uses a token with the [required token scopes](../../admin/external_service/github.md#github-api-token-and-access). Otherwise campaigns won't be able to create changesets (pull requests) on the configured GitHub instance and sync them back to Sourcegraph. - -The user associated with the token also needs to have write-access to the repository in order to create changesets when creating campaigns. diff --git a/doc/user/campaigns/creating_campaign_from_patches.md b/doc/user/campaigns/creating_campaign_from_patches.md deleted file mode 100644 index 226b64bbbb76..000000000000 --- a/doc/user/campaigns/creating_campaign_from_patches.md +++ /dev/null @@ -1,139 +0,0 @@ -# Creating a campaign from patches - -A campaign can be created from a set of patches, one per repository. For each patch, a changeset (what code hosts call _pull request_ or _merge request_) will be created on the code host on which the repository is hosted. - -Here is the short version for how to create a patch set and turn that into changesets by creating a campaign: - -1. Create an action definition and save to a JSON file (e.g. `action.json`). -1. _Optional_: See repositories the action would run over: - - ``` - src actions scope-query -f action.json - ``` -1. Create a set of patches by executing the action: - - ``` - src actions exec -f action.json -create-patchset - ``` -1. Click on the URL that's printed to create a campaign. - -Read on for more detailed steps and documentation and see "[Actions](./actions.md)" for more information about how to define and execute actions. - -## Requirements - -If you have not done so already, first [install](https://github.com/sourcegraph/src-cli), [set up and configure](https://github.com/sourcegraph/src-cli#setup) the `src` CLI to point to your Sourcegraph instance. - -## 1. Defining an action - -The first thing we need is a definition of an "action". An action contains a list of steps to run in each repository returned by the results of the `scopeQuery` search string. There are two types of steps: `docker` and `command`. See "[Actions](./actions.md)" for more information. - -Here is an example of a multi-step action definition using the `docker` and `command` types: - -```json -{ - "scopeQuery": "repo:go-* -repohasfile:INSTALL.md", - "steps": [ - { - "type": "command", - "args": ["sh", "-c", "echo '# Installation' > INSTALL.md"] - }, - { - "type": "command", - "args": ["sed", "-i", "", "s/No install instructions/See INSTALL.md/", "README.md"] - }, - { - "type": "docker", - "image": "golang:1.13-alpine", - "args": ["go", "fix", "/work/..."] - } - ] -} -``` - -**Save that definition in a file called `action.json` (or any other name of your choosing).** - -This action will be executed for each repository that has `go-` in its name and doesn't have an `INSTALL.md` file. - -- **The first step** (a `command` step) creates an `INSTALL.md` file in the root directory of each repository by running `sh` in a temporary copy of each repository. This is executed on the machine on which `src` is being run. -- **The second step**, again a `"command"` step, runs the `sed` command to replace text in the `README.md` file in the root of each repository (the `-i ''` argument is only necessary for BSD versions of `sed` that usually come with macOS). - - > NOTE: The executed command is simply `sed` which means its arguments are _not_ expanded, as they would be in a shell. To achieve that, execute the `sed` as part of a shell invocation (using `sh -c` and passing in a single argument, for example, like in the first step). - -- **The third step** starts a Docker container based on the `golang:1.13-alpine` image and runs `go fix /work/...` in it. - -As you can see from these examples, the "output" of an action is the modified, local copy of a repository. - - -## 2. Executing an action to produce patches - -With our action file defined, we can now execute it: - -``` -src actions exec -f action.json -``` - -This command is going to: - -1. Download a ZIP archive of each repository returned by the `"scopeQuery"` and extract it to a local temporary directory in `/tmp`. -1. Execute the action for each repository in parallel, with each step in an action being executed sequentially on the same copy of a repository. -1. Produce a patch for each repository with creating diff between a fresh copy of the repository and the directory in which the action ran. - -(See "[Actions](./actions.md)" for more information about how actions are executed.) - -The output, a set of patches, can be saved into a file by redirecting it: - -``` -src actions exec -f action.json > patches.json -``` - -## 3. Creating a patch set from patches - -The next step is to save the set of patches on the Sourcegraph instance so they can be turned into a campaign. - -To do that, run: - -``` -src campaign patchset create-from-patches < patches.json -``` - -Or leverage the cache of `src action exec` and re-run the command to pipe the patches directly into `src campaign patchset create-from-patches`: - -``` -src actions exec -f action.json | src campaign patchset create-from-patches -``` - -You can also use `src action exec` with the `-create-patchset` flag, which is equivalent to the last command: - -``` -src actions exec -f action.json -create-patchset -``` - -Once completed, the output will contain: - -- The URL to preview the changesets that would be created on the code hosts. -- The command for the `src` SLI to create a campaign from the patch set. - -## 4. Publishing a campaign - -If you're happy with the preview of the campaign, it's time to trigger the creation of changesets (pull requests) on the code host(s). - -That is done by creating and publishing a campaign with the given patchset. - -You can either do that in the Sourcegraph UI or on the CLI: - -``` -src campaigns create -name='My campaign name' \ - -desc='My first CLI-created campaign' \ - -patchset=Q2FtcGFpZ25QbGFuOjg= \ - -branch=my-first-campaign -``` - -Creating this campaign will asynchronously create a changeset (pull request) for each repository that has a patch in the patch set. You can check the progress of campaign completion by viewing the campaign on your Sourcegraph instance. - -The `-branch` flag specifies the branch name that will be used for each pull request. If a branch with that name already exists for a repository, a fallback will be generated by appending a counter at the end of the name, e.g.: `my-first-campaign-1`. - -If you have defined the `$EDITOR` environment variable, the configured editor will be used to edit the name and Markdown description of the campaign: - -```sh -src campaigns create -patchset=Q2FtcGFpZ25QbGFuOjg= -branch=my-first-campaign -``` diff --git a/doc/user/campaigns/creating_manual_campaign.md b/doc/user/campaigns/creating_manual_campaign.md deleted file mode 100644 index 8469d0775296..000000000000 --- a/doc/user/campaigns/creating_manual_campaign.md +++ /dev/null @@ -1,11 +0,0 @@ -# Creating a manual campaign - -Manual campaigns provide the ability to manage and monitor changesets (pull requests) that already exist on code hosts. - -In order to create a manual campaign, follow these steps: - -1. Go to `/campaigns` on your Sourcegraph instance and click on the **New campaign** button. -1. Click on **Track existing changesets**. -1. Fill in a title for the campaign and a description. -1. Click **Create**. -1. Add changesets by specifying the name of the repository they belong to and their external ID (e.g. the number of a pull request on GitHub) in the **Add changeset** form. diff --git a/doc/user/campaigns/drafts.md b/doc/user/campaigns/drafts.md deleted file mode 100644 index 135173606e01..000000000000 --- a/doc/user/campaigns/drafts.md +++ /dev/null @@ -1,5 +0,0 @@ -# Campaign drafts - -A campaign can be created as a draft, either by adding the `-draft` flag to the `src campaign create` command, or by selecting `Create draft` in the web UI. - -When a campaign is a draft, no changesets will be created until the campaign is published, or each changeset is individually published. This can be done in the Sourcegraph campaign web interface. diff --git a/doc/user/campaigns/examples/eslint_typescript_version.md b/doc/user/campaigns/examples/eslint_typescript_version.md index a6e117f9b627..70ffccec2bf7 100644 --- a/doc/user/campaigns/examples/eslint_typescript_version.md +++ b/doc/user/campaigns/examples/eslint_typescript_version.md @@ -1,5 +1,7 @@ # Example: Using ESLint to automatically migrate to a new TypeScript version +> TODO(sqs): update for new campaigns flow + Our goal for this campaign is to convert all TypeScript code synced to our Sourcegraph instance to make use of new TypeScript features. To do this we convert the code, then update the TypeScript version. To convert the code we install and run ESLint with the desired `typescript-eslint` rules, using the [`--fix` flag](https://eslint.org/docs/user-guide/command-line-interface#fix) to automatically fix problems. We then update the TypeScript version using [`yarn upgrade`](https://legacy.yarnpkg.com/en/docs/cli/upgrade/). diff --git a/doc/user/campaigns/examples/index.md b/doc/user/campaigns/examples/index.md index bdf10574dfe5..472b0fdcb18e 100644 --- a/doc/user/campaigns/examples/index.md +++ b/doc/user/campaigns/examples/index.md @@ -1,7 +1,9 @@ # Example campaigns +> TODO(sqs): update for new campaigns flow + The following examples demonstrate various types of campaigns for different languages using both commands and Docker images. They also provide commentary on considerations such as adjusting the duration (`-timeout`) for actions that exceed the 15 minute default limit. * [Using ESLint to automatically migrate to a new TypeScript version](./eslint_typescript_version.md) * [Adding a GitHub action to upload LSIF data to Sourcegraph](./lsif_action.md) -* [Refactor Go code using Comby](./refactor_go_comby.md) +* [Refactoring Go code using Comby](./refactor_go_comby.md) diff --git a/doc/user/campaigns/examples/lsif_action.md b/doc/user/campaigns/examples/lsif_action.md index c3f1998915b7..151e150cc47f 100644 --- a/doc/user/campaigns/examples/lsif_action.md +++ b/doc/user/campaigns/examples/lsif_action.md @@ -1,11 +1,11 @@ # Example: Adding a GitHub action to upload LSIF data to Sourcegraph +> TODO(sqs): update for new campaigns flow + Our goal for this campaign is to add a GitHub Action that generates and uploads LSIF data to Sourcegraph by adding a `.github/workflows/lsif.yml` file to each repository that doesn't have it yet. The first thing we need is an action definition that we can execute with the [`src` CLI tool](https://github.com/sourcegraph/src-cli) and its `src actions exec` subcommand. ->NOTE: See "[Actions](../actions.md)" for more details on how to define and execute actions. - Here is an `action.json` file that runs a Docker container based on the Docker image called `add-lsif-to-build-pipeline-action` in each repository that has a `go.mod` file, `github` in its name and no `.github/workflows/lsif.yml` file: ```json diff --git a/doc/user/campaigns/examples/refactor_go_comby.md b/doc/user/campaigns/examples/refactor_go_comby.md index 1e68fd35bd64..2555f098093d 100644 --- a/doc/user/campaigns/examples/refactor_go_comby.md +++ b/doc/user/campaigns/examples/refactor_go_comby.md @@ -1,4 +1,6 @@ -# Example: Refactor Go code using Comby +# Example: Refactoring Go code using Comby + +> TODO(sqs): update for new campaigns flow Our goal for this campaign is to simplify Go code by using [Comby](https://comby.dev/) to rewrite statements of this form diff --git a/doc/user/campaigns/getting_started.md b/doc/user/campaigns/getting_started.md deleted file mode 100644 index 63728202717c..000000000000 --- a/doc/user/campaigns/getting_started.md +++ /dev/null @@ -1,24 +0,0 @@ -# Getting started - -Follow these steps to get started with campaigns on your Sourcegraph instance: - -1. **Enable the campaigns feature flag**: "[Enabling Campaigns](./configuration.md#enabling-campaigns)". - - Since campaigns are currently in beta, they're behind a feature flag and need to be enabled by a site admin. - -1. _Optional_: Make campaigns accessible to non-site-admins. "[Read-access for non-site-admins](./configuration.md#read-access-for-non-site-admins)". - -1. **Check your code host configuration**: "[Code host configuration](./configuration.md#code-host-configuration)". - - Since campaigns create changesets on code hosts, the code host configuration in Sourcegraph and the account associated with it need the correct access rights. - -1. **Set up the `src` CLI on your machine**: [Installation and setup instructions](https://github.com/sourcegraph/src-cli/#installation). - -Now you're ready to [create a campaign from patches](./creating_campaign_from_patches.md) or to [create a manual campaign](./creating_manual_campaign.md). Make sure to also take a look at the [**example campaigns**](./examples/index.md). - ---- - -It's optional, but we **highly recommended to setup webhook integration** on your Sourcegraph instance for optimal syncing performance between your code host and Sourcegraph. - -* GitHub: [Configuring GitHub webhooks](https://docs.sourcegraph.com/admin/external_service/github#webhooks). -* Bitbucket Server: [Setup the `bitbucket-server-plugin`](https://github.com/sourcegraph/bitbucket-server-plugin), [create a webhook](https://github.com/sourcegraph/bitbucket-server-plugin/blob/master/src/main/java/com/sourcegraph/webhook/README.md#create) and configure the `"plugin"` settings for your [Bitbucket Server code host connection](https://docs.sourcegraph.com/admin/external_service/bitbucket_server#configuration). diff --git a/doc/user/campaigns/hello_world_campaign.md b/doc/user/campaigns/hello_world_campaign.md new file mode 100644 index 000000000000..7c3b1139f6bb --- /dev/null +++ b/doc/user/campaigns/hello_world_campaign.md @@ -0,0 +1,93 @@ +# Hello World Campaign + +> TODO(sqs): This will eventually go in a new "Sourcegraph Guides" docs section, but it lives here for now. + +Have you ever needed to make the same kind of change to many repositories at once? Campaigns make this much easier. To get you started, let's run a very simple campaign: adding the line `Hello World` to all of your repositories' `README.md` files. After completing this exercise, you'll be able to create your own campaigns to make useful changes, fixes, refactors, and more. + +You'll learn how to: + +- Create a new campaign +- Select the repositories to work with +- Write the script to make changes to each repository +- Publish and monitor the status of your proposed changes + +For more detailed information, see "[Campaigns](index.md)" in Sourcegraph documentation. + +## What is a campaign? + +A campaign lets you make many related code changes, creating many branches and changesets (such as GitHub pull requests) across many repositories. You can track the progress as they are reviewed and merged. See "[About campaigns](index.md#about-campaigns)" for more information. + +To use campaigns, you need to: + +- [Set up a Sourcegraph instance](../../index.md#quickstart) and add some repositories to it. +- [Install Sourcegraph CLI](https://github.com/sourcegraph/src-cli) (`src`). + +## Step 1. Write a campaign spec + +A **campaign spec** is a YAML file that defines a campaign, including: + +- The name and description of the campaign +- The set of repositories to change +- Commands to run in each repository to make the changes +- The commit message and branch name + +Save the following campaign spec as `hello-world.campaign.yaml`: + +```yaml +name: hello-world +description: Add Hello World to READMEs + +# Find all repositories that contain a README.md file. +on: + - repositoriesMatchingQuery: file:README.md + +# In each repository, run this command. Each repository's resulting diff is captured. +steps: + - run: echo Hello World | tee -a $(find -name README.md) + container: alpine:3 + +# Describe the changeset (e.g., GitHub pull request) you want for each repository. +changesetTemplate: + title: Hello World + body: My first campaign! + branch: hello-world # Push the commit to this branch. + commit: + message: Append Hello World to all README.md files + published: false +``` + +## Step 2. Create the campaign + +Let's see the changes that will be made. Don't worry---no commits, branches, or changesets will be published yet (the repositories on your code host will be untouched). + +1. In your terminal, run this command: + +
    src campaign apply -f hello-world.campaign.yaml -preview
    +1. Wait for it to run and compute the changes for each repository. +1. When it's done, click the displayed link to see all of the changes that will be made. +1. Make sure the changes look right. + + > If you want to run the campaign on fewer repositories, change the roots query in `hello-world.campaign.yaml` to something like `file:README.md repo:myproject` (to only match repositories whose name contains `myproject`). +1. Click the **Create campaign** button. + +You created your first campaign! The campaign's changesets are still unpublished, which means they exist only on Sourcegraph and haven't been pushed to your code host yet. + +## Step 3. Publish the changes (optional) + +Publishing causes commits, branches, and changesets to be created on your code host. + +You probably don't want to publish these toy "Hello World" changesets to actively developed repositories, because that might confuse people ("Why did you add this line to our READMEs?"). On a real campaign, you would click the **Publish** button next to a changeset to publish it (or the **Publish all** button to publish all changesets). + +## Congratulations! + +You've created your first campaign! 🎉🎉 + +You can customize your campaign spec and experiment with making other types of changes. To update your campaign, edit `hello-world.campaign.yaml` and run `src campaign apply -f hello-world.campaign.yaml -preview` again. (As before, you'll see a preview before any changes are applied.) + +Here are some [example campaigns](examples/index.md) for inspiratiopn: + +- [Using ESLint to automatically migrate to a new TypeScript version](examples/eslint_typescript_version.md) +- [Adding a GitHub action to upload LSIF data to Sourcegraph](examples/lsif_action.md) +- [Refactoring Go code using Comby](examples/refactor_go_comby.md) + +To learn what else you can do with campaigns, see "[Campaigns](index.md)" in Sourcegraph documentation. diff --git a/doc/user/campaigns/index.md b/doc/user/campaigns/index.md index 0abbc9e3294c..18b734eb6f6f 100644 --- a/doc/user/campaigns/index.md +++ b/doc/user/campaigns/index.md @@ -1,90 +1,253 @@ # Campaigns ->NOTE: **Campaigns are currently in beta.** We're actively building out the feature set and improving the user experience with every update. Let us know what you think! [File an issue](https://github.com/sourcegraph/sourcegraph) with feedback/problems/questions, or [contact us directly](https://about.sourcegraph.com/contact). +Campaigns let you make large-scale code changes across many repositories. -## What are campaigns? +> NOTE: Campaigns are in beta. -Campaigns are part of [Sourcegraph code change management](https://about.sourcegraph.com/product/code-change-management) and let you make large-scale code changes across many repositories and different code hosts. +## About campaigns -You provide the code to make the change, and campaigns provide the plumbing to turn it into a large-scale code change campaign and monitor its progress. +A campaign streamlines the creation and tracking of pull requests across many repositories and code hosts. After you create a campaign, you tell it what changes to make (by providing a list of repositories and a script to run in each). The campaign lets you create pull requests on all affected repositories, and it tracks their progress until they're all merged. You can preview the changes and update them at any time. -
    - -
    +People usually use campaigns to make the following kinds of changes: -## Are you a first time user of campaigns? +- Cleaning up common problems using linters +- Updating uses of deprecated library APIs +- Upgrading dependencies +- Patching critical security issues +- Standardizing build, configuration, and deployment files -If you are a first-time user of campaigns, we recommend that you read through the following sections of the documentation: +For step-by-step instructions to create your first campaign, see [Hello World Campaign](hello_world_campaign.md) in Sourcegraph Guides. -1. Read through the **[How it works](#how-it-works)** section below and **watch the video** to get an understanding of how campaigns work. -1. Go through the "[Getting started](./getting_started.md)" instructions to setup your Sourcegraph instance for campaigns. -1. Create your first campaign from a set of patches by reading "[Creating a campaign from patches](./creating_campaign_from_patches.md)". -1. Create a manual campaign to track the progress of already-existing pull requests on your code host: "[Creating a manual campaign](./creating_manual_campaign.md)". + -
    -
    -
    - - - -
    -
    A campaign tracking multiple changesets in different repositories.
    -
    -
    + -Once the campaign is created, you can track the **review state, CI status and open/closed/merged lifecycle of each changeset in the Sourcegraph UI**. +## Supported code hosts and changeset types -You should use campaigns if you want to +The generic term **changeset** is used to refer to any of the following: -* run code to make changes across a large number of repositories. -* keep track of a large number of pull requests and their status on GitHub or Bitbucket Server instances. -* execute commands to upgrade dependencies in multiple repositories. -* use Sourcegraph's search and replace matches by running code in the matched repositories. +- GitHub pull requests +- Bitbucket Server pull requests +- Bitbucket Cloud pull requests (not yet supported) +- GitLab merge requests (not yet supported) +- Phabricator diffs (not yet supported) +- Gerrit changes (not yet supported) -
    +A single campaign can span many repositories and many code hosts. -## How it works +## Viewing campaigns -See this video for a demonstration of lifecycle of a campaign: +You can view a list of all campaigns by clicking the Campaigns icon campaigns icon in the top navigation bar. -
    -
    -
    - -
    -
    Campaign: Running gofmt in each repository containing a go.mod file.
    -
    -
    +Use the filters to switch between showing all campaigns, open campaigns, or closed campaigns. -1. With the `src` CLI the user **generates a set of patches** by running `gofmt` over every repository that has a `go.mod` file, leveraging Sourcegraphs search capabilities. +If you lack read access to a repository in a campaign, you can only see [limited information about the changes to that repository](managing_access.md#repository-permissions-for-campaigns) (and not the repository name, file paths, or diff). - This is called **executing an _action_** (an _action_ is a series of commands and Docker containers to run in each repository) and yields **set of patches**, one for each repository, which you can inspect either in the CLI or in the Sourcegraph UI. -1. The patches are then used to **create a draft campaign**. -1. At this point, since it's a draft camapaign, no changesets (_pull requests_ in the case of GitHub here) have been created on the code host. -1. The user then selectively **creates GitHub pull requests** by publishing single patches. +### Campaign specs -
    +You can create or update a campaign from a [campaign spec](#campaign-spec), which is a YAML file that defines a campaign. -## Requirements +See the "[Creating a campaign](#creating-a-campaign)" section for an example campaign spec YAML file. -* Sourcegraph instance [configured for campaigns](./configuration.md). -* `src` CLI: [Installation and setup instructions](https://github.com/sourcegraph/src-cli/#installation) +For more information, see: -## Limitations +- [Creating a campaign](#creating-a-campaign) from a campaign spec +- [Updating a campaign](#updating-a-campaign) from a campaign spec +- TODO(sqs) Campaign spec YAML reference +- [Example campaign specs](examples/index.md) + +## Creating a campaign + +> **Creating your first campaign?** See [Hello World Campaign](hello_world_campaign.md) in Sourcegraph Guides for step-by-step instructions. + +You can create a campaign from a [campaign spec](#campaign-spec), which is a YAML file that describes your campaign. + +The following example campaign spec adds "Hello World" to all `README.md` files: + +```yaml +name: hello-world +description: Add Hello World to READMEs + +# Find all repositories that contain a README.md file. +on: + - repositoriesMatchingQuery: file:README.md + +# In each repository, run this command. Each repository's resulting diff is captured. +steps: + - run: echo Hello World | tee -a $(find -name README.md) + container: alpine:3 + +# Describe the changeset (e.g., GitHub pull request) you want for each repository. +changesetTemplate: + title: Hello World + body: My first campaign! + branch: hello-world # Push the commit to this branch. + commit: + message: Append Hello World to all README.md files + published: false +``` + +1. Create a campaign from the campaign spec by running the following [Sourcegraph CLI (`src`)](https://github.com/sourcegraph/src-cli) command: + +
    src campaign apply -f YOUR_CAMPAIGN_SPEC.campaign.yaml -preview
    + + > **Don't worry!** Before any branches are pushed or changesets (e.g., GitHub pull requests) are created, you will see a preview of all changes and can confirm each one before proceeding. +1. Wait for it to run and compute the changes for each repository (using the repositories and commands in the campaign spec). +1. Open the preview URL that the command printed out. +1. Examine the preview. Confirm that the changes are what you intended. (If not, edit the campaign spec and then rerun the command above.) +1. Click the **Create campaign** button. + +After you've applied a campaign spec, you can [publish changesets](#publishing-changesets-to-the-code-host) to the code host when you're ready. This will turn the patches into commits, branches, and changesets (such as GitHub pull requests) for others to review and merge. + +You can share the link to your campaign with other people if you want their help. Any person on your Sourcegraph instance can [view it in the campaigns list](#viewing-campaigns). + +If a person viewing the campaign lacks read access to a repository in the campaign, they can only see [limited information about the changes to that repository](managing_access.md#repository-permissions-for-campaigns) (and not the repository name, file paths, or diff). + +You can update a campaign's changes at any time, even after you've published changesets. For more information, see "[Updating a campaign](#updating-a-campaign)". + +### Example campaigns + +The [example campaigns](examples/index.md) show how to use campaigns to make useful, real-world changes: + +- [Using ESLint to automatically migrate to a new TypeScript version](examples/eslint_typescript_version.md) +- [Adding a GitHub action to upload LSIF data to Sourcegraph](examples/lsif_action.md) +- [Refactoring Go code using Comby](examples/refactor_go_comby.md) + +## Publishing changesets to the code host + +After you've added patches, you can see a preview of the changesets (e.g., GitHub pull requests) that will be created from the patches. Publishing the changesets will, for each repository: + +- Create a commit with the changes (from the patches for that repository) +- Push a branch (using the branch name you chose when creating the campaign) +- Create a changeset (e.g., GitHub pull request) on the code host for review and merging + +When you're ready, you can publish some or all of a campaign's changesets. + +> TODO(sqs): add steps for updating campaign spec's `changesetTemplate` to publish + +You'll see a progress indicator when changesets are being published. Any errors will be shown, and you can retry publishing after you've resolved the problem. You don't need to worry about it creating multiple branches or pull requests when you retry, because it uses the same branch name. + +To publish a changeset, you need admin access to the campaign and write access to the changeset's repository (on the code host). For more information, see "[Code host interactions in campaigns](managing_access.md#code-host-interactions-in-campaigns)". [Forking the repository](#known-issues) is not yet supported. + +## Tracking campaign progress and changeset statuses + +A campaign tracks all of its changesets for updates to: + +- Status: open, merged, or closed +- Checks: passed (green), failed (red), or pending (yellow) +- Review status: approved, changes requested, pending, or other statuses (depending on your code host or code review tool) + +You can see the overall trend of a campaign in the burndown chart, which shows the proportion of changesets that have been merged over time since the campaign was created. + +> TODO(sqs) screenshot + +In the list of changesets, you can see the detailed status for each changeset. + +> TODO(sqs) screenshot + +If you lack read access to a repository, you can only see [limited information about the changes to that repository](managing_access.md#repository-permissions-for-campaigns) (and not the repository name, file paths, or diff). + +## Updating a campaign + + + +You can edit a campaign's name, description, and any other part of its campaign spec at any time. + +To update a campaign, you need [admin access to the campaign](managing_access.md#campaign-access-for-each-permission-level), and [write access to all affected repositories](managing_access.md#repository-permissions-for-campaigns) with published changesets. + +1. In your terminal, run the [Sourcegraph CLI (`src`)](https://github.com/sourcegraph/src-cli) command shown. The command will execute your campaign spec to generate changes and then upload them to the campaign for you to preview and accept. + +
    src campaign apply -f YOUR_CAMPAIGN_SPEC.campaign.yaml -preview
    + + > **Don't worry!** Before any branches or changesets are modified, you will see a preview of all changes and can confirm before proceeding. +1. Open the preview URL that the command printed out. +1. Examine the preview. Confirm that the changes are what you intended. (If not, edit your campaign spec and then rerun the command above.) +1. Click the **Update campaign** button. + +All of the changesets on your code host will be updated to the desired state that was shown in the preview. + +## Tracking existing changesets + + + +1. Click the Campaigns icon campaigns icon in the top navigation bar. +1. *To use an existing campaign:* In the list of campaigns, click the campaign where you'd like to track existing changesets. + + *To create a new campaign:* Click the **+ New campaign** button. For more information, see "[Creating a new campaign](#creating-a-new-campaign)". +1. Click the **Track existing changeset** button in the top right of the **Changesets** list. +1. Type in the name of the changeset's repository. + + This is the repository's name on Sourcegraph. If you can visit the repository at `https://sourcegraph.example.com/foo/bar`, the name is `foo/bar`. Depending on the configuration, it may or may not begin with a hostname (such as `github.com/foo/bar`). +1. Type in the changeset number (e.g., the GitHub pull request number). +1. Click **Add**. + +You'll see the existing changeset in the list. The campaign will track the changeset's status and include it in the overall campaign progress (in the same way as if it had been created by the campaign). For more information, see "[Tracking campaign progress and changeset statuses](#tracking-campaign-progress-and-changeset-statuses)". + +## Closing or deleting a campaign + +You can close a campaign when you don't need it anymore, when all changes have been merged, or when you decided not to proceed with making all of the changes. A closed campaign still appears in the [campaigns list](#viewing-campaigns). To completely remove it, you can delete the campaign. + +Any person with [admin access to the campaign](managing_access.md#permission-levels-for-campaigns) can close or delete it. + +1. Click the Campaigns icon campaigns icon in the top navigation bar. +1. In the list of campaigns, click the campaign that you'd like to close or delete. +1. In the top right, click the **Close**. +1. Select whether you want to close all of the campaign's changesets (e.g., closing all associated GitHub pull requests on the code host). +1. Click **TODO(sqs)** . + +## [Managing access to campaigns](managing_access.md) + +See "[Managing access to campaigns](managing_access.md)". + +## Code host and repository permissions in campaigns + +All actions on the code host (such as pushing a branch or opening a changeset) are performed by your individual user account, not by a bot user. For more information, see "[Code host interactions in campaigns](managing_access.md#code-host-interactions-in-campaigns)". + +[Repository permissions](../../admin/repo/permissions.md) are enforced when campaigns display information. For more information, see "[Repository permissions in campaigns](managing_access.md#repository-permissions-for-campaigns)". + +## Site admin configuration for campaigns + +Using campaigns requires a [code host connection](../../admin/external_service/index.md) to a supported code host (currently GitHub and Bitbucket Server). + +Site admins can also: + +- [Allow users to authenticate via the code host](../../admin/auth/index.md#github), which makes it easier for users to authorize [code host interactions in campaigns](managing_access.md#code-host-interactions-in-campaigns) +- [Configure repository permissions](../../admin/repo/permissions.md), which campaigns will respect +- [Disable campaigns for all users](managing_access.md#disabling-campaigns-for-all-users) + +## Concepts + +- A **campaign** is group of related changes to code, along with a title and description. +- The campaign has associated **changesets**, which is a generic term for pull requests, merge requests, or any other reviewable chunk of code. (Code hosts use different terms for this, which is why we chose a generic term.) +- A **published changeset** means the commit, branch, and changeset have been created on the code host. An **unpublished changeset** is just a preview that you can view in the campaign but does not exist on the code host yet. +- A **spec** (campaign spec or changeset spec) is a "record of intent". When you provide a spec for a thing, the system will continuously try to reconcile the actual thing with your desired intent (as described by the spec). This involves creating, updating, and deleting things as needed. +- {#campaign-spec} A **campaign spec** is a YAML file describing the campaign: repositories to change, commands to run, and a template for changesets and commits. You describe your high-level intent in the campaign spec, such as "lint files in all repositories with a `package.json` file". +- A campaign has many **changeset specs**, which are produced by executing the campaign spec (i.e., running the commands on each selected repository) and then using its changeset template to produce a list of changesets, including the diffs, commit messages, changeset title, and changeset body. You don't need to view or edit the raw changeset specs; you will edit the campaign spec and view the changesets in the UI. +- The **campaign controller** reconciles the actual state of the campaign's changesets on the code host so that they match your desired intent (as described in the changeset specs). + +To learn about the internals of campaigns, see "[Campaigns](../../dev/campaigns_development.md)" in the developer documentation. + +## Roadmap + + + +### Known issues + + + +- The only supported code hosts are GitHub and Bitbucket Server. Support for [all other code hosts](../../admin/external_service/index.md) is planned. +- It is not yet possible for a campaign to have multiple changesets in a single repository (e.g., to make changes to multiple subtrees in a monorepo). +- Forking a repository and creating a pull request on the fork is not yet supported. Because of this limitation, you need write access to each repository that your campaign will change (in order to push a branch to it). +- Campaign steps are run locally (in the [Sourcegraph CLI](https://github.com/sourcegraph/src-cli)). Sourcegraph does not yet support executing campaign steps (which can be arbitrary commands) on the server. For this reason, the APIs for creating and updating a campaign require you to upload all of the changeset specs (which are produced by executing the campaign spec locally). {#server-execution} -Campaigns currently only support **GitHub** and **Bitbucket Server** repositories. If you're interested in using campaigns on other code hosts, [let us know](https://about.sourcegraph.com/contact). diff --git a/doc/user/campaigns/managing_access.md b/doc/user/campaigns/managing_access.md new file mode 100644 index 000000000000..869d5ac50ac0 --- /dev/null +++ b/doc/user/campaigns/managing_access.md @@ -0,0 +1,81 @@ +# Managing access to campaigns + +You can customize access to a campaign and propose changes to repositories with varying permission levels. Other people see the campaign's proposed changes to a repository if they can view that repository; otherwise, they can see only limited, non-identifying information about the change. + +## Permission levels for campaigns + +Any person with a user account can create a campaign. + +The permission levels for a campaign are: + +- **Read:** For people who need to view the campaign. +- **Admin:** For people who need full access to the campaign, including editing, closing, and deleting it. + +To see the campaign's proposed changes on a repository, a person *also* needs read access to that specific repository. Read or admin access on the campaign does not (by itself) entitle a person to viewing all of the campaign's changes. For more information, see "[Repository permissions for campaigns](#repository-permissions-for-campaigns)". + +Site admins have admin permissions on all campaigns. + +### Campaign access for each permission level + +Campaign action | Read | Admin +--------------- | :--: | :----: +View campaign name and description
    Also applies to viewing the input branch name, created/updated date, and campaign status | ⬤ | ⬤ +View burndown chart (aggregate changeset statuses over time) | ⬤ | ⬤ +View list of patches and changesets | ⬤ | ⬤ +View diffstat (aggregate count of added/changed/deleted lines) | ⬤ | ⬤ +View error messages (related to creating or syncing changesets) | | ⬤ +Edit campaign name, description, and branch name | ⬤ | ⬤ +Update campaign patches (and changesets on code hosts) | | ⬤ +Publish changesets to code host | | ⬤ +Add/remove existing changesets to/from campaign | | ⬤ +Refresh changeset statuses | | ⬤ +Close campaign | | ⬤ +Delete campaign | | ⬤ + +Authorization for all actions is also subject to [repository permissions](#repository-permissions-for-campaigns). + +## Setting and viewing permissions for a campaign + +When you create a campaign, you are given admin permissions on the campaign. + +All users are automatically given read permissions to a campaign. Granular permissions are not yet supported. Assigning admin permissions on a campaign to another person or to all organization members is not yet supported. Transferring ownership of a campaign is not yet supported. + +## Code host interactions in campaigns + +All interactions with the code host are performed by your individual user account on the code host, not by a bot user or Sourcegraph machine account. These operations include: + +- Pushing a branch with the changes (the Git author and committer will be you, and the Git push will be authenticated with your credentials) +- Creating a changeset (e.g., on GitHub, the pull request author will be you) +- Updating a changeset +- Closing a changeset + +If you attempt to perform these operations and haven't yet linked your code host account to your Sourcegraph account, you'll be prompted to do so. You can either: + +- Manually enter a personal access token in your user profile +- Sign in via your code host (currently only supported for GitHub; requires a site admin to [configure GitHub user authentication](../../admin/auth/index.md#github)) + +## Repository permissions for campaigns + +Your [repository permissions](../../admin/repo/permissions.md) determine what information in a campaign you can view. You can only see a campaign's proposed changes to a repository if you have read access to that repository. Read or admin permissions on a campaign does not (by itself) permit you to view all of the campaign's changes. + +When you view a campaign, you can see a list of patches and changesets. For each patch and changeset: + +- **If you have read access** to the repository for a patch or changeset, you can see the diff, changeset title, changeset link, detailed status, and other information. +- **If you do not have read access** to the repository for a patch or changeset, you can only see the status, last-updated date, and whether an error occurred (but not the error message). You can't see the diff, changeset title, changeset link, repository name, or any other information. + +When you perform any campaign operation that involves repositories or code host interaction, your current repository permissions are taken into account. + +- Creating, updating, or publishing a campaign; or publishing a single changeset: You must have access to push a branch to the repositories and create the changesets on your code host (e.g., push branches to and open pull requests on the GitHub repositories). +- Adding existing changesets to a campaign: You must have read access to the existing changesets' repository. +- Closing or deleting a campaign: If you choose to also close associated changesets on the code host, you must have access to do so on the code host. If you do not have access to close a changeset on the code host, the changeset will remain in its current state. A person with repository permissions for the remaining changesets can view them and manually close them. + +Your repository permissions can change at any time: + +- If you've already published a changeset to a repository that you no longer have access to, then you won't be able to view its details or update it in your campaign. The changeset on the code host will remain in its current state. A person with permissions to the changeset on the code host will need to manually manage or close it. When possible, you'll be informed of this when updating a campaign that contains changesets you've lost access to. +- You need access to all repositories mentioned in a campaign plan to use it when updating a campaign. + +If you are not permitted to view a repository on Sourcegraph, then you won't be able to perform any operations on it, even if you are authorized on the code host. + +## Disabling campaigns for all users + +A site admin can completely disable campaigns on an instance by setting the [site configuration](../../admin/config/site_config.md) property `campaigns.disable` to `true`. This setting applies to everyone, including site admins. It prevents all viewing, creation, editing, syncing, and other actions on campaigns. Existing campaign changesets on code hosts are left in their current state and are not affected by this setting. diff --git a/doc/user/campaigns/updating_campaigns.md b/doc/user/campaigns/updating_campaigns.md deleted file mode 100644 index 4256af122992..000000000000 --- a/doc/user/campaigns/updating_campaigns.md +++ /dev/null @@ -1,121 +0,0 @@ -# Updating a campaign - -## Updating campaign attributes - -In order to update the title and the description of a campaign, simply click the **Edit** button on the top right when viewing a campaign. You can then update both attributes. The branch name of a campaign can also be edited, but only if the campaign was created from a patch set and doesn't contain any published changesets. - -When you click on **Save** and the campaign will be updated. - -If the campaign was created from a patch set and includes changesets that have already been created on the code host, the title and description of those changesets will be updated on the code host, too. - -## Updating the patch set of a campaign - -You can also apply a new patch set to an existing campaign and update its patches and, if already created, the diff of the changesets on the code hosts. - -1. Use the [`src` CLI](https://github.com/sourcegraph/src-cli) to create a new patch set that reflects the desired state of all patches/changesets in the campaign: - - ``` - $ src action exec -f new-action-definition.json -create-patchset - - # Or: - - $ src action exec -f new-action-definition.json | src campaign patchset create-from-patches - ``` - - For example, the `new-action-definition.json` could have a `"scopeQuery"` that yields _more_ repositories and thus produces _more_ patches. - -2. Following the creation of the patch set with one of the two commands above, a URL will be printed that will guide you to the Sourcegraph web UI: - -
    -
    -
    - -
    -
    -
    - -3. In the UI you can then select which campaign should be updated to use the new patch set. - - Click **Preview** to select the campaign that should be updated. - -
    -
    -
    - -
    -
    Select which campaign should be updated to use the new patch set.
    -
    -
    - -4. You'll then see which changesets will be created, updated (on the code host), deleted and removed from the campaign or left untouched: - -
    -
    -
    - -
    -
    Preview of updating a campaign's patch set.
    -
    -
    - -5. Click on **Update**. - -The patches and changesets in a campaign will be like this: - -* Patches (unpublished changesets) will be updated if their diff has changed. -* Published changesets will be updated on the code host if their diff or the campaign's title or description has changed. -* Published changesets will be closed on the code host and detached from the campaign if the new patch set doesn't contain a patch for their repositories. -* Published changesets will be left untouched if the new patch set contain the exact same patch for their repositories and the campaigns title and description have not been changed. -* Published changesets that are already merged or closed will not be updated and kept attached to the campaign. If a the patch set contains an new patch for a repository for which the campaign already has a merged changeset, a new changeset will be created. - -### Example: Extending the scope of an campaign - -A common reason for updating campaigns is to widen or narrow their scope, wanting more or fewer changesets to be created on a code host. In order to do that, one needs to update the patch set of an existing campaign with a patch set that contains the desired amount of patches. - -Say you have successfully created a campaign based on the patches yielded by running the following action: - -```json -{ - "scopeQuery": "repo:github.com/sourcegraph/sourcegraph$", - "steps": [ - { - "type": "docker", - "image": "golang:1.14-alpine", - "args": ["sh", "-c", "cd /work && go fmt ./..."] - } - ] -} -``` - -The campaign now has a single changeset, for the `github.com/sourcegraph/sourcegraph` repository, which was the only repository yielded by the `"scopeQuery"` above. - -Now you want to run the same action over more repositories and extend the existing campaign by creating additional changesets on the code hosts. - -To do that, first change the action definition's `"scopeQuery"` to yield more repositories: - -```json -{ - "scopeQuery": "repo:github.com/sourcegraph", - "steps": [ - { - "type": "docker", - "image": "golang:1.14-alpine", - "args": ["sh", "-c", "cd /work && go fmt ./..."] - } - ] -} -``` - -This will now run the `go fmt` in every repository in the `github.com/sourcegraph` organization. - -Execute the action and create a new patch set: - -``` -$ src action exec -f extended-action.json -create-patchset -``` - -After the command ran successfully, a URL to continue in the web UI is printed. - -Open it to select which campaign you want to update. Select your existing campaign. The preview now shows you the additional changesets that will be created when you update the campaign and, if something changed in that repository, how the changeset that already exists will be updated. - -Click **Update** to create the additional changesets.