Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Usage: add GitLab, Bitbucket sections #122

Merged
merged 8 commits into from
Oct 26, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion content/docs/ref/pr.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,4 +43,5 @@ else
fi
```

Note: pull requests created with `cml pr` **won't** trigger a new CI/CD run under any circumstances.
Note: pull requests created with `cml pr` **won't** trigger a new CI/CD run
under any circumstances.
10 changes: 5 additions & 5 deletions content/docs/self-hosted-runners.md
Original file line number Diff line number Diff line change
Expand Up @@ -238,12 +238,12 @@ Use either:
For instance, to use a personal access token:

1. Navigate to **User Settings** → **Access Tokens**
- in the "Name" field, type `repo_token`
- in the "Name" field, type `REPO_TOKEN`
- select `api`, `read_repository` and `write_repository`
- click "Create personal access token" and copy it
2. In your GitLab project, navigate to **Settings** → **CI/CD**
→ **Variables** → **Add Variable**
- in the "Key" field, type `repo_token`
- in the "Key" field, type `REPO_TOKEN`
- in the "Value" field, paste your Personal Access Token
- select "Mask variable"
- deselect "Protect variable"
Expand All @@ -255,7 +255,7 @@ credentials.
</tab>
<tab title="Bitbucket">

Bitbucket Cloud does not use access tokens. Instead, create a `repo_token`
Bitbucket Cloud does not use access tokens. Instead, create a `REPO_TOKEN`
variable with a Base64 encoded username and password.

Use either:
Expand All @@ -268,15 +268,15 @@ Use either:
intended use case: you limit the account to only access the repositories where
you plan to deploy runners to.

In either case, the steps to create a `repo_token` are:
In either case, the steps to create a `REPO_TOKEN` are:

1. Use a Base64 encoder of your choice to encode a Bitbucket username and
password:
- `echo $USERNAME:$PASSWORD | base64`
- copy the resulting Base64 token
2. In your repository, go to **Repository Settings** &rightarrow; **Repository
Variables**
- in the "Name" field, type `repo_token`
- in the "Name" field, type `REPO_TOKEN`
- in the "Value" field, paste the Base64 token
- select `Secured` to hide credentials in all Bitbucket logs

Expand Down
13 changes: 6 additions & 7 deletions content/docs/start/gitlab.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,16 +61,15 @@ Here, we'll walk through a tutorial to start using CML on GitLab.
into a new file `.gitlab-ci.yml` and save.

```yaml
stages:
- cml_run

cml:
stage: cml_run
train-model:
image: iterativeai/cml:0-dvc2-base1
script:
- pip3 install -r requirements.txt
- pip install -r requirements.txt
- python train.py

create-cml-report:
needs: train-model
image: iterativeai/cml:0-dvc2-base1
script:
- cat metrics.txt >> report.md
- cml publish confusion_matrix.png --md >> report.md
- cml send-comment report.md
Expand Down
9 changes: 0 additions & 9 deletions content/docs/start/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,12 +34,3 @@ _Need help? Just want to chat about continuous integration for ML?
for hands-on MLOps tutorials using CML! 🌟

https://youtu.be/9BgIDqAzfuA

## Case studies

Here are some example projects using CML.

- [Basic CML project](https://github.com/iterative/cml_base_case)
- [CML with DVC to pull data](https://github.com/iterative/cml_dvc_case)
- [CML with Tensorboard](https://github.com/iterative/cml_tensorboard_case)
- [CML with EC2 GPU](https://github.com/iterative/cml_cloud_case)
112 changes: 96 additions & 16 deletions content/docs/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,21 +5,8 @@ A GitLab, GitHub, or Bitbucket account is required. Familiarity with
[GitLab CI/CD](https://about.gitlab.com/stages-devops-lifecycle/continuous-integration)
may also be beneficial.

## GitLab

Please see our docs on
[CML with GitLab CI/CD](https://github.com/iterative/cml/wiki/CML-with-GitLab)
and in particular the
[personal access token](https://github.com/iterative/cml/wiki/CML-with-GitLab#variables)
requirement.

## Bitbucket

Please see our docs on
[CML with Bitbucket Cloud](https://github.com/iterative/cml/wiki/CML-with-Bitbucket-Cloud).
_Bitbucket Server support estimated to arrive by mid 2021._

## GitHub
<toggle>
<tab title="GitHub">

The key file in any CML project is `.github/workflows/cml.yaml`:

Expand All @@ -36,7 +23,7 @@ jobs:
# may need to setup Node.js & Python3 on e.g. self-hosted
# - uses: actions/setup-node@v2
# with:
# node-version: '12'
# node-version: '14'
# - uses: actions/setup-python@v2
# with:
# python-version: '3.x'
Expand All @@ -55,13 +42,106 @@ jobs:
cml send-comment report.md
```

The example above generates visual reports in pull requests:
[![](/img/cml_first_report.png)](https://github.com/iterative/cml_base_case/pull/2)

We helpfully provide CML and other useful libraries pre-installed on our
[custom Docker images](/doc/self-hosted-runners#docker-images). In the above
example, uncommenting the
`container: docker://ghcr.io/iterative/cml:0-dvc2-base1` field will make the
runner pull the CML Docker image. The image already has Node.js, Python 3, DVC
and CML set up on an Ubuntu LTS base for convenience.

### Example projects

- [Basic CML project](https://github.com/iterative/cml_base_case)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should put the fashion case. It's a REAL case 😉
Lets open a ticket to put that repo under iterative. I would probably remove all of the rest after that.

- [CML with DVC to pull data](https://github.com/iterative/cml_dvc_case)
- [CML with Tensorboard](https://github.com/iterative/cml_tensorboard_case)
- [CML with EC2 GPU](https://github.com/iterative/cml_cloud_case)

</tab>
<tab title="GitLab">

The key file in any CML project is `.gitlab-ci.yml`:

```yml
train-model:
# use a convenient Ubuntu LTS + DVC + CML container
image: iterativeai/cml:0-dvc2-base1
script:
- pip install -r requirements.txt
- python train.py
create-CML-report:
needs: train-model
image: iterativeai/cml:0-dvc2-base1
script:
- cat metrics.txt >> report.md
- cml publish confusion_matrix.png --md >> report.md
- cml send-comment report.md
```

⚠️ You _must_ provide a
[personal or project access token (PAT)](/doc/self-hosted-runners#personal-access-token)
via a `REPO_TOKEN` variable.

The example above generates visual reports in merge requests:
[![](/img/GitLab_CML_report.png '=400')](https://gitlab.com/iterative.ai/cml-base-case/-/merge_requests/3)

We helpfully provide CML and other useful libraries pre-installed on our
[custom Docker images](/doc/self-hosted-runners#docker-images). In the above
example, the `image: iterativeai/cml:0-dvc2-base1` field will make the runner
pull the CML Docker image. The image already has Node.js, Python 3, DVC and CML
set up on an Ubuntu LTS base for convenience.

### Example projects

- [Basic CML project](https://gitlab.com/iterative.ai/cml-base-case)
- [CML with DVC to pull data](https://gitlab.com/iterative.ai/cml-dvc-case)
- [CML with Tensorboard](https://gitlab.com/iterative.ai/cml-tensorboard-case)
- [CML with EC2 GPU](https://gitlab.com/iterative.ai/cml-cloud-case)

</tab>
<tab title="Bitbucket">

The key file in any CML project is `bitbucket-pipelines.yml`:

```yaml
image: iterativeai/cml:0-dvc2-base1
pipelines:
default:
- step:
name: Train model
script:
- pip install -r requirements.txt
- python train.py
- step:
name: Create CML report
script:
- cat metrics.txt > report.md
- echo >> report.md
- cml publish confusion_matrix.png --md >> report.md
- cml send-comment report.md
```

⚠️ You _must_ provide
[access credentials](/doc/self-hosted-runners#personal-access-token) via a
`REPO_TOKEN` variable.

The example above generates visual reports in pull requests:
[![](/img/bitbucket_cloud_pr.png '=600')](https://bitbucket.org/iterative-ai/cml-base-case/pull-requests/2)

⚠️ CML works with Bitbucket Cloud, where you can use the
[Bitbucket Pipelines](https://bitbucket.org/product/features/pipelines) CI/CD
system to run workflows automatically on triggering events. Bitbucket Server is
not yet supported.

### Example projects

- [Basic CML project](https://bitbucket.org/iterative-ai/cml-base-case)

</tab>
</toggle>

## CML Commands

CML provides a number of commands to help package the outputs of ML workflows
Expand Down
Binary file added static/img/GitLab_CML_report.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added static/img/bitbucket_cloud_pr.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.