Merge pull request #7 from iterative/add-cml-utils

Add cml utils

packages/example/content/docs/ref/ci.md

@@ -0,0 +1,30 @@
+# Command Reference: `ci`
+
+```usage
+cml ci [--unshallow] [options]
+```
+
+Prepares Git repository for CML operations (setting Git `user.name` &
+`user.email`; fetching all branch tips; undoing CI oddities such as origin URL
+formatting and HTTP remote proxies; and, optionally, unshallowing clone).
+
+## Options
+
+Any [generic option](/doc/ref) in addition to:
+
+- `--unshallow`: Fetch as much as possible, converting a shallow repository to a
+  complete one.
+- `--user-email=<address>`: Git user email for commits [default:
+  `[email protected]`].
+- `--user-name=<...>`: Git user name for commits [default: `Olivaw[bot]`].
+
+## Examples
+
+Instead of wrangling with
+[unshallowing clones](https://stackoverflow.com/q/6802145) and
+`git config user.email` before being able to `git commit` or use
+[`cml pr`](/doc/ref/pr), simply run:
+
+```usage
+cml ci
+```

packages/example/content/docs/ref/index.md

@@ -0,0 +1,16 @@
+# Command Reference
+
+## Generic Options
+
+All CML commands support the following options:
+
+- `--repo=<repo or org>`: Repository (or Organization) to be used [default:
+  *inferred from environment*].
+- `--token=<PAT>`:
+  [Personal/project access token](https://cml.dev/doc/self-hosted-runners#personal-access-token)
+  to be used [default: *inferred from environment*].
+- `--log={error,warn,info,debug}`: Maximum log level [default: `info`].
+- `--driver={github,gitlab,bitbucket}`: CI provider where the repository is
+  hosted [default: *inferred from environment*].
+- `--help`: Show help.
+- `--version`: Show version number.

packages/example/content/docs/ref/pr.md

@@ -0,0 +1,92 @@
+# Command Reference: `pr`
+
+```usage
+cml pr [options] <pathspec>...
+```
+
+Commit specified files to a new branch and create a pull request. If sending a
+report afterwards, consider using `cml send-comment --pr --update`.
+
+ⓘ Pull requests created with `cml pr` **won't** trigger a new CI/CD run, thereby
+preventing an infinite chain of runs.
+
+ⓘ Files to commit can be specified using any syntax supported by
+[Git pathspec](https://git-scm.com/docs/gitglossary#Documentation/gitglossary.txt-aiddefpathspecapathspec).
+
+## Options
+
+Any [generic option](/doc/ref) in addition to:
+
+- `--merge`, `--rebase`, or `--squash`: Try to merge, rebase, or squash-merge
+  the created PR after CI tests pass.
+- `--md`: Produce output in markdown format (`[CML Pull/Merge Request](url)`
+  instead of `url`).
+- `--remote=<name or URL>`: Git remote name or URL [default: `origin`].
+- `--user-email=<address>`: Git user email for commits [default:
+  `[email protected]`].
+- `--user-name=<...>`: Git user name for commits [default: `Olivaw[bot]`].
+
+## Examples
+
+### Commit all files in current working directory
+
+```usage
+cml pr .
+```
+
+### Automatically merge pull requests
+
+```usage
+date > output.txt
+cml pr --auto-merge output.txt
+```
+
+The `--merge`, `--rebase`, and `--squash` options enable
+[auto–merge](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/incorporating-changes-from-a-pull-request/automatically-merging-a-pull-request)
+(GitHub) or
+[merge when pipeline succeeds](https://docs.gitlab.com/ee/user/project/merge_requests/merge_when_pipeline_succeeds.html)
+(GitLab) to merge the pull request as soon as checks succeed. If waiting for
+checks isn't supported, `cml pr` will try to merge the pull request immediately.
+
+## Command internals
+
+```usage
+cml pr "**/*.py" "**/*.json"
+```
+
+is roughly equivalent to:
+
+```bash
+SHA="$(git log -n1 --format=%h)"
+BASE="$(git branch)"
+
+git checkout "${BASE}-cml-pr-${SHA}"
+
+if [[ $(git ls-remote --exit-code origin\
+        "${BASE}-cml-pr-${SHA}" &>/dev/null) ]]; then
+  # branch already exists; just print its PR URL
+  curl \
+    -H "Accept: application/vnd.github.v3+json" \
+    https://api.github.com/repos/${GITHUB_REPOSITORY}/pulls \
+    | jq -r ".[] | select(.head.ref == '${BASE}-cml-pr-${SHA}') | .url"
+else
+  # create branch & PR
+  git checkout -b "${BASE}-cml-pr-${SHA}"
+  git add "**/*.py" "**/*.json"
+  git commit -m "CML PR for ${SHA} [skip ci]"
+  git push
+  curl \
+    -X POST \
+    -H "Accept: application/vnd.github.v3+json" \
+    https://api.github.com/repos/${GITHUB_REPOSITORY}/pulls \
+    -d "{
+      \"head\": \"${BASE}-cml-pr-${SHA}\",
+      \"base\": \"${BASE}\",
+      \"title\": \"CML PR for ${BASE} ${SHA}\",
+      \"description\":
+        \"Automated commits for\
+          ${GITHUB_REPOSITORY}/commit/${SHA} created by CML.\"
+    }" \
+    | jq -r .url
+fi
+```

packages/example/content/docs/ref/publish.md

@@ -0,0 +1,29 @@
+# Command Reference: `publish`
+
+```usage
+cml publish [options] <image file>
+```
+
+Publicly host an image for displaying in a CML report.
+
+## Options
+
+Any [generic option](/doc/ref) in addition to:
+
+- `--md`: Produce output in markdown format.
+- `-t=<...>`, `--title=<...>`: Title for markdown output.
+- `--mime-type=<...>`: Content
+  [MIME type](https://www.iana.org/assignments/media-types/media-types.xhtml)
+  [default: *inferred from content*].
+- `--native`: Uses CI provider's native storage instead of CML's.
+  [Not available on GitHub](https://github.com/iterative/cml/wiki/Backend-Supported-Features).
+- `--rm-watermark`: Don't inject a watermark into the comment. Will break some
+  CML functionality which needs to distinguish CML reports from other comments.
+
+## Examples
+
+To render an image in a markdown file:
+
+```usage
+cml publish ./plot.png --md >> report.md
+```

packages/example/content/docs/ref/runner.md

@@ -0,0 +1,101 @@
+# Command Reference: `runner`
+
+```usage
+cml runner [options]
+```
+
+Starts a [runner](/doc/self-hosted-runners) (either via any supported cloud
+compute provider or locally on-premise).
+
+## Options
+
+Any [generic option](/doc/ref) in addition to:
+
+- `--labels=<...>`: One or more (comma-delimited) labels for this runner
+  [default: `cml`].
+- `--name=<...>`: Runner name displayed in the CI [default: `cml-{ID}`].
+- `--idle-timeout=<seconds>`: Seconds to wait for jobs before terminating. Set
+  to `-1` to disable timeout [default: `300`].
+- `--no-retry`: Don't restart the workflow when terminated due to instance
+  disposal or
+  [GitHub Actions timeout](https://docs.github.com/en/actions/hosting-your-own-runners/about-self-hosted-runners#usage-limits).
+- `--single`: Terminate runner after one workflow run.
+- `--reuse`: Don't launch a new runner if an existing one has the same name or
+  overlapping labels. If an existing matching (same name or overlapping labels)
+  instance is busy, it'll
+  [still be reused](https://github.com/iterative/cml/issues/610).
+- `--cloud={aws,azure,gcp,kubernetes}`: Cloud compute provider to host the
+  runner.
+- `--cloud-type={m,l,xl,m+k80,m+v100,...}`: Instance
+  [type](https://registry.terraform.io/providers/iterative/iterative/latest/docs/resources/task#machine-type).
+  Also accepts native types such as `t2.micro`.
+- `--cloud-gpu={nogpu,k80,v100,tesla}`: GPU type.
+- `--cloud-hdd-size=<...>`: Disk storage in GB.
+- `--cloud-spot`: Request a preemptible spot instance.
+- `--cloud-spot-price=<...>`: Maximum spot instance USD bidding price, [default:
+  *current price*].
+- `--cloud-region={us-west,us-east,eu-west,eu-north,...}`:
+  [Region](https://registry.terraform.io/providers/iterative/iterative/latest/docs/resources/task#cloud-regions)
+  where the instance is deployed. Also accepts native AWS/Azure region or GCP
+  zone [default: `us-west`].
+- `--cloud-permission-set=<...>`:
+  [AWS instance profile](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html#ec2-instance-profile)
+  or
+  [GCP instance service account](https://cloud.google.com/compute/docs/access/service-accounts).
+- `--cloud-metadata=<...>`: `key=value` pair to associate with cloud runner
+  instances. May be [specified multiple times](http://yargs.js.org/docs/#array).
+- `--cloud-startup-script=<...>`: Run the provided
+  [Base64](https://linux.die.net/man/1/base64)-encoded Linux shell script during
+  the instance initialization.
+- `--cloud-ssh-private=<key>`: Private SSH RSA key [default: *auto-generate
+  throwaway key*]. Only supported on AWS and Azure; intended for debugging
+  purposes.
+- `--cloud-aws-security-group=<...>`:
+  [AWS security group](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html)
+  identifier.
+- `--cloud-aws-subnet=<...>`:
+  [AWS subnet](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Subnets.html#subnet-basics)
+  identifier.
+- `--docker-volumes=<...>`: Volume mount to pass to Docker, e.g.
+  `/var/run/docker.sock:/var/run/docker.sock` for Docker-in-Docker support. May
+  be specified multiple times. Only supported by GitLab.
+
+## FAQs and Known Issues
+
+- Bitbucket: Support for
+  [self-hosted runners for Bitbucket Pipelines](https://support.atlassian.com/bitbucket-cloud/docs/runners)
+  is [coming soon](https://github.com/iterative/cml/pull/798).
+- GitHub Actions by default timeout after a few hours. You can request up to
+  [72 hours](https://docs.github.com/en/actions/hosting-your-own-runners/about-self-hosted-runners#usage-limits)
+  via
+  [`timeout-minutes: 4320`](https://docs.github.com/en/actions/learn-github-actions/workflow-syntax-for-github-actions#jobsjob_idtimeout-minutes).
+  CML will helpfully restart GitHub Actions workflows approaching 72 hours
+  (you'd need to write your code to save intermediate results to take advantage
+  of this).
+
+## Examples
+
+### Using `--cloud-ssh-private`
+
+1. Generate a new RSA PEM private key for debugging purposes:
+
+   ```bash
+   ssh-keygen -t rsa -m pem -b 4096 -f key.pem
+   ```
+
+2. Pass the contents of the generated private key file when invoking the
+   `cml runner` command:
+
+   ```bash
+   cml runner --cloud=... --cloud-ssh-private="$(cat key.pem)"
+   ```
+
+3. Access the instance from your local system by using the generated key as an
+   identity file:
+
+   ```bash
+   ssh -i key.pem ubuntu@IP_ADDRESS
+   ```
+
+   replacing the `IP_ADDRESS` placeholder with the instance address returned by
+   `cml runner` (search the output logs for `instanceIp`).

packages/example/content/docs/ref/send-comment.md

@@ -0,0 +1,47 @@
+# Command Reference: `send-comment`
+
+```usage
+cml send-comment [options] <markdown report file>
+```
+
+Post a markdown report as a comment on a commit or pull/merge request.
+
+ⓘ If there's an associated pull/merge request, consider adding the `--pr` and
+`--update` flags.
+
+ⓘ If `cml pr` was used earlier in the workflow, use `--commit-sha=HEAD` to post
+comments to the new PR if desired.
+
+## Options
+
+Any [generic option](/doc/ref) in addition to:
+
+- `--commit-sha=<rev>`, `--head-sha=<rev>`:
+  [Git revision](https://git-scm.com/docs/gitrevisions) linked to this comment
+  [default: `HEAD`].
+- `--pr`: Post to an existing PR/MR associated with the specified commit.
+- `--update`: Update the last CML comment (if any) instead of creating a new
+  one.
+- `--rm-watermark`: Don't inject a watermark into the comment. Will break some
+  CML functionality (such as `--update`) which needs to distinguish CML reports
+  from other comments.
+
+## FAQs and Known Issues
+
+### Bitbucket
+
+- **Can't create a pull request or commit comment** / **Invalid or unknown
+  installation**.
+
+  This happens because the Pull Request Commit Links application has not been
+  installed into your BitBucket workspace. You can install it by following these
+  instructions from the [Bitbucket docs][bb-docs-install-pr-links]:
+
+  > Pull Request Commit Links app must be installed first before using this API;
+  > installation automatically occurs when 'Go to pull request' is clicked from
+  > the web interface for a commit's details.
+
+  We don't like ClickOps either, but it's the way it is.
+
+[bb-docs-install-pr-links]:
+  https://developer.atlassian.com/bitbucket/api/2/reference/resource/repositories/%7Bworkspace%7D/%7Brepo_slug%7D/commit/%7Bcommit%7D/pullrequests

packages/example/content/docs/ref/send-github-check.md

@@ -0,0 +1,19 @@
+# Command Reference: `send-github-check`
+
+```usage
+cml send-github-check [options] <markdown report file>
+```
+
+Similar to [`send-comment`](/doc/ref/send-comment), but using GitHub's
+[checks interface](https://docs.github.com/en/rest/reference/checks).
+
+## Options
+
+Any [generic option](/doc/ref) in addition to:
+
+- `--commit-sha=<ref>`, `--head-sha=<ref>`:
+  [Git revision](https://git-scm.com/docs/gitrevisions) linked to this check
+  [default: `HEAD`].
+- `--title=<...>`: The check's title [default: `CML Report`].
+- `--conclusion={success,failure,neutral,cancelled,skipped,timed_out}`: The
+  check's status [default: `success`].

packages/example/content/docs/ref/tensorboard-dev.md

@@ -0,0 +1,30 @@
+# Command Reference: `tensorboard-dev`
+
+```usage
+cml tensorboard-dev [options]
+```
+
+Return a link to a <https://tensorboard.dev> page.
+
+## Options
+
+Any [generic option](/doc/ref) in addition to:
+
+- `-c=<file>`, `--credentials=<file>`: TensorBoard JSON credentials (usually
+  found at `~/.config/tensorboard/credentials/uploader-creds.json`) [default:
+  *inferred from environment `TB_CREDENTIALS`*].
+- `--logdir=<path>`: Directory containing the logs to process.
+- `--name=<...>`: TensorBoard experiment title (up to 100 characters).
+- `--description=<...>`: TensorBoard experiment description (markdown format, up
+  to 600 characters).
+- `--md`: Produce output in markdown format (`[title](url)`).
+- `-t=<...>`, `--title=<...>`: Title for markdown output [default: *value of
+  `--name`*].
+- `--rm-watermark`: Don't inject a watermark into the comment. Will break some
+  CML functionality which needs to distinguish CML reports from other comments.
+
+## Examples
+
+```usage
+cml tensorboard-dev --logdir=./logs --title=Training --md >> report.md
+```