Skip to content

Latest commit

 

History

History
648 lines (547 loc) · 19.2 KB

README.md

File metadata and controls

648 lines (547 loc) · 19.2 KB

Home

Jobs

Easily author and add Parameterized Jobs to the src/jobs directory.

Each YAML file within this directory will be treated as an orb job, with a name which matches its filename.

Jobs may invoke orb commands and other steps to fully automate tasks with minimal user configuration.

Quality Gate Job

Name: quality_gate_job

Generic quality gate job

Parameters:

  • resource_class: circleci resource_class, default is medium.
  • executor: circleci executor
  • cache_name: Not required. If not specified cache will not be used/created.
  • cache_path: Not required. If not specified cache will not be created.
  • quality_checks: is a list of steps to run. The default is: install_build_test

For applications that already have defined Makefile and above defined targets usage is quite simple. Otherwise there are two paths to add custom steps to configuration or to add Makefile(recommended).

Examples

In this example you configure:

  • user defined executor
  • custom name
  • custom steps
# ...
jobs:
  # ...
- ric-orb/quality_gate_job:
    executor: cypress-with-gcloud
    name: integration-test
    context: dev
    quality_checks:
      - ric-orb/auth_gke
      - run:
          name: Install newman
          command: npm install newman --global
      - ric-orb/gke_port_forward:
          context: $GKE_CLUSTER_NAME
          port: 8080
          namespace: payment
          target: svc/mysvc-api-svc
      - run:
          name: Run integration tests
          command: make test-integration
    requires:
      - deploy_dev

In this example you configure default settings, and only required parameters are added:

# ...
jobs:
  # ...
  - ric-orb/quality_gate_job:
      executor: go
      context: dev

Cache example:

# ...
jobs:
  # ...
  - ric-orb/quality_gate_job:
      executor: go
      context: dev
      cache_name: dependency-cache-{{ checksum "go.sum" }}
      cache_path: "$GOPATH/pkg/mod"

Docker build & push (Docker Image) Job

Name: docker_build_push

Parameters:

  • image_name: The Docker image name (without the path, e.g. search-solr)
  • path: Path to directory containing dockerfile (also working directory). Default is "."
  • docker_hub_username: Docker hub credentials. Default is context variable $DOCKER_HUB_USERNAME
  • docker_hub_password: Docker hub credentials. Default is context variable $DOCKER_HUB_PASSWORD
  • docker_version: Default is '' (which defaults to CircleCI's default)
  • docker_layer_caching: for enabling docker layer caching. Default is true

This job builds and pushes Docker image to private Docker hub. It uses docker for building and pushing.

Examples

Minimal

#...
jobs:
  # ...
  - ric-orb/docker_build_push:
      image_name: my_image
#...

For a specific (maven) module:

#...
jobs:
  # ...
  - ric-orb/docker_build_push:
      image_name: my_image
      path: my_maven_module
#...

Isopod build & push Docker Image Job

Name: build_push_image (TODO: should be called isopod_build_push, but that would be a breaking change)

Parameters:

  • path: Path to directory containing isopod.yml file. NOTE: this directory will be the working directory for isopod (i.e. dockerfile is also expected there, and should be executable directly from that directory). Default is .
  • isopod_config: Name of the isopod config file. Default is isopod.yml
  • isopod_version: isopod version in executor. Not required. Default is latest
  • docker_hub_username: username for public docker registry(Docker Hub), default is value of context variable $DOCKER_HUB_USERNAME
  • docker_hub_password: password for public docker registry(Docker Hub), default is value of context variable $DOCKER_HUB_PASSWORD
  • cache_name: Not required. If not specified cache will not be used/created
  • maven_credentials: environment/context variable name (just the name, not the actual variable!) which holds base64 encoded content for .m2/settings.xml file. Default is ARTIFACTORY_MAVEN_CREDENTIALS which already defined in our contexts
  • npm_credentials: environment/context variable name (just the name, not the actual variable!) which holds base64 encoded content for .npmrc file. Default is NPM_RC which already defined in our contexts
  • docker_version: see docs. Default is '' (which defaults to CircleCI's default)
  • docker_layer_caching: for enabling docker layer caching. Default is true
  • prebuild_steps: the list of steps that are executed to prepare building image/application. Default is none
  • postbuild_steps: the list of steps that are executed after building image/application. Default is none
  • docker_template: The ricardo-orbs has some "templates" for standard Dockerfiles for some languages. If you want to use one, set the corresponding enum value here (see the code for the values currently available). Note that if you want to use a Dockerfile from a template, you also need to remove the Dockerfile from your repository and add Dockerfile to the repo's .gitignore, to prevent Isopod from throwing a "working directory not clean" error.
  • repository_version: The version of the ricardo-orbs from which to get the Dockerfiles, it can be a tag or a branch. Only makes sense if docker_template is used.

This job builds docker image and pushes the image to the private docker registry. Uses *isopod *****executor from orb. Attaches root workspace to access existing build outputs.

Examples

Example with:

  • custom pre-build steps
  • disabling docker_layer_caching
# ...
jobs:
  # ...
  - ric-orb/build_push_image:
      name: docker
      prebuild_steps:
        - run:
            name: Copy env variable to .env file (to be used in Docker image by Sentry Webpack plugin)
            command: printenv > .env.sentry
      docker_layer_caching: false
      context: dev
      requires:
        - maven_build_test

Example with all default:

# ...
jobs:
  # ...
  - ric-orb/build_push_image:
      context: dev
      requires:
        - quality-gate

Example with custom properties:

# ...
jobs:
  # ...
  - ric-orb/build_push_image:
      context: dev
      isopod_version: "0.31.2"
      docker_version: "20.10.17"
      isopod_config: isopod.yaml
      docker_hub_username: ${MY_DOCKER_HUB_USERNAME}
      docker_hub_password: ${MY_DOCKER_HUB_PASSWORD}
      private_hub_url: ${PDOCKER_REGISTRY_URL}
      cache_name: dependency-cache-{{ checksum "go.sum" }}
      requires:
        - maven_build_test

Example for building docker image from java build outputs (as saved to workspace by maven_build_test job) for a specific app from a monorepo:

# ...
jobs:
  # ...
  - ric-orb/build_push_image:
      context: dev
      path: "myapp"
      isopod_version: "0.31.2"
      docker_version: "20.10.17"
      requires:
        - maven_build_test

Isodpod deploy Job

Name: deploy_job (TODO: should be called isopod_deploy, but that would be a breaking change)

Parameters:

  • path: Path to directory containing isopod.yml file. Default is "."
  • isopod_config: Name of the isopod config file. Default is isopod.yml
  • isopod_version: isopod version in executor. Not required. Default is latest
  • env: environment for which deployment is executed. Values: prod, dev. Default is dev
  • predeploy_steps: steps that are executed to prepare deployment. Default is none
  • postdeploy_steps: steps that are executed after deployment. Default is none
  • slack_notify_success: flag to send slack message if job is successful. Default is false
  • slack_ok_webhook: the slack webhook used to send slack message on success. Default is value from context variable $SLACK_WEBHOOK
  • slack_ok_branches: comma separated list of branches for which successful slack message will be sent. Default is master,main
  • slack_notify_failure: flag to send slack message when job fails. Default is false
  • slack_fail_webhook: the slack webhook used to send slack message on failure. Default is value from context variable $SLACK_WEBHOOK
  • slack_fail_branches: comma separated list of branches for which failure slack message will be sent. Default is master,main
  • work_dir: working directory for default deployment (with the isopod). This should be relative to the job working directory. Default is .

This job executes deployment on GKE. It can send Slack message on success/failure.

Examples:

Deployment to prod with specific isopod version:

# ...
jobs:
  # ...
  - ric-orb/deploy_job:
      isopod_version: 0.31.2
      env: prod
      context: prod
      requires:
          - integration-test

Deployment with Slack notifications:

# ...
jobs:
  # ...
  - ric-orb/deploy_job:
      env: prod
      context: prod
      requires:
        - deploy_dev
      slack_notify_success: true
      slack_notify_failure: true

Slack failure on team channel:

# ...
jobs:
  # ...
  - ric-orb/deploy_job:
      env: prod
      context: prod
      requires:
        - deploy_dev
      slack_notify_failure: true
      slack_fail_webhook: http://...ddd.d.

Custom steps:

# ...
jobs:
  # ...
  - ric-orb/deploy_job:
      env: dev
      context: dev
      predeploy_steps:
        - run:
            command: echo hello world
      postdeploy_steps:
        - run:
            command: echo adieu world
      requires:
        - docker

Deploys java image for a specific app with a specific isopod config file from a monorepo:

# ...
jobs:
  # ...
  - ric-orb/deploy_job:
      context: dev
      path: "myapp"
      isopod_config: "isopod.yml"
      isopod_version: "0.31.2"
      env: "dev"
      requires:
        - build_push_image

Push build to bucket Job

Name: push_build_to_bucket_job

Parameters:

  • isopod_version: isopod version in executor. Not required. Default is latest
  • bucket_name: name of the bucket where assets will be pushed, default is value of context variable GCLOUD_WEB_ASSETS_BUCKET_NAME
  • bucket_path: bucket's path where assets will be pushed, default is /static-assets
  • app_name: this is a folder name for application's assets, the convention is to use a github repo name
  • source: path to source of assets

Examples

All default

This will push static assets from ./build/assets into production bucket ricardo-web-assets/static-assets/my-ricardo-spa using gsutil rsync. the latest isopod image will be used as executor.

# ...
jobs:
  # ...
  - ric-orb/push_build_to_bucket_job:
      context: prod
      source: ./build/assets
      app_name: my-ricardo-spa

Custom isopod version and custom bucket

This will push static assets from ./build/assets into production bucket custom-bucket/custom/path/my-ricardo-spa, the isopod v0.20.4 image will be used as executor.

# ...
jobs:
  # ...
  - ric-orb/push_build_to_bucket_job:
      context: prod
      source: ./build/assets
      app_name: my-ricardo-spa
      isopod_version: 0.20.4
      bucket_name: custom-bucket
      bucket_path: /custom/path

Maven build and test job (for Java applications)

Name: maven_build_test

Parameters:

  • path: Path of maven module for the app, or "." for single-app-repo. Default: "."
  • cache_key_prefix: Prefix for the maven artifacts cache-key (used in combination with checksum of pom.xml). No caching if blank. Default: blank
  • executor: Executor for the build. Values: ric-orb/maven_docker (default; more lightweight), ric-orb/maven_vm (required by builds leveraging testcontainers and therefore depending on docker) or your custom executor

Examples

Builds java sources using a docker builder

# ...
jobs:
  # ...
  - ric-orb/maven_build_test:
      context: dev
      cache_key_prefix: "myrepo"
      executor: ric-orb/maven_docker

Builds java sources using a VM builder

# ...
jobs:
  # ...
  - ric-orb/maven_build_test:
      context: dev
      cache_key_prefix: "myrepo"
      executor: ric-orb/maven_vm

Builds java sources using a custom executor

# ...
executors:
  jdk17:
    docker:
      - image: cimg/openjdk:17.0.4
# ...
jobs:
  # ...
  - ric-orb/maven_build_test:
      context: dev
      cache_key_prefix: "myrepo"
      executor: jdk17

Builds java sources for a specific app from a monorepo

# ...
jobs:
  # ...
  - ric-orb/maven_build_test:
      context: dev
      path: "myapp"
      cache_key_prefix: "myrepo"

Maven deploy Java artifact to Artifactory

Name: maven_deploy_artifact

Parameters:

  • path: Path of maven module for the app, or "." for single-app-repo. Default: "."
  • cache_key_prefix: Prefix for the maven artifacts cache-key (used in combination with checksum of pom_file). No caching if blank. Default: blank
  • pom_file: The pom file to be used for the maven build, the checksum used as part of the cache key is calculated from it. Default: "pom.xml"
  • executor: Executor for the build. Values: ric-orb/maven_docker (default; more lightweight), ric-orb/maven_vm (required by builds leveraging testcontainers and therefore depending on docker) or your custom executor

Examples

Deploy Java artifact using a docker builder

# ...
jobs:
  # ...
  - ric-orb/maven_deploy_artifact:
      context: dev
      cache_key_prefix: "myrepo"
      executor: ric-orb/maven_docker

Deploy Java artifact using a VM builder

# ...
jobs:
  # ...
  - ric-orb/maven_deploy_artifact:
      context: dev
      cache_key_prefix: "myrepo"
      executor: ric-orb/maven_vm

Deploy Java artifact using a custom executor

# ...
executors:
  jdk17:
    docker:
      - image: cimg/openjdk:17.0.4
# ...
jobs:
  # ...
  - ric-orb/maven_deploy_artifact:
      context: dev
      cache_key_prefix: "myrepo"
      executor: jdk17

Deploy Java artifact for a specific module from a monorepo

# ...
jobs:
  # ...
  - ric-orb/maven_deploy_artifact:
      context: dev
      path: "mymodule"
      cache_key_prefix: "myrepo"

Deploy Java artifact using a different pom file

# ...
jobs:
  # ...
  - ric-orb/maven_deploy_artifact:
      context: dev
      pom_file: "another-pom.xml"
      cache_key_prefix: "myrepo"

Assert that the project version is a SNAPSHOT (for Java applications)

Name: maven_assert_snapshot_version

Checks that the Maven project (of the provided pom file) has a SNAPSHOT version, otherwise it fails. Typically used to force that branch builds of library modules have a -SNAPSHOT version when manually deployed.

Parameters:

  • pom_file: The pom file to use. Default: "pom.xml"

Example:

#...
jobs:
  - ric-orb/maven_assert_snapshot_version:
      context: dev
#...

No-Op dummy job

Name: no_op

Parameters: none

Examples

Does nothing. Because sometimes you need a job, but you have nothing to do…

# ...
jobs:
  # ...
  - ric-orb/no_op

Build & Quality Gate Job (for Go Applications)

Name: go_build_test

Parameters:

  • executor: circleci executor
  • additional_tests: is a list of additional steps to run, e.g. run tests on common. The default is: empty list
  • work_dir: working directory for executing build and test commands

Job runs install_build_test command, runs additional steps if there are any, and persist resulting app file to workspace (to be used by subsequent jobs)

Examples

In this example you configure:

  • user defined executor
  • custom steps
# ...
jobs:
  # ...
- ric-orb/go_build_test:
    executor:
      name: go/default
      tag: '1.19'
    context: dev
    additional_tests:
      - run:
          name: Running tests on common
          command: make test
          working_directory: common

This example uses default settings and only the minimum required configuration is added:

# ...
jobs:
  # ...
  - ric-orb/quality_gate_job:
      executor:
        name: go/default
        tag: '1.19'
      context: dev

CodeQL

Name: codeql

Parameters:

  • executor: CircleCI Executor to execute CodeQL. As the code to be analyzed also needs to be built, ensure using an executor that is capable of building the code
  • language: Specify the identifier for the language to create a database for, one of: cpp, csharp, go, java, javascript, python, and ruby (use javascript to analyze TypeScript code). See Supported languages and frameworks
  • pre-init-steps: Steps to execute prior to creating the database. This is usually used to restore a cache of dependencies used when building
  • build-command: Use to specify the build command or script that invokes the build process for the codebase. If absent, tries to automatically build. Not needed for Python and JavaScript/TypeScript analysis

Examples

This example uses auto compile functionality to build the code

jobs:
  - ric-orb/codeql:
      context: dev
      executor: jdk17
      language: java
      pre-init-steps: 
        - ric-orb/maven_restore_artifacts:
            prefix: foo
            path: path-to-directory

This example shows an example for Go

jobs:
  - ric-orb/codeql:
      context: dev
      executor:
        name: go/default
        tag: '1.19'
      language: go
      build-command: 'make install && make build'
      pre-init-steps: 
        - restore_cache:
            keys:
              - 'go-mod-{{ arch }}-{{ checksum foo/go.sum"  }}'
              - 'go-mod-{{ arch }}-'

This example shows an example for javascript, where also a build of the code is not necessary

jobs:
  - ric-orb/codeql:
      context: dev
      executor: js
      language: javascript

Deploy Storybook job

Name: js_deploy_storybook

Parameters:

  • ressource_class: Resource allowed to circle-ci for this step (small, medium, medium+, large, xlarge, 2xlarge, 2xlarge+)

Examples

All default

This will create an artifact of your current version of Storybook with a medium resource class. To open it check the Artifact tab of this step in circle-ci, find the index file and click on it.

# ...
jobs:
  # ...
- ric-orb/js_deploy_storybook:
    name: Deploy Storybook
    requires:
      - setup

Custom resource_class version

# ...
jobs:
  # ...
- ric-orb/js_deploy_storybook:
    name: Deploy Storybook
    resource_class: xlarge
    requires:
      - setup

See: