Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore!: update to SonarQube 10, migrate to upstream chart, add unicorn CGR flavor #100

Merged
merged 27 commits into from
Jul 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
2a261b3
add unicorn flavor
marshall007 Jul 17, 2024
0519772
fix monitoringPasscode and jdbcSecret not being set by default
marshall007 Jul 18, 2024
85bc218
bump upstream image and package versions
marshall007 Jul 18, 2024
5853eb5
test/publish unicorn flavor
marshall007 Jul 18, 2024
784beca
fix bundle version
marshall007 Jul 18, 2024
99149e0
update uds-common actions
marshall007 Jul 19, 2024
e87498b
enable id-token permission"
marshall007 Jul 19, 2024
68a22c0
do not run upgrade tests for unicorn
marshall007 Jul 19, 2024
58f0653
increase tests timeout
marshall007 Jul 19, 2024
363c894
try disabling monitoring
marshall007 Jul 19, 2024
4ed8e5b
big boy
marshall007 Jul 19, 2024
d142bc6
upgrade registry1 image
marshall007 Jul 22, 2024
3fe2a83
fix smoke tests
marshall007 Jul 22, 2024
541e7ae
fix upstream image
marshall007 Jul 22, 2024
f9a6b1d
fix probes in registry1 flavor
marshall007 Jul 22, 2024
cd17e0a
remove monitoring passcode
marshall007 Jul 23, 2024
bb348de
revert to standard runner
marshall007 Jul 23, 2024
f73bcf5
setup CGR in publish workflow
marshall007 Jul 23, 2024
b2a7cfa
remove renvoate config for repo1 chart
marshall007 Jul 23, 2024
7205029
add migrate-db task
marshall007 Jul 24, 2024
626ad04
fix status check
marshall007 Jul 24, 2024
24872be
try big boy runner again
marshall007 Jul 24, 2024
983f1d4
address feedback, migrate db in zarf action
marshall007 Jul 25, 2024
7855d6c
poc using utils:determine-repo task
marshall007 Jul 25, 2024
9ef078c
poc using updated pull/publish tasks
marshall007 Jul 26, 2024
97a9b78
Merge branch 'main' into marshall_unicorn-flavor
Racer159 Jul 29, 2024
1002fd8
uds-common v0.10.0
marshall007 Jul 30, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/commitlint.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,4 @@ on:
jobs:
validate:
name: Validate
uses: defenseunicorns/uds-common/.github/workflows/commitlint.yaml@91515ef86914334356c35ffb4b2c2c5fb4d19174 # v0.9.0
uses: defenseunicorns/uds-common/.github/workflows/commitlint.yaml@e7c0643d3f9d74ce49c7386b48964d2be646d726 # v0.10.0
2 changes: 1 addition & 1 deletion .github/workflows/lint.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:
fetch-depth: 0

- name: Environment setup
uses: defenseunicorns/uds-common/.github/actions/setup@91515ef86914334356c35ffb4b2c2c5fb4d19174 # v0.9.0
uses: defenseunicorns/uds-common/.github/actions/setup@e7c0643d3f9d74ce49c7386b48964d2be646d726 # v0.10.0
with:
registry1Username: ${{ secrets.IRON_BANK_ROBOT_USERNAME }}
registry1Password: ${{ secrets.IRON_BANK_ROBOT_PASSWORD }}
Expand Down
10 changes: 6 additions & 4 deletions .github/workflows/tag-and-release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ jobs:
runs-on: ${{ matrix.architecture == 'arm64' && 'uds-swf-ubuntu-arm64-4-core' || 'ubuntu-latest' }}
strategy:
matrix:
flavor: [upstream, registry1]
flavor: [upstream, unicorn, registry1]
architecture: [amd64, arm64]
exclude:
- flavor: registry1
Expand All @@ -35,26 +35,28 @@ jobs:
permissions:
contents: read
packages: write
id-token: write

steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7

- name: Environment setup
uses: defenseunicorns/uds-common/.github/actions/setup@91515ef86914334356c35ffb4b2c2c5fb4d19174 # v0.9.0
uses: defenseunicorns/uds-common/.github/actions/setup@e7c0643d3f9d74ce49c7386b48964d2be646d726 # v0.10.0
with:
registry1Username: ${{ secrets.IRON_BANK_ROBOT_USERNAME }}
registry1Password: ${{ secrets.IRON_BANK_ROBOT_PASSWORD }}
ghToken: ${{ secrets.GITHUB_TOKEN }}
chainguardIdentity: ${{ secrets.CHAINGUARD_IDENTITY }}

- name: Publish Package
run: uds run -f tasks/publish.yaml package --set FLAVOR=${{ matrix.flavor }} --no-progress
marshall007 marked this conversation as resolved.
Show resolved Hide resolved

- name: Debug Output
if: ${{ always() }}
uses: defenseunicorns/uds-common/.github/actions/debug-output@91515ef86914334356c35ffb4b2c2c5fb4d19174 # v0.9.0
uses: defenseunicorns/uds-common/.github/actions/debug-output@e7c0643d3f9d74ce49c7386b48964d2be646d726 # v0.10.0

- name: Save logs
if: always()
uses: defenseunicorns/uds-common/.github/actions/save-logs@91515ef86914334356c35ffb4b2c2c5fb4d19174 # v0.9.0
uses: defenseunicorns/uds-common/.github/actions/save-logs@e7c0643d3f9d74ce49c7386b48964d2be646d726 # v0.10.0
with:
suffix: ${{ matrix.flavor }}-${{ matrix.architecture }}-${{ github.run_id }}-${{ github.run_attempt }}
19 changes: 13 additions & 6 deletions .github/workflows/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,41 +29,48 @@ concurrency:

permissions:
contents: read
id-token: write

jobs:
run-test:
name: ${{ matrix.type }} ${{ matrix.flavor }}
runs-on: ubuntu-latest
runs-on: uds-swf-ubuntu-big-boy-4-core
Racer159 marked this conversation as resolved.
Show resolved Hide resolved
timeout-minutes: 20
strategy:
matrix:
flavor: [upstream, registry1]
flavor: [upstream, unicorn, registry1]
type: [install, upgrade]
# do not run upgrade tests for unicorn
# TODO @marshall: remove after first unicorn release
exclude:
- flavor: unicorn
type: upgrade

steps:
- name: Checkout repository
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7

- name: Environment setup
uses: defenseunicorns/uds-common/.github/actions/setup@91515ef86914334356c35ffb4b2c2c5fb4d19174 # v0.9.0
uses: defenseunicorns/uds-common/.github/actions/setup@e7c0643d3f9d74ce49c7386b48964d2be646d726 # v0.10.0
with:
registry1Username: ${{ secrets.IRON_BANK_ROBOT_USERNAME }}
registry1Password: ${{ secrets.IRON_BANK_ROBOT_PASSWORD }}
ghToken: ${{ secrets.GITHUB_TOKEN }}
chainguardIdentity: ${{ secrets.CHAINGUARD_IDENTITY }}

- name: Test
uses: defenseunicorns/uds-common/.github/actions/test@91515ef86914334356c35ffb4b2c2c5fb4d19174 # v0.9.0
uses: defenseunicorns/uds-common/.github/actions/test@e7c0643d3f9d74ce49c7386b48964d2be646d726 # v0.10.0
with:
flavor: ${{ matrix.flavor }}
type: ${{ matrix.type }}

- name: Debug Output
if: ${{ always() }}
uses: defenseunicorns/uds-common/.github/actions/debug-output@91515ef86914334356c35ffb4b2c2c5fb4d19174 # v0.9.0
uses: defenseunicorns/uds-common/.github/actions/debug-output@e7c0643d3f9d74ce49c7386b48964d2be646d726 # v0.10.0

- name: Save logs
if: always()
uses: defenseunicorns/uds-common/.github/actions/save-logs@91515ef86914334356c35ffb4b2c2c5fb4d19174 # v0.9.0
uses: defenseunicorns/uds-common/.github/actions/save-logs@e7c0643d3f9d74ce49c7386b48964d2be646d726 # v0.10.0
with:
suffix: ${{ matrix.type }}-${{ matrix.flavor }}-${{ github.run_id }}-${{ github.run_attempt }}

Expand Down
4 changes: 2 additions & 2 deletions chart/templates/sonarqube-sso-secret.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# This secret will be used if sso is disabled, instead of the templated one in uds-package.yaml.
# Sonarqube needs to mount the secret and creating it this way avoids creating an unnecessary
# This secret will be used if sso is disabled, instead of the templated one in uds-package.yaml.
# Sonarqube needs to mount the secret and creating it this way avoids creating an unnecessary
# client in the keycloak realm and unnecessary secret data in the cluster.
{{- if not .Values.sso.enabled }}

Expand Down
10 changes: 6 additions & 4 deletions common/zarf.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,8 @@ components:
localPath: ../chart
- name: sonarqube
namespace: sonarqube
# see note in README.md about use of registry1 chart for both flavors
url: https://repo1.dso.mil/big-bang/product/packages/sonarqube.git
gitPath: chart
version: "8.0.4-bb.5"
url: https://SonarSource.github.io/helm-chart-sonarqube
version: 10.6.1
valuesFiles:
- ../values/common-values.yaml
actions:
Expand All @@ -31,3 +29,7 @@ components:
name: sonarqube
namespace: sonarqube
condition: "'{.status.phase}'=Ready"
- description: Perform DB migrations if necessary
cmd: |
STATUS=$(curl -XPOST -s "https://sonarqube.$ZARF_VAR_DOMAIN/api/system/migrate_db" | ./zarf tools yq '.state')
echo "SonarQube migration state: ${STATUS}"
7 changes: 0 additions & 7 deletions renovate.json
Original file line number Diff line number Diff line change
Expand Up @@ -22,13 +22,6 @@
"labels": ["package-deps"],
"commitMessageTopic": "package-deps",
"matchDatasources": ["docker", "helm", "git-tags"]
},
{
"groupName": "SonarQube Package Dependencies",
"labels": ["package-deps"],
"commitMessageTopic": "package-deps",
"matchPackageNames": ["https://repo1.dso.mil/big-bang/product/packages/sonarqube.git"],
"allowedVersions": "/^8.+-bb.+/"
}
]
}
10 changes: 5 additions & 5 deletions tasks.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@ includes:
- cleanup: ./tasks/cleanup.yaml
- dependencies: ./tasks/dependencies.yaml
- test: ./tasks/test.yaml
- create: https://raw.githubusercontent.com/defenseunicorns/uds-common/v0.9.0/tasks/create.yaml
- lint: https://raw.githubusercontent.com/defenseunicorns/uds-common/v0.9.0/tasks/lint.yaml
- pull: https://raw.githubusercontent.com/defenseunicorns/uds-common/v0.9.0/tasks/pull.yaml
- deploy: https://raw.githubusercontent.com/defenseunicorns/uds-common/v0.9.0/tasks/deploy.yaml
- setup: https://raw.githubusercontent.com/defenseunicorns/uds-common/v0.9.0/tasks/setup.yaml
- create: https://raw.githubusercontent.com/defenseunicorns/uds-common/v0.10.0/tasks/create.yaml
- lint: https://raw.githubusercontent.com/defenseunicorns/uds-common/v0.10.0/tasks/lint.yaml
- pull: https://raw.githubusercontent.com/defenseunicorns/uds-common/v0.10.0/tasks/pull.yaml
- deploy: https://raw.githubusercontent.com/defenseunicorns/uds-common/v0.10.0/tasks/deploy.yaml
- setup: https://raw.githubusercontent.com/defenseunicorns/uds-common/v0.10.0/tasks/setup.yaml

tasks:
- name: default
Expand Down
8 changes: 4 additions & 4 deletions tasks/publish.yaml
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
includes:
- create: https://raw.githubusercontent.com/defenseunicorns/uds-common/v0.9.0/tasks/create.yaml
- publish: https://raw.githubusercontent.com/defenseunicorns/uds-common/v0.9.0/tasks/publish.yaml
- setup: https://raw.githubusercontent.com/defenseunicorns/uds-common/v0.9.0/tasks/setup.yaml
- deploy: https://raw.githubusercontent.com/defenseunicorns/uds-common/v0.9.0/tasks/deploy.yaml
- create: https://raw.githubusercontent.com/defenseunicorns/uds-common/v0.10.0/tasks/create.yaml
- publish: https://raw.githubusercontent.com/defenseunicorns/uds-common/v0.10.0/tasks/publish.yaml
- setup: https://raw.githubusercontent.com/defenseunicorns/uds-common/v0.10.0/tasks/setup.yaml
- deploy: https://raw.githubusercontent.com/defenseunicorns/uds-common/v0.10.0/tasks/deploy.yaml
- dependencies: ./dependencies.yaml
- test: ./test.yaml

Expand Down
2 changes: 1 addition & 1 deletion tasks/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ tasks:
cmd: |
STATUS=$(curl -s 'https://sonarqube.uds.dev/api/system/status' | ./uds zarf tools yq '.status')
echo "SonarQube system status: ${STATUS}"
if [ $STATUS != "UP" ]; then
if [ "$STATUS" != "UP" ]; then
sleep 10
exit 1
fi
Expand Down
4 changes: 1 addition & 3 deletions tests/auth.setup.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import { authFile } from './playwright.config';
setup('authenticate', async ({ page, context }) => {
await page.goto('/sessions/new');

await page.locator('.identity-provider-link').click();
await page.getByRole('button', { name: 'Log in with Keycloak' }).click();
await page.getByLabel('Username or email').fill('doug');
await page.getByLabel('Password').fill('unicorn123!@#UN');

Expand All @@ -23,6 +23,4 @@ setup('authenticate', async ({ page, context }) => {
expect(keycloakCookie?.domain).toContain("sso.");

await page.context().storageState({ path: authFile });

await expect(page).toHaveURL('/projects');
marshall007 marked this conversation as resolved.
Show resolved Hide resolved
})
12 changes: 7 additions & 5 deletions tests/sonarqube.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,17 @@ function randomProjectName() {
}

test('create a project', async ({ page }) => {
await page.goto('/projects/create');
await page.goto('/projects/create?mode=manual');

const projectName = randomProjectName();

await page.getByRole('button', { name: 'Manually' }).click();
await page.getByLabel('Project display name*').fill(projectName);
await page.getByRole('button', { name: 'Set Up' }).click();
await page.getByRole('button', { name: 'Next' }).click();

await expect(page).toHaveURL(`/dashboard?id=${projectName}`);
await page.getByLabel('Use the global setting').check();
await page.getByRole('button', { name: 'Create project' }).click();

await expect(page.getByRole('heading', { level: 1 })).toContainText(projectName);
await expect(page).toHaveURL(`/tutorials?id=${projectName}`);

await expect(page.getByRole('heading', { level: 1 })).toContainText('Analysis Method');
});
24 changes: 8 additions & 16 deletions values/common-values.yaml
Original file line number Diff line number Diff line change
@@ -1,23 +1,22 @@
OpenShift:
enabled: false

edition: "community"

# Name of the secret from which to load additional properties: https://community.sonarsource.com/t/additional-sonar-properties-to-load-from-a-secret/73748
# This secret will be created by the uds operator based on the sso spec defined in chart/templates/uds-package.yaml
sonarSecretProperties: sonarqube-sso

monitoring:
enabled: true
prometheusExporter:
marshall007 marked this conversation as resolved.
Show resolved Hide resolved
enabled: false

prometheusMonitoring:
podMonitor:
enabled: false

initSysctl:
enabled: false

elasticsearch:
configureNode: false

jdbcOverwrite:
enable: true
enable: true # DEPRECATED: use `jdbcOverwrite.enabled` in v10.7.0
marshall007 marked this conversation as resolved.
Show resolved Hide resolved
enabled: true
jdbcUrl: jdbc:postgresql://###ZARF_VAR_SONARQUBE_DB_ENDPOINT###:5432/###ZARF_VAR_SONARQUBE_DB_NAME###
jdbcUsername: ###ZARF_VAR_SONARQUBE_DB_USERNAME###
jdbcPassword: null
Expand All @@ -26,10 +25,3 @@ jdbcOverwrite:

postgresql:
enabled: false
postgresqlServer: ###ZARF_VAR_SONARQUBE_DB_ENDPOINT###
existingSecret: "sonarqube-postgres"
existingSecretPasswordKey: "password"
postgresqlUsername: ###ZARF_VAR_SONARQUBE_DB_USERNAME###
postgresqlDatabase: ###ZARF_VAR_SONARQUBE_DB_NAME###
service:
port: 5432
16 changes: 15 additions & 1 deletion values/registry1-values.yaml
Original file line number Diff line number Diff line change
@@ -1,9 +1,23 @@
image:
repository: registry1.dso.mil/ironbank/sonarsource/sonarqube/sonarqube
tag: 9.9.5-community
tag: 10.6.0-community

initContainers:
image: registry1.dso.mil/ironbank/big-bang/base:2.1.0

prometheusExporter:
image: registry1.dso.mil/ironbank/big-bang/base:2.1.0

# wget used by default probes not available in registry1 images
livenessProbe:
exec: null
httpGet:
path: /
scheme: HTTP
port: 9000
readinessProbe:
exec: null
httpGet:
path: /api/system/status
scheme: HTTP
port: 9000
9 changes: 9 additions & 0 deletions values/unicorn-values.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
image:
marshall007 marked this conversation as resolved.
Show resolved Hide resolved
repository: cgr.dev/du-uds-defenseunicorns/sonarqube
tag: 10.6.0

initContainers:
image: cgr.dev/du-uds-defenseunicorns/busybox-fips:1.36

prometheusExporter:
image: cgr.dev/du-uds-defenseunicorns/curl-fips:8-dev
2 changes: 1 addition & 1 deletion values/upstream-values.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
image:
repository: sonarqube
tag: 9.9.5-community
tag: 10.6.0-community

initContainers:
image: busybox:1.36
Expand Down
20 changes: 18 additions & 2 deletions zarf.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ components:
- values/registry1-values.yaml
images:
- "registry1.dso.mil/ironbank/big-bang/base:2.1.0"
- "registry1.dso.mil/ironbank/sonarsource/sonarqube/sonarqube:9.9.5-community"
- "registry1.dso.mil/ironbank/sonarsource/sonarqube/sonarqube:10.6.0-community"

# Note: upstream flavor is experimental
- name: sonarqube
Expand All @@ -50,6 +50,22 @@ components:
valuesFiles:
- values/upstream-values.yaml
images:
- "sonarqube:9.9.5-community"
- "sonarqube:10.6.0-community"
- "curlimages/curl:8.8.0"
- "busybox:1.36"

- name: sonarqube
required: true
description: "Deploy sonarqube"
import:
path: common
only:
flavor: unicorn
charts:
- name: sonarqube
valuesFiles:
- values/unicorn-values.yaml
images:
- "cgr.dev/du-uds-defenseunicorns/sonarqube:10.6.0"
- "cgr.dev/du-uds-defenseunicorns/curl-fips:8-dev"
- "cgr.dev/du-uds-defenseunicorns/busybox-fips:1.36"