Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for injecting tolerations to sonobuoy pod #1976

Merged
merged 5 commits into from
Jul 29, 2024

Conversation

masap
Copy link
Contributor

@masap masap commented Jun 18, 2024

What this PR does / why we need it:
Add support for injecting tolerations to sonobuoy pod

Which issue(s) this PR fixes

We can inject some tolerations to sonobuoy aggregator pod by adding trailing description into sonobuoy config json.

{
  "AggregatorTolerations": [
    {
      "effect": "NoSchedule",
      "key": "key1",
      "operator": "Equal",
      "value": "value1"
    },
    {
      "effect": "NoSchedule",
      "key": "key2",
      "operator": "Equal",
      "value": "value2"
    }
  ]
}

Special notes for your reviewer:

Release note:

release-note

@leodotcloud
Copy link

@franknstyle Can you please review this?

@leodotcloud
Copy link

@franknstyle Gentle reminder.

@franknstyle
Copy link
Collaborator

@masap can you take a look at CI failures please

@masap
Copy link
Contributor Author

masap commented Jun 27, 2024

@franknstyle I got it, I will take a look.

@masap
Copy link
Contributor Author

masap commented Jun 28, 2024

@franknstyle I checked the golangci-lint errors. It contains some false positives. For example, it says Error: certPEM declared and not used (typecheck). But certPEM is declared at https://github.com/vmware-tanzu/sonobuoy/blob/main/pkg/plugin/driver/base.go#L108 and used at https://github.com/vmware-tanzu/sonobuoy/blob/main/pkg/plugin/driver/base.go#L133. And the codes are not related my PR. I guess these are existing issues. Could you check these errors?

1

@masap
Copy link
Contributor Author

masap commented Jun 28, 2024

@franknstyle I think I can fix these warnings by myself. To do this, I would like to run the lint locally. But trailing command does not show the error. How to run the lint locally?

$ source ./scripts/build_funcs.sh; lint

@franknstyle
Copy link
Collaborator

franknstyle commented Jun 28, 2024

@franknstyle I think I can fix these warnings by myself. To do this, I would like to run the lint locally. But trailing command does not show the error. How to run the lint locally?

$ source ./scripts/build_funcs.sh; lint

Thank you @masap, if you have a look in ./scripts/build_funcs.sh there is a lint func that wraps the call to golangci-lint

@masap
Copy link
Contributor Author

masap commented Jun 28, 2024

if you have a look in ./scripts/build_funcs.sh there is a lint func that wraps the call to golangci-lint

Thank you for your response @franknstyle. It seems the lint() is top level func. And it does not show warning.

$ source ./scripts/build_funcs.sh; lint
level=info msg="[config_reader] Config search paths: [./ /go/src/github.com/vmware-tanzu/sonobuoy /go/src/github.com/vmware-tanzu /go/src/github.com /go/src /go / /root]"
level=info msg="[config_reader] Used config file .golangci.yaml"
level=info msg="[lintersdb] Active 7 linters: [errcheck gosimple govet ineffassign staticcheck typecheck unused]"
level=info msg="[loader] Go packages loading at mode 575 (name|exports_file|imports|files|types_sizes|compiled_files|deps) took 16.109270387s"
level=info msg="[runner/filename_unadjuster] Pre-built 0 adjustments in 12.345302ms"
level=info msg="[linters_context/goanalysis] analyzers took 54.656587971s with top 10 stages: buildir: 47.467751288s, inspect: 1.333221181s, nilness: 1.23890435s, printf: 928.289267ms, fact_deprecated: 749.06002ms, ctrlflow: 735.500759ms, fact_purity: 704.232615ms, typedness: 412.549992ms, SA5012: 394.636286ms, S1038: 75.358947ms"
level=info msg="[runner] Issues before processing: 35, after processing: 0"
level=info msg="[runner] Processors filtering stat (out/in): cgo: 35/35, exclude: 35/35, filename_unadjuster: 35/35, autogenerated_exclude: 35/35, path_prettifier: 35/35, exclude-rules: 0/35, skip_files: 35/35, skip_dirs: 35/35, identifier_marker: 35/35"
level=info msg="[runner] processing took 1.677831ms with stages: identifier_marker: 605.969µs, autogenerated_exclude: 444.957µs, path_prettifier: 437.984µs, skip_dirs: 105.429µs, exclude-rules: 73.819µs, cgo: 3.506µs, filename_unadjuster: 1.983µs, nolint: 842ns, max_same_issues: 641ns, source_code: 330ns, uniq_by_line: 280ns, severity-rules: 270ns, exclude: 261ns, max_from_linter: 260ns, skip_files: 250ns, fixer: 230ns, diff: 210ns, path_shortener: 170ns, sort_results: 160ns, max_per_file_from_linter: 140ns, path_prefixer: 140ns"
level=info msg="[runner] linters took 7.906365733s with stages: goanalysis_metalinter: 7.904619489s"
level=info msg="File cache stats: 0 entries of total size 0B"
level=info msg="Memory: 242 samples, avg is 488.2MB, max is 2266.6MB"
level=info msg="Execution took 24.034036056s"

@franknstyle
Copy link
Collaborator

franknstyle commented Jun 28, 2024

Let's disable typecheck and review in a following issue. Can you add a commit for adding

linters:
  disable:
    - typecheck

to .github/workflows/ci-lint.yaml

@masap
Copy link
Contributor Author

masap commented Jun 28, 2024

Do you mean .golangci.yaml? I sent a fix.

@franknstyle
Copy link
Collaborator

Hi @masap,

Looks like we are failing a vuln scan now. Are you able to update the go mod to 1.21.4 and go mod tidy so stdlib get's updated.

Thank you.

@masap
Copy link
Contributor Author

masap commented Jul 1, 2024

Hmm, lint error still exists. According to [1], we can't disable typecheck.

We upgraded golang 1.20 -> 1.21 by commit 9a64023. But according to [2], go1.21 is officially supported since golangci-lint v1.54.1. So, I upgraded golangci-lint to v1.54.2.

Looks like we are failing a vuln scan now. Are you able to update the go mod to 1.21.4 and go mod tidy so stdlib get's updated.

I will fix this as well.

[1] https://golangci-lint.run/welcome/faq/#why-do-you-have-typecheck-errors
[2] golangci/golangci-lint#3933

@masap
Copy link
Contributor Author

masap commented Jul 1, 2024

Looks like we are failing a vuln scan now. Are you able to update the go mod to 1.21.4 and go mod tidy so stdlib get's updated.

Fixed.

@leodotcloud
Copy link

@franknstyle Could you enable auto pipeline builds for this PR? This would help ensure the pipelines pass and reduce turnaround time, especially considering the timezone differences between all of us?

@masap masap force-pushed the fix-1973 branch 2 times, most recently from 38b876f to 5d7d17c Compare July 1, 2024 07:53
@masap
Copy link
Contributor Author

masap commented Jul 1, 2024

I have confirmed that it passes, except for Windows-related checks with my local Github actions.

@masap masap force-pushed the fix-1973 branch 2 times, most recently from f0b70ae to aa5665c Compare July 1, 2024 09:30
masap added 4 commits July 1, 2024 18:58
We upgraded golang lang 1.20 -> 1.21 by commit
9a64023. But according to [2], go1.21 is
officially supported since golangci-lint v1.54.1. So, this PR upgrades
golangci-lint to v1.54.2.

Signed-off-by: Masashi Honma <[email protected]>
According to trivy, golang 1.21.4 has trailing vulnerabilities. We upgrade it
to 1.21.11 to fix the vulnerabilities.

$ trivy image masap20220915/sonobuoy:amd64-v0.57
2024-07-01T09:50:21+09:00	INFO	Vulnerability scanning is enabled
2024-07-01T09:50:21+09:00	INFO	Secret scanning is enabled
2024-07-01T09:50:21+09:00	INFO	If your scanning is slow, please try '--scanners vuln' to disable secret scanning
2024-07-01T09:50:21+09:00	INFO	Please see also https://aquasecurity.github.io/trivy/v0.52/docs/scanner/secret/#recommendation for faster secret detection
2024-07-01T09:50:24+09:00	INFO	Detected OS	family="debian" version="12.5"
2024-07-01T09:50:24+09:00	INFO	[debian] Detecting vulnerabilities...	os_version="12" pkg_num=3
2024-07-01T09:50:24+09:00	INFO	Number of language-specific files	num=1
2024-07-01T09:50:24+09:00	INFO	[gobinary] Detecting vulnerabilities...

masap20220915/sonobuoy:amd64-v0.57 (debian 12.5)

Total: 0 (UNKNOWN: 0, LOW: 0, MEDIUM: 0, HIGH: 0, CRITICAL: 0)

sonobuoy (gobinary)

Total: 9 (UNKNOWN: 0, LOW: 0, MEDIUM: 7, HIGH: 1, CRITICAL: 1)

┌─────────┬────────────────┬──────────┬────────┬───────────────────┬─────────────────┬──────────────────────────────────────────────────────────────┐
│ Library │ Vulnerability  │ Severity │ Status │ Installed Version │  Fixed Version  │                            Title                             │
├─────────┼────────────────┼──────────┼────────┼───────────────────┼─────────────────┼──────────────────────────────────────────────────────────────┤
│ stdlib  │ CVE-2024-24790 │ CRITICAL │ fixed  │ 1.21.4            │ 1.21.11, 1.22.4 │ golang: net/netip: Unexpected behavior from Is methods for   │
│         │                │          │        │                   │                 │ IPv4-mapped IPv6 addresses                                   │
│         │                │          │        │                   │                 │ https://avd.aquasec.com/nvd/cve-2024-24790                   │
│         ├────────────────┼──────────┤        │                   ├─────────────────┼──────────────────────────────────────────────────────────────┤
│         │ CVE-2023-45288 │ HIGH     │        │                   │ 1.21.9, 1.22.2  │ golang: net/http, x/net/http2: unlimited number of           │
│         │                │          │        │                   │                 │ CONTINUATION frames causes DoS                               │
│         │                │          │        │                   │                 │ https://avd.aquasec.com/nvd/cve-2023-45288                   │
│         ├────────────────┼──────────┤        │                   ├─────────────────┼──────────────────────────────────────────────────────────────┤
│         │ CVE-2023-39326 │ MEDIUM   │        │                   │ 1.20.12, 1.21.5 │ golang: net/http/internal: Denial of Service (DoS) via       │
│         │                │          │        │                   │                 │ Resource Consumption via HTTP requests...                    │
│         │                │          │        │                   │                 │ https://avd.aquasec.com/nvd/cve-2023-39326                   │
│         ├────────────────┤          │        │                   ├─────────────────┼──────────────────────────────────────────────────────────────┤
│         │ CVE-2023-45289 │          │        │                   │ 1.21.8, 1.22.1  │ golang: net/http/cookiejar: incorrect forwarding of          │
│         │                │          │        │                   │                 │ sensitive headers and cookies on HTTP redirect...            │
│         │                │          │        │                   │                 │ https://avd.aquasec.com/nvd/cve-2023-45289                   │
│         ├────────────────┤          │        │                   │                 ├──────────────────────────────────────────────────────────────┤
│         │ CVE-2023-45290 │          │        │                   │                 │ golang: net/http: memory exhaustion in                       │
│         │                │          │        │                   │                 │ Request.ParseMultipartForm                                   │
│         │                │          │        │                   │                 │ https://avd.aquasec.com/nvd/cve-2023-45290                   │
│         ├────────────────┤          │        │                   │                 ├──────────────────────────────────────────────────────────────┤
│         │ CVE-2024-24783 │          │        │                   │                 │ golang: crypto/x509: Verify panics on certificates with an   │
│         │                │          │        │                   │                 │ unknown public key algorithm...                              │
│         │                │          │        │                   │                 │ https://avd.aquasec.com/nvd/cve-2024-24783                   │
│         ├────────────────┤          │        │                   │                 ├──────────────────────────────────────────────────────────────┤
│         │ CVE-2024-24784 │          │        │                   │                 │ golang: net/mail: comments in display names are incorrectly  │
│         │                │          │        │                   │                 │ handled                                                      │
│         │                │          │        │                   │                 │ https://avd.aquasec.com/nvd/cve-2024-24784                   │
│         ├────────────────┤          │        │                   │                 ├──────────────────────────────────────────────────────────────┤
│         │ CVE-2024-24785 │          │        │                   │                 │ golang: html/template: errors returned from MarshalJSON      │
│         │                │          │        │                   │                 │ methods may break template escaping                          │
│         │                │          │        │                   │                 │ https://avd.aquasec.com/nvd/cve-2024-24785                   │
│         ├────────────────┤          │        │                   ├─────────────────┼──────────────────────────────────────────────────────────────┤
│         │ CVE-2024-24789 │          │        │                   │ 1.21.11, 1.22.4 │ golang: archive/zip: Incorrect handling of certain ZIP files │
│         │                │          │        │                   │                 │ https://avd.aquasec.com/nvd/cve-2024-24789                   │
└─────────┴────────────────┴──────────┴────────┴───────────────────┴─────────────────┴──────────────────────────────────────────────────────────────┘

Signed-off-by: Masashi Honma <[email protected]>
ERROR: failed to solve: failed to compute cache key: mount callback failed on /tmp/containerd-mount1917080101: link /tmp/containerd-mount1917080101/Windows/INF/basicrender.inf /tmp/containerd-mount1917080101/Windows/System32/DriverStore/FileRepository/basicrender.inf_amd64_efdc64af60c69a6d/basicrender.inf: no such file or directory
Error: Process completed with exit code 1.

According to [1], we need to use ltsc2022 as a tag.

[1] microsoft/Windows-Containers#493

Signed-off-by: Masashi Honma <[email protected]>
Resolves vmware-tanzu#1973.

We can inject some tolerations to sonobuoy aggregator pod by adding trailing
description into sonobuoy config json.

{
  "AggregatorTolerations": [
    {
      "effect": "NoSchedule",
      "key": "key1",
      "operator": "Equal",
      "value": "value1"
    },
    {
      "effect": "NoSchedule",
      "key": "key2",
      "operator": "Equal",
      "value": "value2"
    }
  ]
}

Signed-off-by: Masashi Honma <[email protected]>
@masap
Copy link
Contributor Author

masap commented Jul 1, 2024

I confirmed all checks are passed.

@leodotcloud
Copy link

Thanks @masap, hope the pipeline will go through once @franknstyle approves.

@leodotcloud
Copy link

@franknstyle Can you please take a look at this PR again?

To fix trailing warning.

Total: 1 (MEDIUM: 1, HIGH: 0, CRITICAL: 0)

┌─────────┬────────────────┬──────────┬────────┬───────────────────┬─────────────────┬──────────────────────────────────────────────────────────┐
│ Library │ Vulnerability  │ Severity │ Status │ Installed Version │  Fixed Version  │                          Title                           │
├─────────┼────────────────┼──────────┼────────┼───────────────────┼─────────────────┼──────────────────────────────────────────────────────────┤
│ stdlib  │ CVE-2024-24791 │ MEDIUM   │ fixed  │ 1.21.11           │ 1.21.12, 1.22.5 │ net/http: Denial of service due to improper 100-continue │
│         │                │          │        │                   │                 │ handling in net/http                                     │
│         │                │          │        │                   │                 │ https://avd.aquasec.com/nvd/cve-2024-24791               │
└─────────┴────────────────┴──────────┴────────┴───────────────────┴─────────────────┴──────────────────────────────────────────────────────────┘
Signed-off-by: Masashi Honma <[email protected]>
@masap
Copy link
Contributor Author

masap commented Jul 15, 2024

I fixed a CI error caused by new CVE (CVE-2024-24791).

@franknstyle franknstyle merged commit 9c04b0b into vmware-tanzu:main Jul 29, 2024
20 checks passed
@leodotcloud
Copy link

Thank you for your support @franknstyle! When can we expect the next release?

@masap masap deleted the fix-1973 branch July 29, 2024 07:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for injecting tolerations to sonobuoy pod
3 participants