KICS scan failing

[ppanchal10]

Expected Behavior

KICS provides IaC scan result

Actual Behavior

KICS fails to scan code repository

panic: interface conversion: interface {} is string, not model.Document

goroutine 2790 [running]:
github.com/Checkmarx/kics/pkg/parser/terraform.processResources(0xc0013e6480, 0xc0018230e0, 0x4c, 0x8, 0xc0007b8408)
/home/runner/work/kics/kics/pkg/parser/terraform/terraform.go:71 +0x25b
github.com/Checkmarx/kics/pkg/parser/terraform.addExtraInfo(0xc000ef8498, 0x1, 0x1, 0xc0018230e0, 0x4c, 0x1, 0x1, 0x0, 0xc001893d40, 0x0)
/home/runner/work/kics/kics/pkg/parser/terraform/terraform.go:82 +0xec
github.com/Checkmarx/kics/pkg/parser/terraform.(*Parser).Parse(0xc0014bc900, 0xc0018230e0, 0x4c, 0xc001992000, 0x425, 0x480, 0xc0018b8a08, 0x0, 0x0, 0x0, ...)
/home/runner/work/kics/kics/pkg/parser/terraform/terraform.go:102 +0x1df
github.com/Checkmarx/kics/pkg/parser.(*Parser).Parse(0xc001af0870, 0xc0018230e0, 0x4c, 0xc001992000, 0x425, 0x480, 0x203000, 0x1, 0x1, 0x203000, ...)
/home/runner/work/kics/kics/pkg/parser/parser.go:89 +0x173
github.com/Checkmarx/kics/pkg/kics.(*Service).sink(0xc0000eba40, 0x344d438, 0xc00012a008, 0xc0018230e0, 0x4c, 0x30543a9, 0x7, 0x3404b00, 0xc000ef8380, 0x0, ...)
/home/runner/work/kics/kics/pkg/kics/sink.go:23 +0xd7
github.com/Checkmarx/kics/pkg/kics.(*Service).StartScan.func1(0x344d438, 0xc00012a008, 0xc0018230e0, 0x4c, 0x341b9b0, 0xc000ef8380, 0xffffffffffffffff, 0xc0018230e0)
/home/runner/work/kics/kics/pkg/kics/service.go:67 +0xd0
github.com/Checkmarx/kics/pkg/engine/provider.(*FileSystemSourceProvider).walkDir.func1(0xc0018230e0, 0x4c, 0x345c8c8, 0xc00182fd40, 0x0, 0x0, 0x0, 0x0)
/home/runner/work/kics/kics/pkg/engine/provider/filesystem.go:151 +0x554
path/filepath.walk(0xc0018230e0, 0x4c, 0x345c8c8, 0xc00182fd40, 0xc00270fdb0, 0x0, 0x0)
/opt/hostedtoolcache/go/1.16.5/x64/src/path/filepath/path.go:414 +0x457
path/filepath.walk(0xc001822fa0, 0x45, 0x345c8c8, 0xc00182fba0, 0xc0012d9db0, 0x0, 0x0)
/opt/hostedtoolcache/go/1.16.5/x64/src/path/filepath/path.go:438 +0x31b
path/filepath.walk(0xc0007a3580, 0x31, 0x345c8c8, 0xc0019444e0, 0xc0012d9db0, 0x0, 0x0)
/opt/hostedtoolcache/go/1.16.5/x64/src/path/filepath/path.go:438 +0x31b
path/filepath.walk(0xc0006d2ea0, 0x28, 0x345c8c8, 0xc000c60270, 0xc0012d9db0, 0x0, 0xc00063e210)
/opt/hostedtoolcache/go/1.16.5/x64/src/path/filepath/path.go:438 +0x31b
path/filepath.Walk(0xc0006d2ea0, 0x28, 0xc0005c2db0, 0x0, 0xc000c601a0)
/opt/hostedtoolcache/go/1.16.5/x64/src/path/filepath/path.go:501 +0x113
github.com/Checkmarx/kics/pkg/engine/provider.(*FileSystemSourceProvider).walkDir(0xc0014af560, 0x344d438, 0xc00012a008, 0xc0006d2ea0, 0x28, 0x0, 0xc0014f8000, 0xc0014f8020, 0xc001af0840, 0x0, ...)
/home/runner/work/kics/kics/pkg/engine/provider/filesystem.go:119 +0xc5
github.com/Checkmarx/kics/pkg/engine/provider.(*FileSystemSourceProvider).GetSources(0xc0014af560, 0x344d438, 0xc00012a008, 0xc001af0840, 0xc0014f8000, 0xc0014f8020, 0x0, 0x0)
/home/runner/work/kics/kics/pkg/engine/provider/filesystem.go:108 +0x29d
github.com/Checkmarx/kics/pkg/kics.(*Service).StartScan(0xc0000eba40, 0x344d438, 0xc00012a008, 0x30543a9, 0x7, 0xc000b2ad20, 0xc00173b534, 0xc000918930)
/home/runner/work/kics/kics/pkg/kics/service.go:63 +0x1c6
created by github.com/Checkmarx/kics/pkg/scanner.StartScan
/home/runner/work/kics/kics/pkg/scanner/scanner.go:30 +0x22b

panic: interface conversion: interface {} is string, not map[string]interface {} [recovered]
panic: interface conversion: interface {} is string, not map[string]interface {}

goroutine 2822 [running]:
gopkg.in/yaml%2ev3.handleErr(0xc001413368)
/home/runner/go/pkg/mod/gopkg.in/[email protected]/yaml.go:294 +0x8d
panic(0x2dca060, 0xc001cfd410)
/opt/hostedtoolcache/go/1.16.8/x64/src/runtime/panic.go:965 +0x1b9
github.com/Checkmarx/kics/pkg/model.(*Document).UnmarshalYAML(0xc001d86c00, 0xc001829cc0, 0x2e619c0, 0x8329310)
/home/runner/work/kics/kics/pkg/model/model_yaml.go:13 +0x14d
gopkg.in/yaml%2ev3.(*decoder).callUnmarshaler(0xc0009bccb0, 0xc001829cc0, 0x8329310, 0xc001d86c00, 0xc001d86c00)
/home/runner/go/pkg/mod/gopkg.in/[email protected]/decode.go:361 +0x48
gopkg.in/yaml%2ev3.(*decoder).prepare(0xc0009bccb0, 0xc001829cc0, 0x2e34de0, 0xc001d86c00, 0x195, 0x0, 0x0, 0x0, 0x0)
/home/runner/go/pkg/mod/gopkg.in/[email protected]/decode.go:418 +0x23d
gopkg.in/yaml%2ev3.(*decoder).unmarshal(0xc0009bccb0, 0xc001829cc0, 0x2e34de0, 0xc001d86c00, 0x195, 0xc000f75eb0)
/home/runner/go/pkg/mod/gopkg.in/[email protected]/decode.go:497 +0x135
gopkg.in/yaml%2ev3.(*decoder).document(0xc0009bccb0, 0xc001829c20, 0x2e34de0, 0xc001d86c00, 0x195, 0x0)
/home/runner/go/pkg/mod/gopkg.in/[email protected]/decode.go:522 +0x7c
gopkg.in/yaml%2ev3.(*decoder).unmarshal(0xc0009bccb0, 0xc001829c20, 0x2e34de0, 0xc001d86c00, 0x195, 0x195)
/home/runner/go/pkg/mod/gopkg.in/[email protected]/decode.go:493 +0x38f
gopkg.in/yaml%2ev3.(*Decoder).Decode(0xc0014133d8, 0x2e619c0, 0xc001d86c00, 0x0, 0x0)
/home/runner/go/pkg/mod/gopkg.in/[email protected]/yaml.go:131 +0x1f2
github.com/Checkmarx/kics/pkg/parser/yaml.(*Parser).Parse(0x446b808, 0xc0004203f0, 0x61, 0xc0017cb710, 0x21, 0x30, 0xc001335f38, 0x0, 0x0, 0x8a0d623b01413528, ...)
/home/runner/work/kics/kics/pkg/parser/yaml/parser.go:26 +0x25f
github.com/Checkmarx/kics/pkg/parser.(*Parser).Parse(0xc001398de0, 0xc0004203f0, 0x61, 0xc0017cb710, 0x21, 0x30, 0xc0010c1e00, 0xc0014134e0, 0x106ac48, 0xc0010c1e00, ...)
/home/runner/work/kics/kics/pkg/parser/parser.go:110 +0x119
github.com/Checkmarx/kics/pkg/kics.(*Service).sink(0xc000922380, 0x349d398, 0xc0001a0000, 0xc0004203f0, 0x61, 0x3091ea4, 0x7, 0x3454c80, 0xc001d86bf8, 0x0, ...)
/home/runner/work/kics/kics/pkg/kics/sink.go:23 +0xd7
github.com/Checkmarx/kics/pkg/kics.(*Service).PrepareSources.func1(0x349d398, 0xc0001a0000, 0xc0004203f0, 0x61, 0x346bf78, 0xc001d86bf8, 0xffffffffffffffff, 0xc0004203f0)
/home/runner/work/kics/kics/pkg/kics/service.go:63 +0xd0
github.com/Checkmarx/kics/pkg/engine/provider.(*FileSystemSourceProvider).walkDir.func1(0xc0004203f0, 0x61, 0x34ad648, 0xc0020eb110, 0x0, 0x0, 0x0, 0x0)
/home/runner/work/kics/kics/pkg/engine/provider/filesystem.go:151 +0x554
path/filepath.walk(0xc0004203f0, 0x61, 0x34ad648, 0xc0020eb110, 0xc001413e30, 0x0, 0x0)
/opt/hostedtoolcache/go/1.16.8/x64/src/path/filepath/path.go:414 +0x457
path/filepath.walk(0xc0021a36e0, 0x55, 0x34ad648, 0xc0020ea820, 0xc001413e30, 0x0, 0x0)
/opt/hostedtoolcache/go/1.16.8/x64/src/path/filepath/path.go:438 +0x31b
path/filepath.walk(0xc0014d6e60, 0x41, 0x34ad648, 0xc0020ea680, 0xc001413e30, 0x0, 0x0)
/opt/hostedtoolcache/go/1.16.8/x64/src/path/filepath/path.go:438 +0x31b
path/filepath.walk(0xc0012f2980, 0x39, 0x34ad648, 0xc00174ca90, 0xc001413e30, 0x0, 0x0)
/opt/hostedtoolcache/go/1.16.8/x64/src/path/filepath/path.go:438 +0x31b
path/filepath.walk(0xc001772a00, 0x31, 0x34ad648, 0xc001005a00, 0xc001413e30, 0x0, 0x0)
/opt/hostedtoolcache/go/1.16.8/x64/src/path/filepath/path.go:438 +0x31b
path/filepath.walk(0xc000058840, 0x27, 0x34ad648, 0xc00172a0d0, 0xc001413e30, 0x0, 0xc000fec000)
/opt/hostedtoolcache/go/1.16.8/x64/src/path/filepath/path.go:438 +0x31b
path/filepath.Walk(0xc000058840, 0x27, 0xc000c37e30, 0x0, 0xc00172a000)
/opt/hostedtoolcache/go/1.16.8/x64/src/path/filepath/path.go:501 +0x113
github.com/Checkmarx/kics/pkg/engine/provider.(*FileSystemSourceProvider).walkDir(0xc0007b15a0, 0x349d398, 0xc0001a0000, 0xc000058840, 0x27, 0x0, 0xc00035e0e0, 0xc00035e3e0, 0xc001398db0, 0x0, ...)
/home/runner/work/kics/kics/pkg/engine/provider/filesystem.go:119 +0xc5
github.com/Checkmarx/kics/pkg/engine/provider.(*FileSystemSourceProvider).GetSources(0xc0007b15a0, 0x349d398, 0xc0001a0000, 0xc001398db0, 0xc00035e0e0, 0xc00035e3e0, 0xc000c37fa8, 0x17c1fd0)
/home/runner/work/kics/kics/pkg/engine/provider/filesystem.go:108 +0x29d
github.com/Checkmarx/kics/pkg/kics.(*Service).PrepareSources(0xc000922380, 0x349d398, 0xc0001a0000, 0x3091ea4, 0x7, 0xc001d82460, 0xc00037d980)
/home/runner/work/kics/kics/pkg/kics/service.go:59 +0x164
created by github.com/Checkmarx/kics/pkg/scanner.PrepareAndScan
/home/runner/work/kics/kics/pkg/scanner/scanner.go:24 +0x179

Steps to Reproduce the Problem

1. step 1
   run following command
   ./kics_1.3.5/kics -s scan --no-progress --minimal-ui -q -p -o --output-name
   OR
   ./kics_1.4.3/kics -s scan --no-progress --minimal-ui -q -p -o --output-name
2. step 2
3. step 3

Specifications

• Version: 1.3.5 & 1.4.3
• Platform: Linux
• Subsystem:RHEL 7

[ticteam]

I have the same issue, starting at:
panic: interface conversion: interface {} is string, not map[string]interface {} [recovered]
panic: interface conversion: interface {} is string, not map[string]interface {}

/usr/bin/kics scan --no-color -p _presets.yaml --disable-full-descriptions --output-path results --report-formats html

cat _presets.yaml
../../folder-helm/templates/_presets.yaml

Specifications

Version: Scanning with Keeping Infrastructure as Code Secure 1.4.3
Platform: ubuntu 20
Subsystem: Jenkins 2.303.1

UPDATE:
if think the kics scanner can not follow file links like "../../second-helm/templates/_presets.yaml"

I have 2 helm charts in the folder which I want to scan, and the _presets.yaml is calling / directing to another one

cat /workspace/infra/deployment/access-helm/charts/first-helm/templates/_presets.yaml
../../second-helm/templates/_presets.yaml

as the content is exactly the same

/workspace/infra/deployment/access-helm/charts/second-helm/templates/_presets.yaml

[joaoReigota1]

Hi @ppanchal10, regarding your issue it seems that we were not checking if the conversion to model.Document was possible, hence the panic. I believe this issue will be fixed with #4307.
For the second panic which @ticteam is also getting it seems to be the same problem where the conversion was not checked, in the yaml parser. This issue was already fixed in #4224 and should be available in the nightly version of KICS

[joaoReigota1]

I have the same issue, starting at: panic: interface conversion: interface {} is string, not map[string]interface {} [recovered] panic: interface conversion: interface {} is string, not map[string]interface {}

/usr/bin/kics scan --no-color -p _presets.yaml --disable-full-descriptions --output-path results --report-formats html

cat _presets.yaml ../../folder-helm/templates/_presets.yaml

Specifications

Version: Scanning with Keeping Infrastructure as Code Secure 1.4.3
Platform: ubuntu 20
Subsystem: Jenkins 2.303.1

if think it is "normal" that the kics scan can not follow links :)

Hi @ticteam, I didn't understand when you mentioned the kics scan can not follow links, can you please elaborate a little more so we can fix the issue?

[ppanchal10]

Hi @joaoReigota1,

I have run the scan with 1.3.5 & 1.4.3 KICS version on the same set of projects and here are my findings.

1. The issue "panic: interface conversion: interface {} is string, not model.Document" which is appearing with 1.3.5 has been fixed in latest version 1.4.3. The failing project contains Terraform files

2. The issue "panic: interface conversion: interface {} is string, not map[string]interface {} [recovered]
   panic: interface conversion: interface {} is string, not map[string]interface {}" which was not there in version 1.3.5 now appearing in 1.4.3 and this is project is mostly contains YAML files.

If these issues are related to some specific source file then is it possible to skip the files and continue with the rest of files scan and avoid the failing of entire scan.

I am assuming reported issue #1 which has been fixed under #4224 (#4223) and #2 will be fixed under #4307. Do we have any timeline when the fixed will be released?

Thank you.

[joaoReigota1]

Hi @joaoReigota1,

I have run the scan with 1.3.5 & 1.4.3 KICS version on the same set of projects and here are my findings.

1. The issue "panic: interface conversion: interface {} is string, not model.Document" which is appearing with 1.3.5 has been fixed in latest version 1.4.3. The failing project contains Terraform files
2. The issue "panic: interface conversion: interface {} is string, not map[string]interface {} [recovered]
   panic: interface conversion: interface {} is string, not map[string]interface {}" which was not there in version 1.3.5 now appearing in 1.4.3 and this is project is mostly contains YAML files.

If these issues are related to some specific source file then is it possible to skip the files and continue with the rest of files scan and avoid the failing of entire scan.

I am assuming reported issue #1 which has been fixed under #4224 (#4223) and #2 will be fixed under #4307. Do we have any timeline when the fixed will be released?

Thank you.

Yes, we do it will be released this Wednesday!