not all GCC warnings are detected with the default sonar.cxx.gcc.regex #1703

stalb · 2019-04-15T14:11:46Z

Please turn debug info on and reproduce your issue.
The debug info in the log file could help to solve or locate the issue.

Description

Some gcc warnings are not reported in SonarQube by the Community SonarQube C plugin

Steps to reproduce the problem

On SonarQube instance activate all gcc rules
Create a C program which produces some warnings:
main.c

/* test file */

/* missing semicolon at end of struct or union */
typedef struct {
	int a;
	int b
} my_struct;

int main(){
	/* unused variable */
	int i;
	return 0;
}

Produce report using gcc

main.c:7:1: warning: no semicolon at end of struct or union
 } my_struct;
 ^
main.c: In function 'main':
main.c:11:6: warning: unused variable 'i' [-Wunused-variable]
  int i;
      ^

Send report to SonarQube instance

Exemple repo (CI is defined for gitlab) is here: https://github.com/stalb/demo_missing_rule_sonar_cxx_plugin

Expected behavior

Community SonarQube C plugin and SonarQube should report 2 issues on main.c.

Actual behavior

Only the 2nd warning (unused variable) is reported.
In fact because of the , the Community SonarQube C plugin detects only the 2nd one.

12:50:52.874 INFO: Parsing 'GCC' initialized with report '/builds/stalb/demo_missing_rule_sonar_cxx_plugin/gcc_report.log', Charset= 'UTF-8'
12:50:52.874 INFO: Using pattern : '(?<file>.*):(?<line>[0-9]+):[0-9]+:\x20warning:\x20(?<message>.*)\x20\[(?<id>.*)\]'
12:50:52.877 DEBUG: Scanner-matches file='main.c' line='11' id='-Wunused-variable' msg=unused variable 'i'
12:50:52.902 DEBUG: C-COMPILER-GCC processed = 1
12:50:52.902 INFO: C-COMPILER-GCC processed = 1
12:50:52.903 INFO: Sensor C (Community) CxxCompilerGccSensor [c] (done) | time=56ms

Known workarounds

none (the id part is missing in the 1st warning)

LOG file

Any warnings/errors in the LOG file? None

12:50:50.330 INFO: Load active rules (done) | time=1851ms
12:50:50.332 INFO: Load metrics repository
12:50:50.347 DEBUG: GET 200 https://SonarCE.instance.org/sonar/api/metrics/search?f=name,description,direction,qualitative,custom&ps=500&p=1 | time=15ms
12:50:50.362 INFO: Load metrics repository (done) | time=30ms
12:50:50.424 INFO: Branch name: master, type: long living
12:50:50.483 DEBUG: Declared extensions of language Python were converted to sonar.lang.patterns.py : **/*.py
12:50:50.483 DEBUG: Declared extensions of language C (Community) were converted to sonar.lang.patterns.c : **/*.c,**/*.h
12:50:50.484 DEBUG: Declared extensions of language JavaScript were converted to sonar.lang.patterns.js : **/*.js,**/*.jsx,**/*.vue
12:50:50.484 DEBUG: Declared extensions of language C# were converted to sonar.lang.patterns.cs : **/*.cs
12:50:50.484 DEBUG: Declared extensions of language Java were converted to sonar.lang.patterns.java : **/*.java,**/*.jav
12:50:50.484 DEBUG: Declared extensions of language Flex were converted to sonar.lang.patterns.flex : **/*.as
12:50:50.484 DEBUG: Declared extensions of language XML were converted to sonar.lang.patterns.xml : **/*.xml,**/*.xsd,**/*.xsl
12:50:50.485 DEBUG: Declared extensions of language PHP were converted to sonar.lang.patterns.php : **/*.php,**/*.php3,**/*.php4,**/*.php5,**/*.phtml,**/*.inc
12:50:50.485 DEBUG: Declared extensions of language TypeScript were converted to sonar.lang.patterns.ts : **/*.ts,**/*.tsx
12:50:50.485 INFO: Language is forced to c
12:50:50.489 INFO: Indexing files...
12:50:50.489 INFO: Project configuration:
12:50:50.491 INFO:   Excluded sources: gcc_report.log, clangtidy_report.log
12:50:50.510 DEBUG: 'main.c' indexed with language 'c'
12:50:50.510 WARN: File '/builds/stalb/demo_missing_rule_sonar_cxx_plugin/sonar-project.properties' is ignored because it doesn't belong to the forced language 'c'
12:50:50.511 INFO: 1 file indexed
12:50:50.511 INFO: 2 files ignored because of inclusion/exclusion patterns
12:50:50.515 INFO: Quality profile for c: Sonar extended way
12:50:50.516 INFO: ------------- Run sensors on module demo_missing_rule_sonar_cxx_plugin
12:50:52.553 DEBUG: 'JavaSquidSensor' skipped because there is no related file in current project
12:50:52.554 DEBUG: 'Import external issues report' skipped because one of the required properties is missing
12:50:52.555 DEBUG: 'Python Squid Sensor' skipped because there is no related file in current project
12:50:52.555 DEBUG: 'Cobertura Sensor for Python coverage' skipped because there is no related file in current project
12:50:52.555 DEBUG: 'PythonXUnitSensor' skipped because there is no related file in current project
12:50:52.555 DEBUG: 'PylintSensor' skipped because there is no related file in current project
12:50:52.556 DEBUG: 'PylintImportSensor' skipped because there is no related file in current project
12:50:52.557 DEBUG: 'Import of Bandit issues' skipped because there is no related file in current project
12:50:52.558 DEBUG: 'C (Community) RatsSensor' skipped because one of the required properties is missing
12:50:52.559 DEBUG: 'C (Community) CppCheckSensor' skipped because one of the required properties is missing
12:50:52.559 DEBUG: 'C (Community) PCLintSensor' skipped because there is no related rule activated in the quality profile
12:50:52.560 DEBUG: 'C (Community) DrMemorySensor' skipped because there is no related rule activated in the quality profile
12:50:52.561 DEBUG: 'C (Community) CxxCompilerVcSensor' skipped because there is no related rule activated in the quality profile
12:50:52.562 DEBUG: 'C (Community) VeraxxSensor' skipped because one of the required properties is missing
12:50:52.563 DEBUG: 'C (Community) ValgrindSensor' skipped because one of the required properties is missing
12:50:52.564 DEBUG: 'C (Community) ClangSASensor' skipped because one of the required properties is missing
12:50:52.565 DEBUG: 'C (Community) ExternalRulesSensor' skipped because there is no related rule activated in the quality profile
12:50:52.566 DEBUG: 'C (Community) CoverageSensor' skipped because one of the required properties is missing
12:50:52.567 DEBUG: 'SonarJS' skipped because there is no related file in current project
12:50:52.567 DEBUG: 'ESLint-based SonarJS' skipped because there is no related file in current project
12:50:52.568 DEBUG: 'SonarJS Coverage' skipped because there is no related file in current project
12:50:52.568 DEBUG: 'Import of ESLint issues' skipped because one of the required properties is missing
12:50:52.568 DEBUG: 'C# Properties' skipped because there is no related file in current project
12:50:52.570 DEBUG: 'Import of Checkstyle issues' skipped because there is no related file in current project
12:50:52.571 DEBUG: 'Import of PMD issues' skipped because one of the required properties is missing
12:50:52.571 DEBUG: 'Import of SpotBugs issues' skipped because there is no related file in current project
12:50:52.571 DEBUG: 'SurefireSensor' skipped because there is no related file in current project
12:50:52.572 DEBUG: 'JaCoCoSensor' skipped because there is no related file in current project
12:50:52.573 DEBUG: 'Flex' skipped because there is no related file in current project
12:50:52.573 DEBUG: 'Flex Cobertura' skipped because there is no related file in current project
12:50:52.573 DEBUG: 'XML Sensor' skipped because there is no related file in current project
12:50:52.574 DEBUG: 'PHP sensor' skipped because there is no related file in current project
12:50:52.574 DEBUG: 'Analyzer for "php.ini" files' skipped because there is no related file in current project
12:50:52.574 DEBUG: 'SonarTS' skipped because there is no related file in current project
12:50:52.575 DEBUG: 'SonarTS Coverage' skipped because there is no related file in current project
12:50:52.576 DEBUG: 'Import of TSLint issues' skipped because one of the required properties is missing
12:50:52.581 DEBUG: 'Generic Coverage Report' skipped because one of the required properties is missing
12:50:52.581 DEBUG: 'Generic Test Executions Report' skipped because one of the required properties is missing
12:50:52.582 DEBUG: 'C (Community) XunitSensor' skipped because one of the required properties is missing
12:50:52.582 DEBUG: 'C#' skipped because there is no related file in current project
12:50:52.582 DEBUG: 'C# Tests Coverage Report Import' skipped because there is no related file in current project
12:50:52.582 DEBUG: '[Deprecated] C# Integration Tests Coverage Report Import' skipped because there is no related file in current project
12:50:52.583 DEBUG: 'C# Unit Test Results Import' skipped because there is no related file in current project
12:50:52.583 DEBUG: Sensors : C (Community) SquidSensor -> C (Community) CxxCompilerGccSensor -> C (Community) ClangTidySensor -> JavaXmlSensor -> Zero Coverage Sensor
12:50:52.585 INFO: Sensor C (Community) SquidSensor [c]
12:50:52.592 WARN: Property 'sonar.c.includeDirectories' is not declared as multi-values/property set but was read using 'getStringArray' method. The SonarQube plugin declaring this property should be updated.
12:50:52.593 WARN: Property 'sonar.c.forceIncludes' is not declared as multi-values/property set but was read using 'getStringArray' method. The SonarQube plugin declaring this property should be updated.
12:50:52.593 WARN: Property 'sonar.c.cFilesPatterns' is not declared as multi-values/property set but was read using 'getStringArray' method. The SonarQube plugin declaring this property should be updated.
12:50:52.703 DEBUG: Cyclomatic complexity threshold: 10
12:50:52.704 DEBUG: Function size threshold: 20
12:50:52.718 DEBUG: global settings for: '/builds/stalb/demo_missing_rule_sonar_cxx_plugin/main.c'
12:50:52.718 DEBUG: Parse 'main.c' as C file, ends in '.c'
12:50:52.760 DEBUG: API File: main.c
12:50:52.760 DEBUG: Header file suffixes: [.h]
12:50:52.760 DEBUG: finished preprocessing '/builds/stalb/demo_missing_rule_sonar_cxx_plugin/main.c'
12:50:52.778 DEBUG: 'main.c' generated metadata with charset 'UTF-8'
12:50:52.802 DEBUG: Not enough content in 'main.c' to have CPD blocks, it will not be part of the duplication detection
12:50:52.810 WARN: Metric 'comment_lines_data' is deprecated. Provided value is ignored.
12:50:52.823 DEBUG: CxxFileLinesVisitor: '/builds/stalb/demo_missing_rule_sonar_cxx_plugin/main.c'
12:50:52.823 DEBUG:    lines:           '13'
12:50:52.823 DEBUG:    executableLines: '[12]'
12:50:52.823 DEBUG:    linesOfCode:     '[9, 11, 12]'
12:50:52.823 DEBUG:    linesOfComments: '[1, 3, 10]'
12:50:52.847 INFO: Sensor C (Community) SquidSensor [c] (done) | time=263ms
12:50:52.847 INFO: Sensor C (Community) CxxCompilerGccSensor [c]
12:50:52.847 INFO: Searching reports by relative path with basedir '/builds/stalb/demo_missing_rule_sonar_cxx_plugin' and search prop 'sonar.c.gcc.reportPath'
12:50:52.850 DEBUG: Normalized report includes to '[/builds/stalb/demo_missing_rule_sonar_cxx_plugin/gcc_report.log]'
12:50:52.850 DEBUG: Scanner uses normalized report path(s): '/builds/stalb/demo_missing_rule_sonar_cxx_plugin/gcc_report.log'
12:50:52.873 INFO: Parser will parse '1' report file(s)
12:50:52.873 INFO: Processing report '/builds/stalb/demo_missing_rule_sonar_cxx_plugin/gcc_report.log'
12:50:52.874 INFO: Parsing 'GCC' initialized with report '/builds/stalb/demo_missing_rule_sonar_cxx_plugin/gcc_report.log', Charset= 'UTF-8'
12:50:52.874 INFO: Using pattern : '(?<file>.*):(?<line>[0-9]+):[0-9]+:\x20warning:\x20(?<message>.*)\x20\[(?<id>.*)\]'
12:50:52.877 DEBUG: Scanner-matches file='main.c' line='11' id='-Wunused-variable' msg=unused variable 'i'
12:50:52.902 DEBUG: C-COMPILER-GCC processed = 1
12:50:52.902 INFO: C-COMPILER-GCC processed = 1
12:50:52.903 INFO: Sensor C (Community) CxxCompilerGccSensor [c] (done) | time=56ms
12:50:52.903 INFO: Sensor C (Community) ClangTidySensor [c]
12:50:52.903 INFO: Searching reports by relative path with basedir '/builds/stalb/demo_missing_rule_sonar_cxx_plugin' and search prop 'sonar.c.clangtidy.reportPath'
12:50:52.904 DEBUG: Normalized report includes to '[/builds/stalb/demo_missing_rule_sonar_cxx_plugin/clangtidy_report.log]'
12:50:52.904 DEBUG: Scanner uses normalized report path(s): '/builds/stalb/demo_missing_rule_sonar_cxx_plugin/clangtidy_report.log'
12:50:52.904 INFO: Parser will parse '1' report file(s)
12:50:52.904 INFO: Processing report '/builds/stalb/demo_missing_rule_sonar_cxx_plugin/clangtidy_report.log'
12:50:52.905 DEBUG: Parsing 'clang-tidy' report, CharSet= 'UTF-8'
12:50:52.907 DEBUG: C-CLANG-TIDY processed = 2
12:50:52.907 INFO: C-CLANG-TIDY processed = 2
12:50:52.907 INFO: Sensor C (Community) ClangTidySensor [c] (done) | time=4ms
12:50:52.907 INFO: Sensor JavaXmlSensor [java]
12:50:52.909 INFO: Sensor JavaXmlSensor [java] (done) | time=2ms
12:50:52.909 INFO: Sensor Zero Coverage Sensor
12:50:52.927 INFO: Sensor Zero Coverage Sensor (done) | time=18ms
12:50:52.930 INFO: ------------- Run sensors on project
12:50:52.938 DEBUG: 'Java CPD Block Indexer' skipped because there is no related file in current project
12:50:52.938 DEBUG: Sensors : 
12:50:52.943 INFO: SCM provider for this project is: git
12:50:52.944 INFO: 1 files to be analyzed
12:50:53.014 DEBUG: Blame file main.c
12:50:53.166 INFO: 1/1 files analyzed
12:50:53.169 INFO: 1 file had no CPD blocks
12:50:53.169 INFO: Calculating CPD for 0 files
12:50:53.170 INFO: CPD calculation finished
12:50:53.350 INFO: Analysis report generated in 179ms, dir size=234 KB
12:50:53.370 INFO: Analysis report compressed in 20ms, zip size=35 KB
12:50:53.371 INFO: Analysis report generated in /builds/stalb/demo_missing_rule_sonar_cxx_plugin/.scannerwork/scanner-report
12:50:53.371 DEBUG: Upload report
12:50:53.406 DEBUG: POST 200 https://SonarCE.instance.org/sonar/api/ce/submit?projectKey=demo_missing_rule_sonar_cxx_plugin&projectName=demo_missing_rule_sonar_cxx_plugin&characteristic=branch%3Dmaster&characteristic=branchType%3DLONG | time=33ms
12:50:53.408 INFO: Analysis report uploaded in 37ms
12:50:53.410 INFO: ANALYSIS SUCCESSFUL, you can browse https://SonarCE.instance.org/sonar/dashboard?id=demo_missing_rule_sonar_cxx_plugin&branch=master
12:50:53.411 INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
12:50:53.411 INFO: More about the report processing at https://SonarCE.instance.org/sonar/api/ce/task?id=AWohDOFU4uFZCednbAYV
12:50:53.412 DEBUG: Report metadata written to /builds/stalb/demo_missing_rule_sonar_cxx_plugin/.scannerwork/report-task.txt
12:50:53.553 DEBUG: 'GitLab Commit Issue Publisher' skipped because one of the required properties is missing
12:50:53.555 DEBUG: Post-jobs : Final report
12:50:53.555 INFO: Executing post-job 'Final report'
12:50:53.555 WARN: Source code parser: 2 syntax error(s) detected. Syntax errors could cause invalid software metric values. Root cause are typically missing includes, missing macros or compiler specific extensions.
12:50:53.558 INFO: Analysis total time: 6.184 s
12:50:53.644 INFO: ------------------------------------------------------------------------
12:50:53.644 INFO: EXECUTION SUCCESS
12:50:53.644 INFO: ------------------------------------------------------------------------
12:50:53.645 INFO: Total time: 13.045s
12:50:53.779 INFO: Final Memory: 24M/342M
12:50:53.780 INFO: ------------------------------------------------------------------------
Job succeeded

Related information

c plugin version: 1.2.2 (build 1653)
SonarQube version: 7.6.0.21501

The text was updated successfully, but these errors were encountered:

guwirth · 2019-04-16T05:46:16Z

Hello @stalb,

thanks for your feedback.

Which version of GCC are you using?
The current list of GCC messages is here: https://github.com/SonarOpenCommunity/sonar-cxx/blob/master/cxx-sensors/src/main/resources/compiler-gcc.xml

Maybe you can provide an update?

Regards,

stalb · 2019-04-16T10:35:14Z

GCC version : 8.3.0

The problem is that the warning message is missing the id part, so the regex doesn't match and there is no id, so no rule to match...

guwirth · 2019-04-16T11:00:57Z

Hi,

Expected behavior: Community SonarQube C plugin and SonarQube should report 2 issues on main.c.

regex can be changed. But because the message id is missing, to which rule should warning be mapped?

Regards,

stalb · 2019-04-16T14:32:05Z

There's a rule which key is "enabled by default" and description "Default compiler warnings".
Seems fine for me.

Is there any reason why all gcc warnings are tagged critical ?

guwirth · 2019-04-19T06:14:31Z

Is there any reason why all gcc warnings are tagged critical ?

@stalb No. Think the initial author of the rule definition did it to have less work. Assigning them to different types and severities is a lot of work. If you have a better proposal would be glad if you could share it here.

guwirth · 2019-04-19T06:27:09Z

@stalb thanks for your PR #1708.

I’m not sure if this is the right solution?

First im wondering why there are GCC warnings without unique ID? I’m not familiar with GCC but all other compilers / static code analyzer seems to have it?
The original idea was that every message of a tool is mapped to an unique rule violation in SQ. Each rule violation have an individual type, severity, effort assigned and can be turned on/off depending on the message ID.
All other sensor would behave different then. Up to now the idea was always to extend the rule definition file in case of a missing ID and create a warning if mapping is not possible.
If someone wanna have a different solution he can override the regex or provide a customized rule definition file.

So my initial question is: Why should we behave different in this sensor?

Regards,

stalb · 2019-04-19T10:58:56Z

In gcc, there are warnings which are not associated with any switch (they cannot be activated or deactivated using a specific switch as -Wuninitialized or -Wno-uninitialized).
I don't know why, but it's like that. They have designed gcc like that.
If it easier for you, think it as core warnings...

The plugin is using the warning switch parameter (used to enable or disable the warning) as the rule id for all the warnings messages enabled by a switch. Ideally all this warnings should have different individual type, severity, effort assigned to each of them, however since they all have the same rule id, they all share the same type, severity, effort assigned.
I understand the logic behind that. However the result is that there are a bunch of warnings (which cannot be disabled, except if you disable all warnings) and the plugin simply and completely discards them.
So you can have code that compiles with warnings and you simply ignore theses warnings...

What I propose is not to ignore these warnings and to map them with a rule that, for now, in not used. This rule as the same type, severity, effort assigned as every other gcc rules. If someone has a better solution, it's ok for me.
The fact is theses warning actually don't have a id, how do you want to cope with that ?

By the way, clang behaves the same. It also has warnings which are not associated with any switch.

guwirth · 2019-04-20T09:31:36Z

@stalb ok let’s try it.

guwirth added the question label Apr 16, 2019

stalb mentioned this issue Apr 16, 2019

support gcc warnings without rule id #1708

Merged

guwirth added enhancement and removed question labels Apr 20, 2019

guwirth added this to the 1.3.0 milestone Apr 20, 2019

guwirth assigned stalb Apr 20, 2019

guwirth closed this as completed in #1708 Apr 25, 2019

guwirth mentioned this issue Jul 28, 2019

Plugin 1.3 release #1740

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

not all GCC warnings are detected with the default sonar.cxx.gcc.regex #1703

not all GCC warnings are detected with the default sonar.cxx.gcc.regex #1703

stalb commented Apr 15, 2019

guwirth commented Apr 16, 2019

stalb commented Apr 16, 2019

guwirth commented Apr 16, 2019

stalb commented Apr 16, 2019

guwirth commented Apr 19, 2019

guwirth commented Apr 19, 2019 •

edited

Loading

stalb commented Apr 19, 2019

guwirth commented Apr 20, 2019

not all GCC warnings are detected with the default sonar.cxx.gcc.regex #1703

not all GCC warnings are detected with the default sonar.cxx.gcc.regex #1703

Comments

stalb commented Apr 15, 2019

Description

Steps to reproduce the problem

Expected behavior

Actual behavior

Known workarounds

LOG file

Related information

guwirth commented Apr 16, 2019

stalb commented Apr 16, 2019

guwirth commented Apr 16, 2019

stalb commented Apr 16, 2019

guwirth commented Apr 19, 2019

guwirth commented Apr 19, 2019 • edited Loading

stalb commented Apr 19, 2019

guwirth commented Apr 20, 2019

guwirth commented Apr 19, 2019 •

edited

Loading