cxx:FileHeader not validated when using regular expression

[lg2de]

Describe the bug
We have configured a regular expression for the rule cxx:FileHeader:

//\s*<copyright>\s*//\s*Copyright \(c\) (Kontron AIS|AIS Automation Dresden) GmbH. All rights reserved.\s*//\s*</copyright>\s*

Recently we have changed the name of the company. So, for some time we have two different valid headers. The current should look like this:

// <copyright>
//     Copyright (c) Kontron AIS GmbH. All rights reserved.
// </copyright>

I guess the expression is correct because the same expression is working fine for C# files (csharpsquid:S1451).

Anyway the CPP files shows the issue:

To Reproduce
Unfortunately I do not know how to reproduce.
Maybe it is related to the multiline expression?

Expected behavior
The header should be accepted in old and new format.

Desktop:

• OS: Windows Server 2019
• SonarQube version: 8.7.0
• cxx plugin version: 1.3.3
• sonar-scanner version: SonarScanner for MSBuild 5.0.4

Please help to identify the root cause. What information do you need?

[guwirth]

Hi @lg2de,

the check is using the Java regex format.

• Source code:

  sonar-cxx/cxx-checks/src/main/java/org/sonar/cxx/checks/regex/FileHeaderCheck.java

  Line 124 in 046073c

  private void checkRegularExpression(String fileContent) {

• Unit tests: https://github.com/SonarOpenCommunity/sonar-cxx/blob/master/cxx-checks/src/test/java/org/sonar/cxx/checks/regex/FileHeaderCheckTest.java
• Samples Header: https://github.com/SonarOpenCommunity/sonar-cxx/tree/master/cxx-checks/src/test/resources/checks/FileHeaderCheck

See also https://www.baeldung.com/java-regexp-escape-char, you have to escape the metacharacters <([{\^-=$!|]})?*+.>. This includes the backslash character \ => \\!

Regards,

[guwirth]

@lg2de see also https://github.com/SonarOpenCommunity/sonar-cxx/wiki/CXX-Rules#regex-rules

[lg2de]

Thanks @guwirth for your comments.
I've tested expression and text with regex101.com using Java flavor according to the wiki. I've generated the java code and used the generated string as input in the SQ configuration.

//\\s*<copyright>\\s*//\\s*Copyright \\(c\\) (Kontron AIS|AIS Automation Dresden) GmbH. All rights reserved.\\s*//\\s*</copyright>\\s*

But the issue is still not solved. The header is not accepted.

[guwirth]

Hi @lg2de,

I wrote an unit test with your sample. In the test everything is working (#2121).

Now could still be due to the following:

1. isRegularExpression is not true
2. file encoding: Are source files UTF-8 or something different?
3. UI magic: That backslashes are added or removed when typing in the UI? Maybe try \ instead of \\, maybe the doubling is done automatically?

Regards,

[lg2de]

1. The parameter is true as you can see in the initial screenshot.
2. The problem applies to files with UTF8 encoding and without, e.g. simple ASCII.
3. I also tried again with only one backslash.

I opened this issue because I already tried before "everything" to find the root cause for myself. But I failed e.g. because I'm not able to compile the plugin locally.
Would it be possible that you (or someone else) tests the constellation end-to-end, means as configuration in the SQ UI and a real analysis using SonarScanner.
In the past I had already an issue that the transport of the configuration from SQ to analyzer failed due to encoding issue. Maybe this problem is close!?

[guwirth]

@lg2de that is really strange. Next step I could do is to setup an integration test. That would be an "end-to-end" test. Could you provide a source code file (file with header and at least one line of code). Upload it "somewhere" and provide a link here. Only wanna be sure I'm using same encoding, line endings, ...

The problem applies to files with UTF8 encoding and without, e.g. simple ASCII.

Did you try setting sonar.sourceEncoding=UTF-8?

[lg2de]

All files I checked are either UTF8 with BOM or UTF8 without BOM or simple ASCII.
The line ending is Windows standard \r and \n.

It is not that easy to provide sample code files. I cannot share internal code because it is protected.
As soon as I change the file, the preconditions are changed, I guess.
Anyway I'll try to setup a demo project in our testing environment. As soon I have some (the same) result, I'll share the project.

BTW, it would be great to have "all" combination of encoding and line ending in the integration test(s).

sonar.sourceEncoding=UTF-8 is already activated.

[guwirth]

Hi @lg2de,

I did some tests and found problems with files with BOM. I fixed it in #2121 for cxx plugin 2.0.

Regards,

[lg2de]

Thanks for taking care!!
I've loaded the plugin from the build after your merge to test it in our test environment.
The next analysis did not identified issues anymore. I'm using cppcheck (recently updated to 2.4.1). Is there any known issue or breaking change from 1.X to 2.X?

[guwirth]

Hi @lg2de,

please read https://github.com/SonarOpenCommunity/sonar-cxx/wiki/Upgrade-the-Plugin, there are some changes with 2.0. Please turn debug info on during scanning, there should be according hints.

Regards,

[guwirth]

Hi @lg2de,

we also added an integration test with files with/without BOM and different line endings. They are working we can’t reproduce the problem.

• https://github.com/SonarOpenCommunity/sonar-cxx/tree/master/integration-tests/testdata/regex_project
• https://github.com/SonarOpenCommunity/sonar-cxx/blob/master/integration-tests/features/regex.feature

Please try to isolate the problem on your system:

• on which files with which encoding does the problem occur?
• only with files with/without BOM?
• simplify the regex and search only for initial text, does it work then ( e.g. first line only)
  • //\s*<copyright>
  • // <copyright>
• does a normal header without regex work?
  • // <copyright> with isRegularExpression=true/false

Regards

[guwirth]

@lg2de I still have no idea what the issue here could be. We did a fix for header files maybe that helps (#2142)?
https://ci.appveyor.com/project/SonarOpenCommunity/sonar-cxx/branch/master/artifacts

[lg2de]

With the latest version sonar-cxx-plugin-2.0.1.2662.jar the header check seems to be ok now.