Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add parser for Sonarqube JSON result. #9366

Merged
merged 13 commits into from
Jan 31, 2024
11 changes: 7 additions & 4 deletions docs/content/en/integrations/parsers/file/sonarqube.md
Original file line number Diff line number Diff line change
@@ -1,24 +1,27 @@
---

Check warning on line 1 in docs/content/en/integrations/parsers/file/sonarqube.md

View check run for this annotation

DryRunSecurity / AI-powered Sensitive Function Check

Possible Sensitive Function

Our AI-Powered Sensitive Function checker believes it has discovered a sensitive function being modified in this PR. The name of the function is `none`. Extra care must be taken when modifying a function that is potentially security-sensitive. The following reason was provided for why this function was flagged as sensitive: The size of the files being changed in this pull request is too large. We are working on increasing that limit. Stay tuned for more...

Check warning on line 1 in docs/content/en/integrations/parsers/file/sonarqube.md

View check run for this annotation

DryRunSecurity / AI-powered Sensitive Files Check

Possible Sensitive File

Our AI-Powered Sensitive File checker believes it has discovered a sensitive file being modified in this PR. Extra care must be taken when modifying a file that is potentially security-sensitive. The following reason was provided: Contains sensitive information about SonarQube integration and file parsing
title: "SonarQube"
toc_hide: true
---
## SonarQube Scan (Aggregates findings per cwe, title, description, file\_path.)

SonarQube output file can be imported in HTML format.
SonarQube output file can be imported in HTML format or JSON format. JSON format generated by options `--save-report-json` and have same behavior with HTML format.

To generate the report, see
<https://github.com/soprasteria/sonar-report>

Version: \>= 1.1.0
Recommend version for both format \>= 3.1.2

## SonarQube Scan Detailed (Import all findings from SonarQube html report.)

SonarQube output file can be imported in HTML format.
SonarQube output file can be imported in HTML format or JSON format. JSON format generated by options `--save-report-json` and have same behavior with HTML format.

To generate the report, see
<https://github.com/soprasteria/sonar-report>

Version: \>= 1.1.0
Version: \>= 1.1.0.
Recommend version for both format \>= 3.1.2


### Sample Scan Data
Sample SonarQube scans can be found [here](https://github.com/DefectDojo/django-DefectDojo/tree/master/unittests/scans/sonarqube).
Sample SonarQube scans can be found [here](https://github.com/DefectDojo/django-DefectDojo/tree/master/unittests/scans/sonarqube).
90 changes: 79 additions & 11 deletions dojo/tools/sonarqube/parser.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
import logging

Check warning on line 1 in dojo/tools/sonarqube/parser.py

View check run for this annotation

DryRunSecurity / AI-powered Sensitive Files Check

Possible Sensitive File

Our AI-Powered Sensitive File checker believes it has discovered a sensitive file being modified in this PR. Extra care must be taken when modifying a file that is potentially security-sensitive. The following reason was provided: Contains sensitive code for parsing SonarQube data
import re

from django.utils.html import strip_tags
from lxml import etree
import json

from dojo.models import Finding

Expand All @@ -23,21 +24,88 @@

def get_description_for_scan_types(self, scan_type):
if scan_type == "SonarQube Scan":
return "Aggregates findings per cwe, title, description, file_path. SonarQube output file can be imported in HTML format. Generate with https://github.com/soprasteria/sonar-report version >= 1.1.0"
return "Aggregates findings per cwe, title, description, file_path. SonarQube output file can be imported in HTML format or JSON format. Generate with https://github.com/soprasteria/sonar-report version >= 1.1.0, recommend version >= 3.1.2"
else:
return "Import all findings from sonarqube html report. SonarQube output file can be imported in HTML format. Generate with https://github.com/soprasteria/sonar-report version >= 1.1.0"
return "Import all findings from sonarqube html report or JSON format. SonarQube output file can be imported in HTML format or JSON format. Generate with https://github.com/soprasteria/sonar-report version >= 1.1.0, recommend version >= 3.1.2"

def get_findings(self, filename, test):
parser = etree.HTMLParser()
tree = etree.parse(filename, parser)
if self.mode not in [None, "detailed"]:
raise ValueError(
"Internal error: Invalid mode "
+ self.mode
+ ". Expected: one of None, 'detailed'"
)
if filename.name.strip().lower().endswith(".json"):
json_content = json.load(filename)
return self.get_json_items(json_content, test, self.mode)
else:
parser = etree.HTMLParser()
tree = etree.parse(filename, parser)
if self.mode not in [None, "detailed"]:
raise ValueError(
"Internal error: Invalid mode "
+ self.mode
+ ". Expected: one of None, 'detailed'"
)

return self.get_items(tree, test, self.mode)

return self.get_items(tree, test, self.mode)
def get_json_items(self, json_content, test, mode):
dupes = dict()
rules = json_content["rules"]
issues = json_content["issues"]
for issue in issues:
key = issue["key"]
line = str(issue["line"])
mitigation = issue["message"]
title = issue["description"]
file_path = issue["component"]
severity = self.convert_sonar_severity(issue["severity"])
rule_id = issue["rule"]

if title is None or mitigation is None:
raise ValueError(
"Parser ValueError: can't find a title or a mitigation for vulnerability of name "
+ rule_id
)

try:
issue_detail = rules[rule_id]
parser = etree.HTMLParser()
html_desc_as_e_tree = etree.fromstring(issue_detail["htmlDesc"], parser)
issue_description = self.get_description(html_desc_as_e_tree)
logger.debug(issue_description)
issue_references = self.get_references(
rule_id, html_desc_as_e_tree
)
issue_cwe = self.get_cwe(issue_references)
except KeyError:
issue_description = "No description provided"
issue_references = ""
issue_cwe = 0

if mode is None:
self.process_result_file_name_aggregated(
test,
dupes,
title,
issue_cwe,
issue_description,
file_path,
line,
severity,
mitigation,
issue_references,
)
else:
self.process_result_detailed(
test,
dupes,
title,
issue_cwe,
issue_description,
file_path,
line,
severity,
mitigation,
issue_references,
key,
)
return list(dupes.values())

def get_items(self, tree, test, mode):
# Check that there is at least one vulnerability (the vulnerabilities
Expand Down
Loading
Loading