SCANPY-237 Improved documentation and output, especially regarding error reporting. Minor refactoring.

Author: marc-jasper-sonarsource

DRY_RUN_MODE.md

@@ -7,6 +7,32 @@ The Pysonar scanner supports a **dry-run mode** that helps you troubleshoot conf
 - Validating configuration properties
 - Debugging analysis failures related to configuration
 
+## Use Cases
+
+### 1. Validating Configuration Before First Analysis
+
+Use dry-run mode to verify your configuration is correct before running a full analysis. This is especially useful when setting up new projects or making configuration changes:
+
+```bash
+pysonar \
+  --token "myToken" \
+  --project-key "my:project" \
+  --sonar-python-coverage-report-paths "coverage/cobertura.xml" \
+  --dry-run
+```
+
+Or with minimal configuration:
+
+```bash
+pysonar --dry-run -v
+```
+
+This validates all configuration sources (pyproject.toml, environment variables, CLI arguments) and checks that coverage report paths are valid before submitting any analysis. This also helps ensure that environment variables, CLI arguments, and configuration files are being read correctly.
+
+### 2. Troubleshooting Failed Analysis
+
+If an analysis fails, use dry-run mode to quickly identify configuration issues without waiting for a full analysis. This is especially useful for catching problems with configuration files, environment variables, or required properties before attempting a full analysis.
+
 ## Enabling Dry Run Mode
 
 To run the scanner in dry-run mode, add the `--dry-run` flag:
@@ -79,101 +105,44 @@ DRY RUN MODE - Validation Results
 
 ## Coverage Report Validation
 
-The scanner validates coverage reports by checking:
+The scanner performs basic validation of coverage reports by checking:
 
 1. **File existence** - Verifies that the file exists at the specified path
 2. **File readability** - Ensures the file is readable and accessible
-3. **File format** - Validates that coverage reports are in valid Cobertura XML format
+3. **XML format** - Validates that the file can be parsed as valid XML
 4. **Root element** - Checks that XML root element is `<coverage>` (expected Cobertura format)
 
+Note: This is a basic sanity check and does not perform full schema validation of the Cobertura format.
+
 ### Example: Coverage Report Validation Output
 
 Successful validation:
 
 ```
 Coverage Report Paths: coverage.xml
 
-✓ Coverage report is valid Cobertura XML: coverage.xml
+✓ Coverage report check passed: coverage.xml
 ```
 
 Missing file error:
 
 ```
 ✗ Configuration validation FAILED with the following issues:
-  • Coverage report not found: coverage.xml (resolved to /project/coverage.xml)
+✗ Coverage report not found: coverage.xml (resolved to /project/coverage.xml)
 ```
 
 Invalid format error:
 
 ```
 ✗ Configuration validation FAILED with the following issues:
-  • Coverage report is not valid XML (Cobertura format): coverage.xml
-    Parse error: XML not well-formed (invalid token)
+✗ Coverage report is not valid XML (Cobertura format): coverage.xml (Parse error: XML not well-formed (invalid token))
 ```
 
 ## Exit Codes
 
 - **0**: Configuration validation passed, no errors found
 - **1**: Configuration validation failed, errors were found
 
-## Use Cases
-
-### 1. Validating Coverage Report Paths
-
-Before running a full analysis, verify that coverage reports are correctly configured:
-
-```bash
-pysonar \
-  --token "myToken" \
-  --project-key "my:project" \
-  --sonar-python-coverage-report-paths "coverage/cobertura.xml" \
-  --dry-run
-```
-
-### 2. Checking Configuration Resolution
-
-Verify that all configuration sources are properly resolved:
-
-```bash
-# Set configuration in multiple places
-export SONAR_HOST_URL="https://sonarqube.example.com"
-pysonar \
-  --token "myToken" \
-  --project-key "my:project" \
-  --dry-run
-```
-
-This helps ensure that environment variables, CLI arguments, and configuration files are being read correctly.
-
-### 3. Troubleshooting Failed Analysis
-
-If an analysis fails, use dry-run mode to quickly identify configuration issues without waiting for a full analysis:
-
-```bash
-# First, validate the configuration
-pysonar \
-  --token "myToken" \
-  --project-key "my:project" \
-  --sonar-python-coverage-report-paths "coverage/cobertura.xml" \
-  --dry-run
-
-# If successful, run the full analysis
-pysonar \
-  --token "myToken" \
-  --project-key "my:project" \
-  --sonar-python-coverage-report-paths "coverage/cobertura.xml"
-```
-
-### 4. Setting Up New Projects
-
-When onboarding a new project, use dry-run mode to validate the setup before the first full analysis:
-
-```bash
-# Create your configuration in pyproject.toml or via CLI
-# Then validate it:
-pysonar --dry-run -v
-```
-
 ## Common Issues and Solutions
 
 ### Issue: Coverage report not found
@@ -199,20 +168,17 @@ Coverage report is not readable (permission denied): coverage.xml
 **Solution:**
 - Check file permissions: `ls -l coverage.xml`
 - Make the file readable: `chmod 644 coverage.xml`
-- Ensure the process running the scanner has read access
 
 ### Issue: Invalid XML format
 
 **Error message:**
 ```
-Coverage report is not valid XML (Cobertura format): coverage.xml
-  Parse error: XML not well-formed (invalid token)
+Coverage report is not valid XML (Cobertura format): coverage.xml (Parse error: XML not well-formed (invalid token))
 ```
 
 **Solution:**
-- Verify the coverage report was generated correctly
-- Try generating the coverage report again
-- Check the coverage tool documentation for proper output format
+- Verify the coverage report was generated correctly with your coverage tool
+- Check the coverage tool documentation for the proper output format
 
 ### Issue: Wrong root element
 
@@ -223,8 +189,8 @@ Coverage report root element is 'report', expected 'coverage' (Cobertura format)
 
 **Solution:**
 - The coverage report may not be in Cobertura XML format
-- Check that your coverage tool is configured to output Cobertura XML
-- For Python projects using coverage.py, use: `coverage xml`
+- For Python projects using coverage.py, generate the report with: `coverage xml`
+- Check that your coverage tool is configured to output Cobertura XML format
 
 ## Integration with CI/CD
 

src/pysonar_scanner/__main__.py

@@ -140,10 +140,8 @@ def run_dry_run(config: dict[str, Any]) -> int:
 
     DryRunReporter.report_configuration(config)
 
-    validation_result = ValidationResult()
-
     coverage_paths = config.get(SONAR_PYTHON_COVERAGE_REPORT_PATHS)
     project_base_dir = config.get(SONAR_PROJECT_BASE_DIR, ".")
-    CoverageReportValidator.validate_coverage_reports(coverage_paths, project_base_dir, validation_result)
+    validation_result = CoverageReportValidator.validate_coverage_reports(coverage_paths, project_base_dir)
 
     return DryRunReporter.report_validation_results(validation_result)

src/pysonar_scanner/dry_run_reporter.py

@@ -47,7 +47,7 @@ def report_configuration(config: dict[str, Any]) -> None:
             {
                 SONAR_PROJECT_KEY: config.get(SONAR_PROJECT_KEY),
                 SONAR_PROJECT_NAME: config.get(SONAR_PROJECT_NAME),
-                SONAR_ORGANIZATION: config.get(SONAR_ORGANIZATION, "N/A (likely SonarQube Server)"),
+                SONAR_ORGANIZATION: config.get(SONAR_ORGANIZATION, "N/A"),
             },
         )
 
@@ -90,9 +90,9 @@ def report_validation_results(validation_result: "ValidationResult") -> int:
         else:
             logging.warning("✗ Configuration validation FAILED with the following issues:")
             for error in validation_result.errors:
-                logging.error(f"  • {error}")
+                logging.error(f"✗ {error}")
             for warning in validation_result.warnings:
-                logging.warning(f"  • {warning}")
+                logging.warning(f"⚠ {warning}")
             logging.info("=" * 80)
             return 1
 
@@ -136,18 +136,20 @@ class CoverageReportValidator:
     def validate_coverage_reports(
         coverage_paths: Optional[str],
         project_base_dir: str,
-        validation_result: ValidationResult,
-    ) -> None:
+    ) -> ValidationResult:
+        validation_result = ValidationResult()
         if not coverage_paths:
             validation_result.add_warning("No coverage report paths specified")
-            return
+            return validation_result
 
         base_path = Path(project_base_dir)
         report_paths = [p.strip() for p in coverage_paths.split(",")]
 
         for report_path in report_paths:
             CoverageReportValidator._validate_single_report(report_path, base_path, validation_result)
 
+        return validation_result
+
     @staticmethod
     def _validate_single_report(report_path: str, base_path: Path, validation_result: ValidationResult) -> None:
         # Resolve relative path
@@ -170,7 +172,7 @@ def _validate_single_report(report_path: str, base_path: Path, validation_result
                         f"Coverage report root element is '{root.tag}', expected 'coverage' (Cobertura format)"
                     )
                 else:
-                    validation_result.add_info(f"Coverage report is valid Cobertura XML: {report_path}")
+                    validation_result.add_info(f"Coverage report check passed: {report_path}")
         except PermissionError:
             validation_result.add_error(f"Coverage report is not readable (permission denied): {report_path}")
         except UnicodeDecodeError:
@@ -179,7 +181,7 @@ def _validate_single_report(report_path: str, base_path: Path, validation_result
             )
         except ET.ParseError as e:
             validation_result.add_error(
-                f"Coverage report is not valid XML (Cobertura format): {report_path}\n  Parse error: {str(e)}"
+                f"Coverage report is not valid XML (Cobertura format): {report_path} (Parse error: {str(e)})"
             )
         except Exception as e:
-            validation_result.add_error(f"Error validating coverage report format: {report_path}\n  Error: {str(e)}")
+            validation_result.add_error(f"Error validating coverage report format: {report_path} (Error: {str(e)})")

tests/unit/test_dry_run.py

@@ -145,18 +145,16 @@ def setUp(self):
         self.setUpPyfakefs()
 
     def test_validate_coverage_reports_no_paths(self):
-        result = ValidationResult()
-        CoverageReportValidator.validate_coverage_reports(None, ".", result)
+        result = CoverageReportValidator.validate_coverage_reports(None, ".")
 
         assert result.is_valid()
         assert len(result.warnings) == 1
         assert "No coverage report paths specified" in result.warnings[0]
 
     def test_validate_single_report_file_not_found(self):
         self.fs.create_dir("/project")
-        result = ValidationResult()
 
-        CoverageReportValidator.validate_coverage_reports("coverage.xml", "/project", result)
+        result = CoverageReportValidator.validate_coverage_reports("coverage.xml", "/project")
 
         assert not result.is_valid()
         assert len(result.errors) == 1
@@ -166,51 +164,46 @@ def test_validate_single_report_file_not_found(self):
     def test_validate_single_report_valid_cobertura(self, mock_logging):
         self.fs.create_dir("/project")
         self.fs.create_file("/project/coverage.xml", contents='<?xml version="1.0"?>\n<coverage></coverage>')
-        result = ValidationResult()
 
-        CoverageReportValidator.validate_coverage_reports("coverage.xml", "/project", result)
+        result = CoverageReportValidator.validate_coverage_reports("coverage.xml", "/project")
 
         assert result.is_valid()
         assert len(result.warnings) == 0
         assert len(result.infos) == 1
-        assert "valid Cobertura XML" in result.infos[0]
+        assert "Coverage report check passed" in result.infos[0]
 
     def test_validate_multiple_coverage_reports(self):
         self.fs.create_dir("/project")
         self.fs.create_file("/project/coverage1.xml", contents='<?xml version="1.0"?>\n<coverage></coverage>')
         self.fs.create_file("/project/coverage2.xml", contents='<?xml version="1.0"?>\n<coverage></coverage>')
-        result = ValidationResult()
 
-        CoverageReportValidator.validate_coverage_reports("coverage1.xml, coverage2.xml", "/project", result)
+        result = CoverageReportValidator.validate_coverage_reports("coverage1.xml, coverage2.xml", "/project")
 
         assert result.is_valid()
 
     def test_validate_report_not_a_file(self):
         self.fs.create_dir("/project")
         self.fs.create_dir("/project/coverage.xml")
-        result = ValidationResult()
 
-        CoverageReportValidator.validate_coverage_reports("coverage.xml", "/project", result)
+        result = CoverageReportValidator.validate_coverage_reports("coverage.xml", "/project")
 
         assert not result.is_valid()
         assert "not a file" in result.errors[0]
 
     def test_validate_report_invalid_xml(self):
         self.fs.create_dir("/project")
         self.fs.create_file("/project/coverage.xml", contents="not valid xml")
-        result = ValidationResult()
 
-        CoverageReportValidator.validate_coverage_reports("coverage.xml", "/project", result)
+        result = CoverageReportValidator.validate_coverage_reports("coverage.xml", "/project")
 
         assert not result.is_valid()
         assert "not valid XML" in result.errors[0]
 
     def test_validate_report_wrong_root_element(self):
         self.fs.create_dir("/project")
         self.fs.create_file("/project/coverage.xml", contents='<?xml version="1.0"?>\n<report></report>')
-        result = ValidationResult()
 
-        CoverageReportValidator.validate_coverage_reports("coverage.xml", "/project", result)
+        result = CoverageReportValidator.validate_coverage_reports("coverage.xml", "/project")
 
         assert result.is_valid()
         assert len(result.warnings) == 1
@@ -220,9 +213,8 @@ def test_validate_report_wrong_root_element(self):
     def test_validate_mixed_valid_and_missing_reports(self):
         self.fs.create_dir("/project")
         self.fs.create_file("/project/exists.xml", contents='<?xml version="1.0"?>\n<coverage></coverage>')
-        result = ValidationResult()
 
-        CoverageReportValidator.validate_coverage_reports("exists.xml, missing.xml", "/project", result)
+        result = CoverageReportValidator.validate_coverage_reports("exists.xml, missing.xml", "/project")
 
         assert not result.is_valid()
         assert len(result.errors) == 1