Skip to content

SCANPY-156 Expand documentation for the pysonar-scanner #186

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Seppli11 merged 4 commits into from
Apr 2, 2025
Merged

SCANPY-156 Expand documentation for the pysonar-scanner #186

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
3fd8bbd
SCANPY-156 Expand documentation for the pysonar-scanner
guillaume-dequenne Apr 2, 2025
8642097
Automatically generate documentation for CLI args
guillaume-dequenne Apr 2, 2025
c8d9d54
Add CI task to validate documentation
guillaume-dequenne Apr 2, 2025
335ac21
Fix after review
guillaume-dequenne Apr 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion .cirrus.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ poetry_win_install: &POETRY_WIN_INSTALL
poetry_cache_template: &POETRY_CACHE
poetry_cache:
folder: ~/.cache/poetry/
fingerprint_script: cat poetry.lock
fingerprint_script: cat poetry.lock

.poetry_template: &POETRY_TEMPLATE
<<: *POETRY_CACHE
Expand Down Expand Up @@ -152,6 +152,14 @@ formatting_task:
- poetry run licenseheaders -t license_header.tmpl -o "SonarSource SA" -y 2011-2024 -n "Sonar Scanner Python" -E .py -d tests/
- git diff --name-only --exit-code ./src ./tests

documentation_task:
Copy link
Copy Markdown
Contributor

@Seppli11 Seppli11 Apr 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome 💪

<<: *POETRY_LINUX_TEMPLATE
alias: documentation
name: "CLI Documentation"
cli_docs_script:
- poetry run python tools/generate_cli_documentation.py
- git diff --exit-code CLI_ARGS.md

analysis_linux_task:
<<: *POETRY_LINUX_TEMPLATE
alias: analysis
Expand Down
100 changes: 100 additions & 0 deletions CLI_ARGS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
# Sonar Scanner Python CLI Arguments

## Authentication

| Option | Description |
| ------ | ----------- |
| `--sonar-host-url`, `-Dsonar.host.url` | SonarQube Server base URL. For example, http://localhost:9000 for a local instance of SonarQube Server |
| `--sonar-organization`, `-Dsonar.organization` | The key of the organization to which the project belongs |
| `--sonar-region`, `-Dsonar.region` | The region to contact, only for SonarQube Cloud |
| `-t`, `--token`, `--sonar-token`, `-Dsonar.token` | Token used to authenticate against the SonarQube Server or SonarQube Cloud |

## Project Configuration

| Option | Description |
| ------ | ----------- |
| `--sonar-project-base-dir`, `-Dsonar.projectBaseDir` | Directory containing the project to be analyzed. Default is the current directory |
| `--sonar-project-description`, `-Dsonar.projectDescription` | Description of the project |
| `--sonar-project-key`, `-Dsonar.projectKey` | Key of the project that usually corresponds to the project name in SonarQube |
| `--sonar-project-name`, `-Dsonar.projectName` | Name of the project in SonarQube |
| `--sonar-project-version`, `-Dsonar.projectVersion` | Version of the project |
| `--sonar-sources`, `-Dsonar.sources` | The analysis scope for main source code (non-test code) in the project |
| `--sonar-tests`, `-Dsonar.tests` | The analysis scope for test source code in the project |

## Analysis Configuration

| Option | Description |
| ------ | ----------- |
| `--sonar-filesize-limit`, `-Dsonar.filesize.limit` | Sets the limit in MB for files to be discarded from the analysis scope if the size is greater than specified |
| `--sonar-python-version`, `-Dsonar.python.version` | Python version used for the project |
| `-v`, `--verbose`, `--no-verbose`, `--sonar-verbose`, `--no-sonar-verbose`, `-Dsonar.verbose` | Increase output verbosity |

## Report Integration

| Option | Description |
| ------ | ----------- |
| `--sonar-external-issues-report-paths`, `-Dsonar.externalIssuesReportPaths` | Comma-delimited list of paths to generic issue reports |
| `--sonar-python-bandit-report-paths`, `--bandit-report-paths`, `-Dsonar.python.bandit.reportPaths` | Comma-separated bandit report paths, relative to project's root |
| `--sonar-python-coverage-report-paths`, `--coverage-report-paths`, `-Dsonar.python.coverage.reportPaths` | Comma-delimited list of paths to coverage reports in the Cobertura XML format. |
| `--sonar-python-flake8-report-paths`, `--flake8-report-paths`, `-Dsonar.python.flake8.reportPaths` | Comma-separated flake8 report paths, relative to project's root |
| `--sonar-python-mypy-report-paths`, `--mypy-report-paths`, `-Dsonar.python.mypy.reportPaths` | Comma-separated mypy report paths, relative to project's root |
| `--sonar-python-pylint-report-path`, `--pylint-report-path`, `-Dsonar.python.pylint.reportPath` | Path to third-parties issues report file for pylint |
| `--sonar-python-ruff-report-paths`, `--ruff-report-paths`, `-Dsonar.python.ruff.reportPaths` | Comma-separated ruff report paths, relative to project's root |
| `--sonar-python-xunit-report-path`, `--xunit-report-path`, `-Dsonar.python.xunit.reportPath` | Path to the report of test execution, relative to project's root |
| `--sonar-python-xunit-skip-details`, `--no-sonar-python-xunit-skip-details`, `--xunit-skip-details`, `--no-xunit-skip-details` | When enabled, the test execution statistics is provided only on project level |
| `--sonar-sarif-report-paths`, `-Dsonar.sarifReportPaths` | Comma-delimited list of paths to SARIF issue reports |

## Other

| Option | Description |
| ------ | ----------- |
| `--skip-jre-provisioning`, `-Dsonar.scanner.skipJreProvisioning` | If provided, the provisioning of the JRE will be skipped |
| `--sonar-branch-name`, `-Dsonar.branch.name` | Name of the branch being analyzed |
| `--sonar-build-string`, `-Dsonar.buildString` | The string passed with this property will be stored with the analysis and available in the results of api/project_analyses/search, thus allowing you to later identify a specific analysis and obtain its key for use with api/new_code_periods/set on the SPECIFIC_ANALYSIS type |
| `--sonar-cpd-python-minimum-lines`, `-Dsonar.cpd.python.minimumLines` | Minimum number of tokens to be considered as a duplicated block of code |
| `--sonar-cpd-python-minimum-tokens`, `-Dsonar.cpd.python.minimumTokens` | Minimum number of tokens to be considered as a duplicated block of code |
| `--sonar-links-ci`, `-Dsonar.links.ci` | The URL of the continuous integration system used |
| `--sonar-links-homepage`, `-Dsonar.links.homepage` | The URL of the build project home page |
| `--sonar-links-issue`, `-Dsonar.links.issue` | The URL to the issue tracker being used |
| `--sonar-links-scm`, `-Dsonar.links.scm` | The URL of the build project source code repository |
| `--sonar-log-level`, `-Dsonar.log.level` | Log level during the analysis |
| `--sonar-modules`, `-Dsonar.modules` | Comma-delimited list of modules to analyze |
| `--sonar-newcode-reference-branch`, `-Dsonar.newCode.referenceBranch` | Reference branch for new code definition |
| `--sonar-pullrequest-base`, `-Dsonar.pullrequest.base` | Base branch of the pull request being analyzed |
| `--sonar-pullrequest-branch`, `-Dsonar.pullrequest.branch` | Branch of the pull request being analyzed |
| `--sonar-pullrequest-key`, `-Dsonar.pullrequest.key` | Key of the pull request being analyzed |
| `--sonar-python-skip-unchanged`, `--no-sonar-python-skip-unchanged` | Override the SonarQube configuration of skipping or not the analysis of unchanged Python files |
| `--sonar-qualitygate-timeout`, `-Dsonar.qualitygate.timeout` | The number of seconds that the scanner should wait for a report to be processed |
| `--sonar-qualitygate-wait`, `--no-sonar-qualitygate-wait` | Forces the analysis step to poll the server instance and wait for the Quality Gate status |
| `--sonar-scanner-api-url`, `-Dsonar.scanner.apiUrl` | Base URL for all REST-compliant API calls, https://api.sonarcloud.io for example |
| `--sonar-scanner-arch`, `-Dsonar.scanner.arch` | Architecture on which the scanner will be running |
| `--sonar-scanner-cloud-url`, `-Dsonar.scanner.cloudUrl` | SonarQube Cloud base URL, https://sonarcloud.io for example |
| `--sonar-scanner-connect-timeout`, `-Dsonar.scanner.connectTimeout` | Time period to establish connections with the server (in seconds) |
| `--sonar-scanner-internal-dump-to-file`, `-Dsonar.scanner.internal.dumpToFile` | Filename where the input to the scanner engine will be dumped. Useful for debugging |
| `--sonar-scanner-internal-sq-version`, `-Dsonar.scanner.internal.sqVersion` | Emulate the result of the call to get SQ server version. Useful for debugging with --sonar-scanner-internal-dump-to-file |
| `--sonar-scanner-java-exe-path`, `-Dsonar.scanner.javaExePath` | If defined, the scanner engine will be run with this JRE |
| `--sonar-scanner-java-opts`, `-Dsonar.scanner.javaOpts` | Arguments provided to the JVM when running the scanner |
| `--sonar-scanner-keystore-password`, `-Dsonar.scanner.keystorePassword` | Password to access the keystore |
| `--sonar-scanner-keystore-path`, `-Dsonar.scanner.keystorePath` | Path to the keystore containing the client certificates used by the scanner. By default, <sonar.userHome>/ssl/keystore.p12 |
| `--sonar-scanner-metadata-filepath`, `-Dsonar.scanner.metadataFilepath` | Sets the location where the scanner writes the report-task.txt file containing among other things the ceTaskId |
| `--sonar-scanner-os`, `-Dsonar.scanner.os` | OS running the scanner |
| `--sonar-scanner-proxy-host`, `-Dsonar.scanner.proxyHost` | Proxy host |
| `--sonar-scanner-proxy-password`, `-Dsonar.scanner.proxyPassword` | Proxy password |
| `--sonar-scanner-proxy-port`, `-Dsonar.scanner.proxyPort` | Proxy port |
| `--sonar-scanner-proxy-user`, `-Dsonar.scanner.proxyUser` | Proxy user |
| `--sonar-scanner-response-timeout`, `-Dsonar.scanner.responseTimeout` | Time period required to process an HTTP call: from sending a request to receiving a response (in seconds) |
| `--sonar-scanner-socket-timeout`, `-Dsonar.scanner.socketTimeout` | Maximum time of inactivity between two data packets when exchanging data with the server (in seconds) |
| `--sonar-scanner-truststore-password`, `-Dsonar.scanner.truststorePassword` | Password to access the truststore |
| `--sonar-scanner-truststore-path`, `-Dsonar.scanner.truststorePath` | Path to the keystore containing trusted server certificates, used by the Scanner in addition to OS and the built-in certificates |
| `--sonar-scm-exclusions-disabled`, `--no-sonar-scm-exclusions-disabled` | Defines whether files ignored by the SCM, e.g., files listed in .gitignore, will be excluded from the analysis or not |
| `--sonar-scm-force-reload-all`, `--no-sonar-scm-force-reload-all` | Set this property to true to load blame information for all files, which may significantly increase analysis duration |
| `--sonar-scm-revision`, `-Dsonar.scm.revision` | Overrides the revision, for instance, the Git sha1, displayed in analysis results |
| `--sonar-source-encoding`, `-Dsonar.sourceEncoding` | Encoding of the source files. For example, UTF-8, MacRoman, Shift_JIS |
| `--sonar-user-home`, `-Dsonar.userHome` | Base sonar directory, ~/.sonar by default |
| `--sonar-working-directory`, `-Dsonar.working.directory` | Path to the working directory used by the Sonar scanner during a project analysis to store temporary data |
| `--toml-path` | Path to the pyproject.toml file. If not provided, it will look in the SONAR_PROJECT_BASE_DIR |
| `-Dsonar.python.skipUnchanged` | Equivalent to --sonar-python-skip-unchanged |
| `-Dsonar.python.xunit.skipDetails` | Equivalent to -Dsonar.python.xunit.skipDetails |
| `-Dsonar.qualitygate.wait` | Equivalent to --sonar-qualitygate-wait |
| `-Dsonar.scm.exclusions.disabled` | Equivalent to --sonar-scm-exclusions-disabled |
| `-Dsonar.scm.forceReloadAll` | Equivalent to --sonar-scm-force-reload-all |
71 changes: 40 additions & 31 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,15 +1,10 @@
# pysonar
A wrapper around SonarScanner CLI, available on PyPI.

# Disclaimer

This project is currently in beta and APIs are subject to change.
These changes include configuration parameter names.
A Python scanner for SonarQube, available on PyPI.

# Requirements

- SonarQube v9.9 or higher
- Python 3.8 or above
- SonarQube v10.6 or above
- Python 3.9 or above

# Installation

Expand All @@ -28,25 +23,33 @@ It assumes a running SonarQube server or a project configured on SonarCloud.
In order for the analysis to run, analysis properties need to be defined.
There are multiple ways of providing these properties, described below in descending order of priority:

* Through CLI arguments to the `pysonar` command
* Under the `[tool.sonar]` key of the `pyproject.toml` file
* Through common properties extracted from the `pyproject.toml`
* In a dedicated `sonar-project.properties` file
* Through environment variables
1. Through CLI arguments to the `pysonar` command
2. Environment variables for individual properties (e.g. `SONAR_TOKEN`, `SONAR_VERBOSE`, `SONAR_HOST_URL`, ...)
3. Generic environment variable `SONAR_SCANNER_JSON_PARAMS`
4. Under the `[tool.sonar]` key of the `pyproject.toml` file
5. In a dedicated `sonar-project.properties` file
6. Through common properties extracted from the `pyproject.toml`
Comment on lines +26 to +31
Copy link
Copy Markdown
Contributor

@thomas-serre-sonarsource thomas-serre-sonarsource Apr 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

<3


### Through CLI arguments

Analysis properties can be provided as CLI arguments to the `pysonar` command.
They follow the same convention as when running the SonarScanner CLI directly
(see [documentation](https://docs.sonarsource.com/sonarqube/9.9/analyzing-source-code/scanners/sonarscanner/#running-from-zip-file)).
They can be provided in a similar way as when running the SonarScanner CLI directly
Copy link
Copy Markdown
Contributor

@Seppli11 Seppli11 Apr 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally, I would have formulated this the other way around. E.g. use --token and --sonar-region by default. If the argument is not available, -D... works for everything, but we don't verify and blindly forward it

Copy link
Copy Markdown
Contributor Author

@guillaume-dequenne guillaume-dequenne Apr 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I intentionally framed it that way so that the first information is the one that will definitely work, then only later we provide a way to make it less verbose for specific properties.
I feel that reduces the risk of mistakes if someone only skims the documentation. Does it make sense?

Copy link
Copy Markdown
Contributor

@Seppli11 Seppli11 Apr 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I think so.

(see [documentation](https://docs.sonarsource.com/sonarqube-server/2025.1/analyzing-source-code/scanners/sonarscanner/#running-from-zip-file)).
This means that analysis properties provided that way should be prepended with `-D`, for instance:

```
$ pysonar -Dsonar.login=myAuthenticationToken
$ pysonar -Dsonar.token=myAuthenticationToken
```

You can use all the argument allowed by __SonarScanner__.
For more information on __SonarScanner__ please refer to the [SonarScanner documentation](https://docs.sonarsource.com/sonarqube/9.9/analyzing-source-code/scanners/sonarscanner/)
You can use all the arguments allowed by __SonarScanner__.
Copy link
Copy Markdown
Contributor

@thomas-serre-sonarsource thomas-serre-sonarsource Apr 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It could be worth it to mention that we support also Python specific properties like sonar.python.version

Copy link
Copy Markdown
Contributor Author

@guillaume-dequenne guillaume-dequenne Apr 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm tempted to think it's implied, since CLI_ARGS.md mentions them explicitly.
We could keep a list of Python specific properties documented separately though, but I feel it might be overkill at this point?

For more information on __SonarScanner__ please refer to the [SonarScanner documentation](https://docs.sonarsource.com/sonarqube-server/2025.1/analyzing-source-code/analysis-parameters/).

Additionally, some common properties can be provided using a shorter alias, such as:
```
pysonar --token "MyToken"
```

See [CLI_ARGS](https://github.com/SonarSource/sonar-scanner-python/blob/master/CLI_ARGS.md) for more details.

### With a pyproject.toml file

Expand All @@ -70,37 +73,43 @@ projectKey=my:project
#sourceEncoding=UTF-8
```

The configuration parameters can be found in the [SonarQube documentation](https://docs.sonarsource.com/sonarqube/9.9/analyzing-source-code/analysis-parameters/).
The configuration parameters can be found in the [SonarQube documentation](https://docs.sonarsource.com/sonarqube-server/2025.1/analyzing-source-code/analysis-parameters/).

In the `pyproject.toml` file the prefix `sonar.` for parameter keys should be omitted.
For example, `sonar.scm.provider` in the documentation will become `scm.provider` in the `pyproject.toml` file.

By default, the scanner will expect the `pyproject.toml` file to be present in the current directory.
However, its path can be provided manually through the `toml.path` ([SCANPY-40](https://sonarsource.atlassian.net/jira/software/c/projects/PYSCAN/issues/PYSCAN-40)) CLI argument as well as through the `sonar.projectHome` argument. For instance:
Properties in `pyproject.toml` files are expected to be provided in camel case. However, kebab case is also accepted:

```
pysonar --toml.path "path/to/pyproject.toml"
[tool.sonar]
project-key=My Project key # valid alias for projectKey
```

Or:
By default, the scanner will expect the `pyproject.toml` file to be present in the current directory. However, its path can be provided manually through the `toml-path` CLI argument as well as through the `sonar.projectBaseDir` argument. For instance:

```
pysonar --sonar-project-home "path/to/projectHome"
pysonar --toml-path "path/to/pyproject.toml"
```

Or:

### Through project properties extracted from the `pyproject.toml`
```
pysonar --sonar-project-base-dir "path/to/projectBaseDir"
```

Or:

When a `pyproject.toml` file is available, it is possible to set the `-read-project-config` flag
to allow the scanner to deduce analysis properties from the project configuration.
```
pysonar -Dsonar.projectBaseDir="path/to/projectBaseDir"
```

This is currently supported only for projects using `poetry`.
### Through project properties extracted from the `pyproject.toml`

The Sonar scanner will then use the project name and version defined through Poetry, they won't have to be duplicated under a dedicated `tool.sonar` section.
When a `pyproject.toml` file is available, the scanner can deduce analysis properties from the project configuration. This is currently supported only for projects using `poetry`.

### With a sonar-project.properties file

Exactly like [__SonarScanner__](https://docs.sonarsource.com/sonarqube/9.9/analyzing-source-code/scanners/sonarscanner/),
Exactly like [__SonarScanner__](https://docs.sonarsource.com/sonarqube-server/2025.1/analyzing-source-code/scanners/sonarscanner/),
the analysis can also be configured with a `sonar-project.properties` file:

```
Expand Down Expand Up @@ -130,7 +139,7 @@ $ export SONAR_HOST_URL="http://localhost:9000"
$ pysonar
```

See the __SonarScanner__ [documentation](https://docs.sonarsource.com/sonarqube/9.9/analyzing-source-code/scanners/sonarscanner/) for more information.
See the __SonarScanner__ [documentation](https://docs.sonarsource.com/sonarqube-server/2025.1/setup-and-upgrade/environment-variables/) for more information.

# Installation from testPyPI

Expand Down
7 changes: 6 additions & 1 deletion src/pysonar_scanner/configuration/cli.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,11 @@ def load(cls) -> dict[str, any]:

@classmethod
def __parse_cli_args(cls) -> tuple[argparse.Namespace, list[str]]:
parser = cls.__create_parser()
return parser.parse_known_args()

@classmethod
def __create_parser(cls):
parser = argparse.ArgumentParser(
description="Sonar scanner CLI for Python",
epilog="Analysis properties not listed here will also be accepted, as long as they start with the -D prefix.",
Expand Down Expand Up @@ -493,4 +498,4 @@ def __parse_cli_args(cls) -> tuple[argparse.Namespace, list[str]]:
"--sonar-modules", "-Dsonar.modules", type=str, help="Comma-delimited list of modules to analyze"
)

return parser.parse_known_args()
return parser
Loading