Bugfix/490 artifacts validate fails for newly created project (#493)

* Add .gitattributes to project-template and update .gitignore

* Adapt pyproject.toml to be the same

* Define noxconfig values as iterable & set defaults with immutable type tuple to reduce warnings

* Refactor in preparation of additional testing

* Add tests for nox tasks that affect starting off a project

* Add lint:code test for project-template & modify version.py to pass linting

* Add test for artifacts:validate

* Fix artifacts:validate and sonar:check to pass when working with newly created project

* Remove and fix typings

* Remove unneeded import

* Fix typing for python 3.9

* Add coverage for prepare_coverage_xml

* Fix hint for version.py

* Remove assert of mock

* Ignore warning on import for subprocess

* Fix gitattributes

* Elaborate on comment regarding coverage XML

* Update comment with improved formatting using dashes

* Add sub-types to noxconfig.py

* Remove non-ASCII Unicodes in issue templates

* Update comment to be less vague

* Switch from is_file to exists

* Test out what happens to sonarqubebot if no coverage included in pysonar upload

* Finalize how we will handle no coverage for new projects in the PTB

* Switch COVERAGE_FILE to COVERAGE_DB

* Fix formatting issue in doc

* Fix cwd which differs between the tests & real running

* Fix type again; think it's a given , = | in my brain

Author: ArBridgeman

.gitattributes

@@ -1 +1,2 @@
-poetry.lock linguist-generated=true
+poetry.lock linguist-generated=true
+exasol/toolbox/version.py linguist-generated=true

doc/changes/unreleased.md

@@ -2,4 +2,6 @@
 
 ## Bugfixes
 
-* #489: Fixed .pre-commit-config.yaml to use existing nox tasks
+* #489: Fixed .pre-commit-config.yaml to use existing nox tasks
+* #490: Fixed artifacts:validate & sonar:check to work for newly created projects
+* #484: Fixed hint command text in version.py to include -s for executing nox task

doc/user_guide/getting_started.rst

@@ -204,7 +204,14 @@ We also need to configure settings for github-pages environment:
 8. Set up for Sonar
 +++++++++++++++++++
 PTB supports using SonarQube Cloud to analyze, visualize, & track linting, security, &
-coverage. In order to set it up, you'll need to do the following instructions.
+coverage. All of our Python projects are evaluated against the
+`Exasol Way <https://sonarcloud.io/organizations/exasol/quality_gates/show/AXxvLH-3BdtLlpiYmZhh>`__
+and subscribe to the
+`Clean as You Code <https://docs.sonarsource.com/sonarqube-server/9.8/user-guide/clean-as-you-code/>`__
+methodology, which means that SonarQube analysis will fail and, if its included in the branch protections, block a PR
+if code modified in that PR does not meet the standards of the Exasol Way.
+
+In order to set up Sonar, you will need to perform the following instructions.
 
 For a **public** project
 ^^^^^^^^^^^^^^^^^^^^^^^^
@@ -224,6 +231,12 @@ For a **public** project
         organization = "exasol"
 6. Post-merge, update the branch protections to include SonarQube analysis
 
+  * This should only be done when tests exist for the project, & that the project is
+    at a state in which enforced code coverage would not be a burden. For new projects,
+    we recommend creating an issue to add the SonarQube analysis to the branch protections
+    at a later point. In such scenarios, SonarQube analysis will still report its analysis
+    results to the PR, but it will not prevent the PR from being merged.
+
 For a **private** project
 ^^^^^^^^^^^^^^^^^^^^^^^^^
 1. Specify in the `noxconfig.py` the relative path to the project's source code in `Config.source`

exasol/toolbox/nox/_artifacts.py

@@ -3,10 +3,14 @@
 import re
 import shutil
 import sqlite3
+import subprocess  # nosec
 import sys
 from collections.abc import Iterable
 from pathlib import Path
-from typing import Optional
+from typing import (
+    Optional,
+    Union,
+)
 
 import nox
 from nox import Session
@@ -17,13 +21,13 @@
     Config,
 )
 
-COVERAGE_FILE = ".coverage"
+COVERAGE_DB = ".coverage"
 COVERAGE_XML = "ci-coverage.xml"
 LINT_JSON = ".lint.json"
 LINT_TXT = ".lint.txt"
 SECURITY_JSON = ".security.json"
 
-ALL_LINT_FILES = {COVERAGE_FILE, LINT_JSON, LINT_TXT, SECURITY_JSON}
+ALL_LINT_FILES = {COVERAGE_DB, LINT_JSON, LINT_TXT, SECURITY_JSON}
 COVERAGE_TABLES = {"coverage_schema", "meta", "file", "line_bits"}
 LINT_JSON_ATTRIBUTES = {
     "type",
@@ -54,7 +58,7 @@ def check_artifacts(session: Session) -> None:
         _is_valid_lint_txt(Path(PROJECT_CONFIG.root, LINT_TXT)),
         _is_valid_lint_json(Path(PROJECT_CONFIG.root, LINT_JSON)),
         _is_valid_security_json(Path(PROJECT_CONFIG.root, SECURITY_JSON)),
-        _is_valid_coverage(Path(PROJECT_CONFIG.root, COVERAGE_FILE)),
+        _is_valid_coverage(Path(PROJECT_CONFIG.root, COVERAGE_DB)),
     ]
     if not all(all_is_valid_checks):
         sys.exit(1)
@@ -145,7 +149,7 @@ def copy_artifacts(session: Session) -> None:
 
     artifact_dir = Path(session.posargs[0])
     suffix = _python_version_suffix()
-    _combine_coverage(session, artifact_dir, f"coverage{suffix}*/{COVERAGE_FILE}")
+    _combine_coverage(session, artifact_dir, f"coverage{suffix}*/{COVERAGE_DB}")
     _copy_artifacts(
         artifact_dir,
         artifact_dir.parent,
@@ -183,9 +187,45 @@ def _copy_artifacts(source: Path, dest: Path, files: Iterable[str]):
             print(f"File not found {path}", file=sys.stderr)
 
 
-def _prepare_coverage_xml(session: Session, source: Path) -> None:
-    command = ["coverage", "xml", "-o", COVERAGE_XML, "--include", f"{source}/*"]
-    session.run(*command)
+def _prepare_coverage_xml(
+    session: Session, source: Path, cwd: Union[Path, None] = None
+) -> None:
+    """
+    Prepare the coverage XML for input into Sonar
+
+    The coverage XML is used within Sonar to determine the overall coverage in our
+    project and the coverage associated with source code changes made in a PR. If the
+    code coverage associated with changes made in our PR does not meet the set Sonar
+    guidelines for that project, then when the sonarqubecloud bot writes in the PR, it
+    will indicate that the coverage constraint was not met, &, if the bot is required in
+    the branch protections for that project, then it will block the PR from being
+    merged. Thus, in the preparation of the coverage XML, we set `--fail-under=0` so
+    that this sub-step will not fail prior to sending the data to Sonar. Otherwise, this
+    sub-step would fail whenever the `fail_under` in `[tool.coverage.report]` of the
+    `pyproject.toml` was not met.
+    """
+    command = [
+        "coverage",
+        "xml",
+        "-o",
+        COVERAGE_XML,
+        "--include",
+        f"{source}/*",
+        "--fail-under=0",
+    ]
+    output = subprocess.run(command, capture_output=True, text=True, cwd=cwd)  # nosec
+    if output.returncode != 0:
+        if output.stdout.strip() == "No data to report.":
+            # Assuming that previous steps passed in the CI, this indicates — as
+            # is in the case for newly created projects — that no coverage over the
+            # `source` files was found. In this scenario, we do not need the CI to
+            # break from a non-zero exit code. This is because without a coverage file
+            # the sonarqubecloud bot will still report that the coverage is 0%, but as
+            # long as the bot is not required in the project's branch protections, then
+            # the PR can still be merged, provided that the other required constraints
+            # have been met.
+            return
+        session.error(output.returncode, output.stdout, output.stderr)
 
 
 def _upload_to_sonar(
@@ -195,8 +235,6 @@ def _upload_to_sonar(
         "pysonar",
         "--sonar-token",
         sonar_token,
-        "--sonar-python-coverage-report-paths",
-        COVERAGE_XML,
         "--sonar-python-pylint-report-path",
         LINT_JSON,
         "--sonar-python-bandit-report-paths",
@@ -206,6 +244,8 @@ def _upload_to_sonar(
         "--sonar-sources",
         config.source,
     ]
+    if Path(COVERAGE_XML).exists():
+        command.extend(["--sonar-python-coverage-report-paths", COVERAGE_XML])
     session.run(*command)  # type: ignore
 
 

exasol/toolbox/nox/_package_version.py

@@ -20,12 +20,14 @@
 
 # fmt: off
 _VERSION_MODULE_TEMPLATE = cleandoc('''
-    # ATTENTION:
-    # This file is generated by exasol/toolbox/nox/_package_version.py when using:
-    #   * either "poetry run -- nox -s project:fix"
-    #   * or "poetry run -- nox version:check -- --fix"
-    # Do not edit this file manually!
-    # If you need to change the version, do so in the pyproject.toml, e.g. by using `poetry version X.Y.Z`.
+    """ ATTENTION:
+    This file is generated by exasol/toolbox/nox/_package_version.py when using:
+       * either "poetry run -- nox -s project:fix"
+       * or "poetry run -- nox -s version:check -- --fix"
+    Do not edit this file manually!
+    If you need to change the version, do so in the pyproject.toml, e.g. by using
+    `poetry version X.Y.Z`.
+    """
     MAJOR = {major}
     MINOR = {minor}
     PATCH = {patch}

exasol/toolbox/templates/github/ISSUE_TEMPLATE/blank.md

@@ -1,8 +1,8 @@
 ---
-name: 📝 Blank Issue 
+name: Blank Issue
 about: Blank Issue
-title: 📝 <Insert Title>
-labels: 
+title: <Insert Title>
+labels:
 assignees: ''
 
 ---

exasol/toolbox/templates/github/ISSUE_TEMPLATE/bug.md

@@ -1,7 +1,7 @@
 ---
-name: 🐞 Bug Report
+name: Bug Report
 about: File a bug report
-title: 🐞 <Insert Title>
+title: <Insert Title>
 labels: bug
 assignees: ''
 
@@ -74,4 +74,4 @@ Add any additional context about the problem here.
 #### Related Issues (optional)
 <!---
 Link related issues form the issue tracker here.
--->
+-->

exasol/toolbox/templates/github/ISSUE_TEMPLATE/documentation.md

@@ -1,7 +1,7 @@
 ---
-name: 📚 Documentation
+name: Documentation
 about: Add/Improve Documentation
-title: 📚 <Insert Title>
+title: <Insert Title>
 labels: documentation
 assignees: ''
 
@@ -38,4 +38,4 @@ e.g.:
 - [ ] Bump version number
 - [ ] Update changelog
 - [ ] ...
---->
+--->

exasol/toolbox/templates/github/ISSUE_TEMPLATE/feature.md

@@ -1,7 +1,7 @@
 ---
-name: ✨ Feature
+name: Feature
 about: Add/Implement Feature
-title: ✨ <Insert Title>
+title: <Insert Title>
 labels: feature
 assignees: ''
 
@@ -44,4 +44,4 @@ e.g.:
 - [ ] Update documentation
 - [ ] Update changelog
 - [ ] ...
---->
+--->

exasol/toolbox/templates/github/ISSUE_TEMPLATE/refactoring.md

@@ -1,7 +1,7 @@
 ---
-name: 🔧 Refactoring
+name: Refactoring
 about: Refactor
-title: 🔧 <Insert Title>
+title: <Insert Title>
 labels: refactoring
 assignees: ''
 
@@ -44,4 +44,4 @@ e.g.:
 - [ ] Bump version number
 - [ ] Update changelog
 - [ ] ...
---->
+--->