Documentation/555 formatting documentation (#556)

* Add documentation about formatting

* Switch references in getting_started to cookiecutter-template

* Remove changing output of nox -l as users can just use

* Remove unused and likely out of place reference

* Clean up from self-review

* Move pre-commit information to formatting

* Fix broken link

* Update dependencies to latest versions

* Add sphinx-toolbox as a dependency for collapsible options

* Move pre-commit to more general section

* Remove description about PYTHON_PATH setting as systemwide concern and fix some typos

Author: ArBridgeman

doc/conf.py

@@ -33,6 +33,7 @@
     "sphinx.ext.napoleon",
     "sphinx.ext.intersphinx",
     "sphinx.ext.autosummary",
+    "sphinx_toolbox.collapse",
     "sphinx_copybutton",
     "myst_parser",
     "sphinx_design",

doc/faq.rst

@@ -37,12 +37,6 @@ Documentation of the Exasol-Toolbox
 
         Document outlining the architectural and design principles and decisions in this project.
 
-    .. grid-item-card:: :octicon:`question` FAQ
-        :link: faq_toolbox
-        :link-type: ref
-
-        Frequently asked questions.
-
 
 .. toctree::
    :maxdepth: 4
@@ -52,5 +46,4 @@ Documentation of the Exasol-Toolbox
    developer_guide/developer_guide
    tools
    github_actions/github_actions
-   faq
    changes/changelog

doc/index.rst

@@ -0,0 +1,80 @@
+.. _formatting_code:
+
+Formatting code
+===============
+
+.. toctree::
+    :maxdepth: 2
+
+    troubleshooting
+
+The PTB automatically formats code and ensures via a step in the ``checks.yml`` GitHub
+workflow that committed code adheres to these standards. The goals of this are to
+improve the readability and maintainability of the code and to provide a uniform
+experience across projects.
+
+.. _formatting_sessions:
+
+Nox sessions
+++++++++++++
+
+For autoformatting, the following tools are used:
+
+* `black <https://black.readthedocs.io/en/stable/the_black_code_style/index.html>`__ -
+  formats Python code to maintain consistent styling standards.
+* `isort <https://pycqa.github.io/isort/index.html>`__ - organizes and formats Python
+  import statements alphabetically and by type (from __future__, standard library
+  packages, third party packages, and local application imports).
+* `pyupgrade <https://pypi.org/project/pyupgrade/>`__ - upgrades syntax for newer
+  versions of the Python language.
+
+In the PTB, these tools are bundled into nox sessions to ensure that they are run in a
+deterministic manner.
+
++--------------------+------------------+------------------------------------+
+| Nox session        | CI Usage         | Action                             |
++====================+==================+====================================+
+| ``project:fix``    | No               | Applies code formatting changes    |
++--------------------+------------------+------------------------------------+
+| ``project:format`` | ``checks.yml``   | Checks that current code does not  |
+|                    |                  | need to be re-formatted            |
++--------------------+------------------+------------------------------------+
+
+.. _formatting_configuration:
+
+Configuration
++++++++++++++
+black
+^^^^^
+
+Your ``black`` configuration should look similar to this:
+
+.. literalinclude:: ../../../../project-template/{{cookiecutter.repo_name}}/pyproject.toml
+  :language: toml
+  :start-at: [tool.black]
+  :end-before: [tool.isort]
+
+For further configuration options, see
+`black configuration <https://black.readthedocs.io/en/stable/usage_and_configuration/the_basics.html#configuration-format>`__.
+
+isort
+^^^^^
+Ensure ``isort`` is configured with compatibility for ``black``:
+
+.. literalinclude:: ../../../../project-template/{{cookiecutter.repo_name}}/pyproject.toml
+  :language: toml
+  :start-at: [tool.isort]
+  :end-before: [tool.pylint.master]
+
+For further configuration options, see
+`isort options <https://pycqa.github.io/isort/docs/configuration/options.html>`__.
+
+
+pyupgrade
+^^^^^^^^^
+No initial configuration of ``pyupgrade`` is required.
+
+For individual configuration, see the
+`pyupgrade CLI options <https://pypi.org/project/pyupgrade/>`__. These can
+be passed to the :ref:`formatting_sessions` via the ``pyupgrade_args``
+attribute of the :class:`noxconfig.Config`.

doc/user_guide/features/formatting_code/index.rst

@@ -0,0 +1,64 @@
+.. _formatting_troubleshooting:
+
+Troubleshooting
+===============
+
+Formatting still fails after running ``project:fix``
+----------------------------------------------------
+
+If when you execute:
+
+#. Run ``project:fix``
+#. Run ``project:format``
+
+you receive an error from ``project:format`` (i.e. ``isort`` or ``black``), it it
+likely that you need to update your configuration to align with
+:ref:`formatting_configuration`.
+
+The automatic formatting is doing x, but we shouldn't do that because of y
+---------------------------------------------------------------------------
+Usually, automatic formatting is helpful, but there are rare cases where a developer
+might want to ignore automatically applied formatting.
+
+.. note::
+ To ensure that automatic formatting remains useful, developers should always seek
+ to use the minimum fix reasonable for the affected code. In most cases, this would
+ mean adding a comment for a single line.
+
++-----------------------+--------------------------------+-----------------------+
+| Undesired Action      | Single line                    | Within a file         |
++=======================+================================+=======================+
+| formatting from black | .. code-block:: python                                 |
+|                       |                                                        |
+|                       |   # fmt: off                                           |
+|                       |   <code being ignored by black>                        |
+|                       |   # fmt: on                                            |
++-----------------------+--------------------------------+-----------------------+
+| formatting from isort | .. code-block:: python         | .. code-block:: python|
+|                       |                                |                       |
+|                       |   <line-to-ignore> # isort:skip|    # isort:skip_file  |
++-----------------------+--------------------------------+-----------------------+
+
+
+In the ``checks.yml``, ``project:format`` wants me to reformat code I did not modify
+------------------------------------------------------------------------------------
+
+This is likely due to one of our tools (i.e. ``black``) being upgraded. Within the
+``pyproject.toml`` of the PTB, dependencies are specified to allow
+compatible versions or a restricted version range (i.e., ``^6.0.1``, ``>=24.1.0,<26.0.0``).
+Such specifications should restrict major reformatting changes to coincide only with a
+new major version of the PTB. However, sometimes a tool's versioning may not properly
+adhere to semantic versioning.
+
+If you encounter this scenario, please:
+
+#. Ensure that your ``pyproject.toml`` has the PTB restricted to compatible versions
+   (i.e., ``^1.7.0``).
+#. Identify which tool is trying to reformat files that you did not modify.
+#. Reset your ``poetry.lock`` to align with what's in the project's **default branch**.
+#. More selectively update your ``poetry.lock`` with `poetry update <package-name>`.
+#. Share with your team which tool & version led to the unexpected changes. So that
+   other PTB users do not experience the same difficulties, we will update the PTB with
+   a patch version to avoid this tool's version and later do a major release to better
+   indicate the breaking changes. You could later create an issue in your GitHub
+   repository to update to the new major version of the PTB & do the reformatting.

doc/user_guide/features/formatting_code/troubleshooting.rst

@@ -0,0 +1,82 @@
+.. _git_hooks:
+
+Enabling Git Hooks
+==================
+
+.. toctree::
+    :maxdepth: 2
+
+    troubleshooting
+
+Git hooks are automated scripts that run at specific points during Git's execution,
+allowing developers to enforce quality standards, automate tasks, and customize Git's
+behavior. The PTB uses the `pre-commit <https://pre-commit.com/>`__ framework to
+define common Git hooks for Git commits and pushes. The configuration for
+``pre-commit``  is provided in a ``.pre-commit-config.yaml`` file and described in
+:ref:`pre-commit_configuration`.
+
+.. _pre-commit_configuration:
+
+Configuration
++++++++++++++
+
+#. Add a ``.pre-commit-config.yaml`` file to your project's root directory. Feel free to
+   take inspiration from the example ``.pre-commit-config.yaml``:
+
+    .. collapse:: .pre-commit-config.yaml
+
+        .. literalinclude:: ../../../../project-template/{{cookiecutter.repo_name}}/.pre-commit-config.yaml
+           :language: yaml
+
+
+#. Enable pre-commit hooks for your workspace:
+
+    .. code-block:: shell
+
+        poetry run -- pre-commit install --hook-type pre-commit --hook-type pre-push
+
+
+Working with ``pre-commit``
++++++++++++++++++++++++++++
+
+.. _committing:
+
+Committing
+----------
+
+Once ``pre-commit`` has been configured, the process for performing a ``git commit`` is:
+
+#. Make your code changes
+#. ``git add`` changed files and ``git commit -m "<message>"``
+#. ``pre-commit`` performs checks on the changed files and produces an output like
+
+    .. code-block:: bash
+
+        code-format..........................................(no files to check)Skipped
+        check yaml...........................................(no files to check)Skipped
+        fix end of files.........................................................Passed
+        trim trailing whitespace.................................................Passed
+
+   * If all steps pass, then no action is needed.
+   * If a step fails, then check the output further. If it was an automatic fix, then
+     just add the altered file to your commit and execute your ``git commit`` line again.
+     Otherwise, manual intervention is needed.
+
+Pushing
+-------
+
+Once ``pre-commit`` has been configured, the process for performing a ``git push`` is:
+
+#. Perform one or more iterations of :ref:`committing`.
+#. ``git push``
+#. ``pre-commit`` performs checks on the changed files and produces an output like
+
+    .. code-block:: bash
+
+        type-check...............................................................Passed
+        lint.....................................................................Passed
+
+   * If all steps pass, then no action is needed.
+   * If a step fails, then check the output further. The suggested ``pre-push`` actions
+     given in the :ref:`pre-commit_configuration` require manual intervention. Create
+     a new commit to resolve the issue & try ``git push`` again.

doc/user_guide/features/git_hooks/index.rst

@@ -0,0 +1,12 @@
+Troubleshooting
+===============
+
+The checks for ``git commit`` or ``git push`` keep failing
+----------------------------------------------------------
+
+The ``pre-commit`` hooks, as defined in the ``.pre-commit-config.yaml``
+use multiple nox sessions. If a step fails multiple times, despite a user adding fixes
+or manually resolving the error, then please check which nox session is failing
+and refer to its troubleshooting documentation for help:
+
+* :ref:`Formatting Troubleshooting <formatting_troubleshooting>`

doc/user_guide/features/git_hooks/troubleshooting.rst

@@ -9,6 +9,8 @@ Features
     metrics/collecting_metrics
     creating_a_release
     documentation/index
+    git_hooks/index
+    formatting_code/index
     managing_dependencies
 
 Uniform Project Layout