Skip to content

Commit b961fe5

Browse files
Add Documentation on Metrics (#263)
--------- Co-authored-by: Sebastian Bär <rcb4gh@proton.me>
1 parent 99654df commit b961fe5

File tree

5 files changed

+135
-0
lines changed

5 files changed

+135
-0
lines changed

doc/_static/metrics-artifact.png

68.9 KB
Loading
86.2 KB
Loading

doc/changes/unreleased.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,11 @@
44

55
* Fixed CD workflow template
66

7+
## 📚 Documentation
8+
9+
* Add Documentation on Metrics
10+
711
## 🔩 Internal
812

913
* Relock dependencies
14+

doc/index.rst

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,12 @@ Documentation of the Exasol-Toolbox
5555

5656
Document outlining the architectural and design principles and decisions in this project.
5757

58+
.. grid-item-card:: :octicon:`graph` Metrics
59+
:link: metrics
60+
:link-type: ref
61+
62+
Details on metrics collection and support related to and supported by the Python toolbox.
63+
5864
.. grid-item-card:: :octicon:`question` FAQ
5965
:link: faq_toolbox
6066
:link-type: ref
@@ -73,6 +79,7 @@ Documentation of the Exasol-Toolbox
7379
tools
7480
github_actions/github_actions
7581
api
82+
metrics
7683
faq
7784
changes/changelog
7885

doc/metrics.rst

Lines changed: 123 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,123 @@
1+
.. _metrics:
2+
3+
:octicon:`graph` Metrics
4+
========================
5+
6+
To establish a unified approach for metrics and reporting across our projects, we have developed a simple and straightforward metrics schema, see: `Exasol Schemas`_ (Section: *Exasol Project Metrics*).
7+
This schema is designed for ease of use and can be adopted universally across our projects for reporting fundamental metric data.
8+
9+
Why a Custom Format?
10+
--------------------
11+
12+
#. **Simplicity and Clarity**:
13+
14+
- The schema is straightforward, making it easy to understand and implement.
15+
16+
#. **Vendor Neutrality and Extensibility**:
17+
18+
- It is not tied to any specific vendor, allowing us to customize and extend it as needed.
19+
20+
#. **Ease of Integration**:
21+
22+
- Existing tools and vendor solutions can be seamlessly integrated by creating mappings from vendor results to our metrics schema.
23+
24+
#. **Customizable Metrics Generation and Evaluation**:
25+
26+
- We have the flexibility to adjust how metric data is generated and evaluated based on our specific needs. This, for example, allows us the freedom to rate maintainability differently across various language ecosystems, avoiding lock-in to a single method.
27+
28+
29+
Schema
30+
------
31+
32+
see also `Exasol Schemas`_.
33+
34+
Examples
35+
________
36+
37+
**0.1.0**
38+
39+
.. code-block:: json
40+
41+
{
42+
"commit": "99654dfcbd6169f6c78e4f5117a3184891d36f5d",
43+
"date": "2024-10-25 10:43:10.145235",
44+
"coverage": 33.910533910533914,
45+
"maintainability": "B",
46+
"reliability": "N/A",
47+
"security": "F",
48+
"technical_debt": "N/A"
49+
}
50+
51+
**0.2.0**
52+
53+
.. code-block:: json
54+
55+
{
56+
"project": "python-toolbox",
57+
"commit": "99654dfcbd6169f6c78e4f5117a3184891d36f5d",
58+
"date": "2024-10-25 10:43:10.145235",
59+
"coverage": 33.910533910533914,
60+
"maintainability": "B",
61+
"reliability": "N/A",
62+
"security": "F",
63+
"technical_debt": "N/A"
64+
}
65+
66+
67+
68+
69+
70+
Workflows
71+
---------
72+
73+
The Python toolbox comes with various tools to help maintain our Python projects. One of these tools are workflow templates, which can be generated and updated by default. The toolbox also provides a workflow called *report*, which gathers and reports a metrics summary for the project's CI run. For details on the workflow, run :code:`tbx workflow show report` to see the full workflow definitions. Below, you can see two screenshots of the results and artifacts this workflow provides.
74+
75+
.. figure:: _static/metrics-workflow-summary.png
76+
:alt: Metrics Workflow Summary
77+
78+
.. figure:: _static/metrics-artifact.png
79+
:alt: Metrics Artifact
80+
81+
Tooling
82+
-------
83+
84+
Projects
85+
________
86+
87+
Projects can run the nox task :code:`nox -s report` to generate their project metrics if they are using the :code:`exasol-toolbox`.
88+
89+
90+
Development
91+
___________
92+
93+
If the metrics schema needs to be updated, the `Metrics Schema Project`_ provides a convenient way (Pydantic model-based) to update and generate an updated schema for the metrics.
94+
95+
.. note::
96+
97+
The updated version needs to be integrated into the `Exasol Schemas Project`_.
98+
99+
100+
Ratings
101+
-------
102+
The strategy on how the rating of the Python code and projects work can be seen in the `metrics.py`_ file.
103+
104+
105+
Collection
106+
----------
107+
For our open-source projects, there is a scheduled job that regularly collects metrics information from projects. This data is then aggregated and added to a central data store. For more details, please refer to the crawler project documentation.
108+
109+
110+
Links & References
111+
------------------
112+
* `Exasol Schemas`_
113+
* `Metrics Schema`_
114+
* `Metrics Schema Project`_
115+
* `metrics.py`_
116+
117+
118+
.. _Exasol Schemas: https://schemas.exasol.com
119+
.. _Exasol Schemas Project: https://github.com/exasol/schemas
120+
.. _Metrics Schema: https://schemas.exasol.com/project-metrics-0.2.0.html
121+
.. _metrics.py: https://github.com/exasol/python-toolbox/blob/main/exasol/toolbox/metrics.py
122+
.. _Metrics Schema Project: https://github.com/exasol/python-toolbox/tree/main/metrics-schema
123+

0 commit comments

Comments
 (0)