Merge branch 'main' into dependabot/pip/urllib3-2.5.0

Author: TG1999

.github/workflows/pypi-release-aboutcode-federated.yml

@@ -0,0 +1,32 @@
+name: Build aboutcode.federated Python distributions and publish on PyPI
+
+on:
+  workflow_dispatch:
+  push:
+   tags:
+     - "aboutcode.federated/*"
+
+jobs:
+  build-and-publish:
+    name: Build and publish library to PyPI
+    runs-on: ubuntu-22.04
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: 3.11
+
+      - name: Install flot
+        run: python -m pip install flot --user
+
+      - name: Build binary wheel and source tarball
+        run:  python -m flot --pyproject pyproject-aboutcode.federated.toml --sdist --wheel --output-dir dist/
+
+      - name: Publish to PyPI
+        if: startsWith(github.ref, 'refs/tags')
+        uses: pypa/gh-action-pypi-publish@release/v1
+        with:
+          password: ${{ secrets.PYPI_API_TOKEN_ABOUTCODE_FEDERATED }}

README.rst

@@ -19,10 +19,10 @@ VulnerableCode
 
 
 VulnerableCode is a free and open database of open source software package
-vulnerabilities **because open source software vulnerabilities data and tools
+vulnerabilities **because open source software vulnerability data and tools
 should be free and open source themselves**:
 
-we are trying to change this and evolve the status quo in a few other areas!
+We are trying to change this and evolve the status quo in a few other areas!
 
 - Vulnerability databases have been **traditionally proprietary** even though they
   are mostly about free and open source software. 
@@ -31,13 +31,13 @@ we are trying to change this and evolve the status quo in a few other areas!
   means a lot of false positive signals that require extensive expert reviews.
 
 - Vulnerability databases are also mostly about vulnerabilities first and software
-  package second, making it difficult to find if and when a vulnerability applies
-  to a piece of code. VulnerableCode focus is on software package first where
-  a Package URL is a key and natural identifier for packages; this is making it
+  packages second, making it difficult to find if and when a vulnerability applies
+  to a piece of code. VulnerableCode's focus is on software packages first where
+  a Package URL (PURL) is a key and natural identifier for packages; this makes it
   easier to find a package and whether it is vulnerable.
 
-Package URL themselves were designed first in ScanCode and VulnerableCode
-and are now a de-facto standard for vulnerability management and package references.
+PURLs were designed initially for ScanCode and VulnerableCode. PURL is
+now a de-facto standard for vulnerability management and package references.
 See https://github.com/package-url/purl-spec
 
 The VulnerableCode project is a FOSS community resource to help improve the
@@ -49,17 +49,14 @@ the database current.
 
 .. pull-quote::
    **Warning**
+   VulnerableCode is under active development and may not be ready for production
+use depending on your use cases.
 
-   VulnerableCode is under active development and is not yet fully
-   usable.
+Read more about VulnerableCode at https://vulnerablecode.readthedocs.org/
 
-
-Read more about VulnerableCode https://vulnerablecode.readthedocs.org/
-
-VulnerableCode tech stack is Python, Django, PostgreSQL, nginx and Docker and
+The VulnerableCode tech stack is Python, Django, PostgreSQL, nginx and Docker and
 several libraries.
 
-
 Getting started
 ===============
 

aboutcode/federated/CHANGELOG.rst

@@ -0,0 +1,9 @@
+Changelog
+=============
+
+
+v0.1.0 (October 20, 2025)
+---------------------------
+
+- Initial release of the ``aboutcode.federated`` library based on
+  original work in the ``aboutcode.hashid`` library.

aboutcode/federated/README.rst

@@ -0,0 +1,69 @@
+aboutcode.federated
+===================
+
+This is a library of utilities to compute ids and file paths for AboutCode
+federated data based on Package URL
+
+
+Federated data utilities goal is to handle content-defined and hash-addressable
+Package data keyed by PURL stored in many Git repositories. This approach to
+federate decentralized data is called FederatedCode.
+
+
+Overview
+========
+
+The main design elements for these utilities are:
+
+1. **Data Federation**: A Data Federation is a database, representing a consistent,
+non-overlapping set of data kind clusters (like scans, vulnerabilities or SBOMs)
+across many package ecosystems, aka. PURL types.
+A Federation is similar to a traditional database.
+
+2. **Data Cluster**: A Data Federation contains Data Clusters, where a Data Cluster
+purpose is to store the data of a single kind (like scans) across multiple PURL
+types. The cluster name is the data kind name and is used as the prefix for
+repository names. A Data Cluster is akin to a table in a traditional database.
+
+3. **Data Repository**: A DataCluster contains of one or more Git Data Repository,
+each storing datafiles of the cluster data kind and a one PURL type, spreading
+the datafiles in multiple Data Directories. The name is data-kind +PURL-
+type+hashid. A Repository is similar to a shard or tablespace in a traditionale
+database.
+
+4. **Data Directory**: In a Repository, a Data Directory contains the datafiles for
+PURLs. The directory name PURL-type+hashid
+
+5. **Data File**: This is a Data File of the DataCluster's Data Kind that is
+stored in subdirectories structured after the PURL components::
+
+   namespace/name/version/qualifiers/subpath:
+
+- Either at the level of a PURL name: namespace/name,
+- Or at the PURL version level namespace/name/version,
+- Or at the PURL qualifiers+PURL subpath level.
+
+A Data File can be for instance a JSON scan results file, or a list of PURLs in
+YAML.
+
+For example, a list of PURLs as a Data Kind  would stored at the name
+subdirectory level::
+
+    gem-0107/gem/random_password_generator/purls.yml
+
+Or a ScanCode scan as a Data Kind at the version subdirectory level::
+
+    gem-0107/npm/file/3.24.3/scancode.yml
+
+
+
+License
+-------
+
+Copyright (c) AboutCode and others. All rights reserved.
+
+SPDX-License-Identifier: Apache-2.0
+
+See https://github.com/aboutcode-org/vulnerablecode for support or download.
+
+See https://aboutcode.org for more information about AboutCode OSS projects.