v7.1.0

What's Changed

• Set python package content value by @JonoYang in #623
• Order filter by package content type by @JonoYang in #624
• Update migrations by @JonoYang in #625
• Update python 3.13 by @JonoYang in #629
• Use requirements.txt to build dockerfile by @AyanSinhaMahapatra in #675
• Update scancode dependencies by @AyanSinhaMahapatra in #676
• Fix incorrect repo name in documentation by @JonTheNiceGuy in #628
• Add support to mine pypi Package-URLs #662 by @AyanSinhaMahapatra in #677
• Move write_packageurls_to_file and write_data_to_file to minecode_pip… by @JonoYang in #681
• Rename minecode_pipeline to minecode_pipelines by @AyanSinhaMahapatra in #682
• Add composer collector by @TG1999 in #689
• Add CRAN collector by @TG1999 in #690
• add on demand package data collection for golang, gitlab and bitbucket #596 by @chinyeungli in #608
• Add support cpan on demand collection by @TG1999 in #693
• Support mining pypi purls with checkpoints by @AyanSinhaMahapatra in #708
• Add support for Arch Linux ALPM on-demand data collection by @ziadhany in #709
• Fix: Code style formatting by @ziadhany in #715
• Add collector for luarocks ecosystem by @TG1999 in #714
• Add hackage collector for on demand collection by @TG1999 in #716
• Add huggingface collector for on demand collection by @TG1999 in #717
• Add collector for pub ecosystem by @TG1999 in #712
• Add support for Conda on-demand data collection by @ziadhany in #713
• Add support for on-demand data collection in Alpine Linux APK. by @ziadhany in #711
• Add hex collector by @TG1999 in #722
• Add support to mine maven Package-URLs #660 by @JonoYang in #678
• Add support for DockerHub on-demand collection by @ziadhany in #720
• Add support for Swift- on demand, data collection by @ziadhany in #719
• 664 purl next debian by @JonoYang in #718
• Add support to mine cargo Package-URLs by @ziadhany in #679
• Update pypi packageURL mining by @AyanSinhaMahapatra in #710
• Mine PURL from alpine #665 by @JonoYang in #724
• Fix debian and maven pipeline runs by @AyanSinhaMahapatra in #728
• Use argument when instantiating MavenNexusCollector #660 by @JonoYang in #729
• Properly set index_location #660 by @JonoYang in #730
• Add support to mine conan Package-URLs by @ziadhany in #684
• Update pipelines by @JonoYang in #732
• Update debian pipeline by @JonoYang in #733
• Do not clobber name variable #665 by @JonoYang in #734
• Add pipeline to mine NuGet PURLs by @keshav-space in #721
• Purl next update pipelines by @JonoYang in #736
• Update alpine pipeline by @JonoYang in #737
• Update alpine pipeline 2 by @JonoYang in #738
• Add support to mine CRAN Package-URLs by @ziadhany in #688
• Add support to mine Swift Package-URLs by @ziadhany in #725
• Add support to mine Composer Package-URLs by @ziadhany in #691
• Fix dependency issues related to Swift and correct typos in Composer. by @ziadhany in #741
• Bump version of minecode_pipelines by @ziadhany in #742
• Update CRAN, Swift and composer by @ziadhany in #743
• Mine Cargo PackageURLs without creating checkpoints by @ziadhany in #746
• Update deployment by @JonoYang in #748
• Resolve logger issue and update Swift pipeline to use commit hashes by @ziadhany in #745
• Abstract git repo handling in base MineCodeBasePipeline by @keshav-space in #750
• Refactor cargo mining pipeline for git deployment by @keshav-space in #781
• Abstract purl publish logic in reusable function by @keshav-space in #782
• Refactor alpine mining pipeline for git deployment by @keshav-space in #783
• Support periodic checkpoint updates in base pipeline by @keshav-space in #784
• Refactor maven mining pipeline for git deployment by @keshav-space in #785
• Refactor conan mining pipeline for git deployment by @TG1999 in #787
• Refactor mine_composer to use MineCodeBasePipeline for git deployment #772 by @ziadhany in #788
• Improve documentation #768 by @chinyeungli in #769
• Decentralized purl data #668 by @pombredanne in #790
• Refactor debian mining pipeline for git deployment by @keshav-space in #791
• Refactor swift mining pipeline for git deployment by @TG1999 in #789
• Refactor mine_cran to use MineCodeBasePipeline for git deployment by @ziadhany in #792
• Add support for mining cpan packageURLs by @AyanSinhaMahapatra in #731
• Migrate pypi pipeline to MineCodeBasePipeline by @AyanSinhaMahapatra in #794
• Add support to mine npm PackageURLs by @AyanSinhaMahapatra in #726
• Fix maven pipeline failure when there is no new increment to mine by @keshav-space in #801
• update federatedcode related commands by @JonoYang in #800
• Restructure readme by @DennisClark in #810
• Update index.rst by @DennisClark in #811
• Remove trailing whitespace by @JonoYang in #812
• Fix doc test failures by @AyanSinhaMahapatra in #813
• Refine docs by @pombredanne in #815
• Update README.rst to fix doc links by @DennisClark in #816
• Update traefik version #821 by @JonoYang in #822
• Exclude current package when checking for dupes by @JonoYang in #827
• Fix typo in introduction.rst by @ismaeeelxd in #825
• Fix spelling typos in docs, source, and scripts by @mr-raj12 in #818
• Fix docstring by @JonoYang in #829
• Retrieve SHA-256 hash for pypi packages by @rogu-beta in #765
• Fix minecode_pipelines bugs by @AyanSinhaMahapatra in #802
• Use traefik 2.11 #821 by @JonoYang in #831
• Extend indexing for maven by @chinyeungli in #805
• Fix console entry point for purldb CLI #622 by @adharsh0713 in #832
• Fix installation restore commands, detach typo, and clarify SCIO setup by @hiyasanghvi in #836
• Update purldb to support Python 3.13 by @kaokab33 in #830
• Fix NoneType iteration error in npm miner when versions are missing by @Monal-Reddy in #839
• Update formatting by @JonoYang in #842
• Update pypi pipeline by @JonoYang in #880
• Pin github workflows to specific commits by @JonoYang in #882
• Bump minecode-pipelines version by @JonoYang in https:...

v7.0.0

This is a major new release with all core code fragment and AI-generated code search features now available, including index-time matching.

• We also have improved support for on demand data collection for more ecosystems.
• The purldb-toolkit has moved to its own repo at https://github.com/aboutcode-org/purldb-toolkit
• The matchcode-toolkit has moved to its own repo at https://github.com/aboutcode-org/matchcode-toolkit

What's Changed

• Use JsonResponse #585 by @JonoYang in #586
• Index time matching by @JonoYang in #591
• Fixes Issue #587 by @MohannadAK in #593
• Collect pypi PURL ondemand #468 by @chinyeungli in #592
• Add PyPI to supported_ecosystems #599 by @JonoYang in #600
• Collect Cargo PURL ondemand #594 by @chinyeungli in #598
• Output verbose test failure #589 by @pombredanne in #590
• Collect Rubygems PURL ondemand #595 by @chinyeungli in #603
• Update skeleton by @JonoYang in #604
• add on demand package data collection for nuget #597 by @chinyeungli in #606
• Remove purldb-toolkit by @JonoYang in #610

New Contributors

• @MohannadAK made their first contribution in #593
• @chinyeungli made their first contribution in #592

Full Changelog: v6.0.0...v7.0.0

v6.0.0

Overview

This release is a major release with a new feature set for code fragments (aka. snippet) matching, including an indexing pipeline and early support for AI-generated code search

What's Changed

• #508 collect api default sort by @JonoYang in #513
• #512 #515 reformat reorganization by @JonoYang in #526
• Update CHANGELOG.rst and version number by @JonoYang in #529
• Update purl2vcs dep version in setup.cfg by @JonoYang in #530
• Increase Party.name field size to 255 by @JonoYang in #537
• Prefetch packages when using resource/filter_by_checksums by @JonoYang in #538
• Bump scancode-toolkit and scancode-io versions by @AyanSinhaMahapatra in #544
• #551 resources filter by checksums by @JonoYang in #553
• Fix error in django autoreload by @keshav-space in #556
• Add fragment matching to MatchCode by @pombredanne in #548
• Fixing the dead link by cross reference in documentation by @alok1304 in #550
• Add FederatedCode integration by @keshav-space in #557
• Update snippet matching by @JonoYang in #572
• Update query for matching packages and resources by @JonoYang in #564
• Index snippets by @JonoYang in #574
• 6.0.0 release prep by @JonoYang in #584

New Contributors

• @alok1304 made their first contribution in #550

Full Changelog: v5.0.1...v6.0.0

v5.0.1

Changelog

• Update link references of ownership from nexB to aboutcode-org

v5.0.0

Changelog

• Support indexing of PURLs listed in #326, #327, #328, #329 and #356.
• Support addon_pipelines for symbol and string collection in /api/collect endpoint. #393
• Store source_symbols and source_strings in extra_data field. #351
• Add /api/docs Swagger API documentation for API endpoints.
• Add /api/from_purl/purl2git endpoint to get a git repo for a purl.
• Add /api/to_purl/go endpoint to get a purl from a golang import string or a package string in go.mod.

What's Changed

• Update scancode-toolkit to v32.0.6 by @AyanSinhaMahapatra in #142
• Rename license file to use ABOUT conventions by @pombredanne in #145
• Improve docs by @pombredanne in #146
• Package set by @JonoYang in #141
• 139 api indexing multiple purls by @JonoYang in #148
• History field update by @JonoYang in #149
• Add support to find source repo for a package by @TG1999 in #137
• Checksum filtering endpoint by @JonoYang in #158
• Add timeout for fetching URL by @TG1999 in #156
• Increase priority on rescanned packages by @JonoYang in #161
• Add support for custom maven repository URLs by @AyanSinhaMahapatra in #164
• Raise caught exception in _call_scan_get_api #167 by @JonoYang in #168
• Add option to get enhanced package data in API by @TG1999 in #157
• Update spdx fields on packages #166 by @JonoYang in #169
• Update test expectations by @JonoYang in #172
• Always merge ancestor pom data #160 by @JonoYang in #171
• Set env var DATA_UPLOAD_MAX_NUMBER_FIELDS by @JonoYang in #174
• Add vers support in index_packages endpoint by @keshav-space in #170
• Ignore version-less purl when version range is missing by @keshav-space in #176
• Fix filter by checksums by @JonoYang in #180
• Improve version range resolution by @keshav-space in #183
• Don't process empty request in filter_by_checksums by @JonoYang in #184
• Bump scancode-toolkit to v32.0.7 by @AyanSinhaMahapatra in #194
• Bump scancode-toolkit to v32.0.8 by @AyanSinhaMahapatra in #196
• 179 focused maven visitor by @JonoYang in #192
• Remove lowercased purl fields on Package by @JonoYang in #210
• Support indexing of all the versions of a package in index_packages endpoint by @keshav-space in #214
• Remove search_vector field and index and create SearchVector per request by @JonoYang in #213
• Bump purldb dependencies by @JonoYang in #226
• Package field update method by @JonoYang in #222
• Create collect endpoint for ondemand task by @keshav-space in #227
• Add endpoint for PURL validation by @keshav-space in #228
• Updates for public PurlDB by @JonoYang in #234
• Improve handling of unsupported ecosystem by @keshav-space in #262
• 263 remove scan and fingerprint package from entrypoints by @JonoYang in #265
• Add validate endpoint to public API by @JonoYang in #266
• Create PURL CLI tool and library by @johnmhoran in #267
• Watch for packages (model and implementation) #244 by @pombredanne in #271
• Create server-side matching pipeline #224 by @JonoYang in #276
• Read .env for matchcode.io settings by @JonoYang in #282
• Add management command to watch all PURLs by @keshav-space in #268
• Update docker-compose_purldb.yml by @JonoYang in #287
• Get download URL from proper field by @JonoYang in #288
• Add API Documentation by @keshav-space in #232
• Ensure that PYTHON_EXE from Makefile is passed to configure by @JonoYang in #292
• Add cli commands, restructure data and update code and tests by @johnmhoran in #281
• Release purldb toolkit by @pombredanne in #295
• Update docker-compose_purldb.yml by @pombredanne in #294
• Remove tool_version key/value pairs from tests #296 by @johnmhoran in #297
• Update scan_and_fingerprint_package pipeline #49 #285 by @JonoYang in #302
• Update matchcode-toolkit readme by @JonoYang in #303
• Remove matchcode-toolkit from purldb repo by @JonoYang in #304
• Add API docs to public purldb urls by @JonoYang in #310
• Update validate and versions, refactor metadata and urls by @johnmhoran in #305
• Add golang purl service by @TG1999 in #316
• Get metadata for and scan debian packages from Purls by @AyanSinhaMahapatra in #300
• Add service for purl2git by @TG1999 in #299
• Support indexing of directory listed and GNU packages by @keshav-space in #318
• Expose new scan queue to many workers #49 by @pombredanne in #290
• Update data structure and create new set of mocked tests by @johnmhoran in #348
• Add collect_symbols to the default pipelines and store symbols in resources by @keshav-space in #351
• Disallow PUT requests on /api/watch endpoint by @keshav-space in #301
• Add inspect elf binaries pipeline by @TG1999 in #353
• Update sctk to V32.1 by @AyanSinhaMahapatra in #357
• Support conan package indexing by @keshav-space in #289
• purl2sym: Support indexing of Batch3, Batch4 and Batch5 packages by @keshav-space in #360
• Add endpoint to create or update a package set #205 by @404-geek in #350
• Fixes Github Mapper route by @404-geek in #370
• 362 update status task by @JonoYang in #375
• Add collect_source_strings to the default pipelines by @keshav-space in #384
• Add doc structure by @AyanSinhaMahapatra in #386
• Update gunicorn to 22.0.0 by @JonoYang in #392
• Docker compose update by @JonoYang in #394
• removed redundant PackageViewSet class code and added history field into package API nexB#389 nexB#221 by @404-geek in #390
• Add support for on demand addon_pipeline in /api/collect endpoints by @keshav-space in #393
• Add pure python pipeline for symbol collection by @keshav-space in #409
• Update api/matching to handle multiple url inputs by @JonoYang in #417
• Integrate D2D in purldb by @TG1999 in #404
• Create new purl2vcs library #374 by @pombredanne in #415
• Add tutorial for symbol and string collection using /api/collect endpoint by @keshav-space in #411

New Contributors

• @johnmhoran made their first contribution in #267
• @404-geek made their first contribution in #350

Full Changelog: v3.0.0...v5.0.0

Note

Use the command below to load the sample data for purl2sym.

pg_restore --clean -h localhost -p 5432 -U postgres -d packagedb packagedb_purl2sym_dump_v5_0_0

v3.0.0

This is a major release with:

• support for new package repositories and ecosystems
• new matching capabilities for exact files
• improved deployment with docker-compose
• a new and improved scan queue
• a new matchcode-toolkit for matching packaged as a ScanCode plugin
  This is now using the latest version of ScanCode toolkit

What's Changed

• Add fdroid packages by @pombredanne in #10
• update spdx-tools version in minecode by @armintaenzertng in #16
• Reorg code by @JonoYang in #15
• Add missing minecode migration by @JonoYang in #24
• Some minor fixes in minecode. by @35C4n0r in #19
• Documentation Updates. by @35C4n0r in #36
• Add Docker files and additional Visitors and Mappers by @JonoYang in #43
• Fixes in Debian Index Visitor function. by @35C4n0r in #41
• Fixes in setup.cfg for the python version specifier. by @35C4n0r in #48
• README file modified by @Sahajj in #45
• Scan queue update by @JonoYang in #50
• Update Package API to accept multi-filter #61 by @JonoYang in #67
• Update readme #51 by @JonoYang in #68
• Add support for npm priority queue by @keshav-space in #79
• 75 api index request by @pombredanne in #81
• 49 on demand mining by @pombredanne in #80
• PurlDB updates by @JonoYang in #97
• Purldb-updates by @JonoYang in #92
• Update resource license migration #73 by @JonoYang in #103
• Remove extra slash from projects API url by @JonoYang in #106
• Index hash fields on Resources and Packages by @JonoYang in #109
• Generic on demand handler by @JonoYang in #114
• Prep matchcode-toolkit codebase for release #113 by @JonoYang in #115
• Clean matchcode-toolkit directories in configure by @JonoYang in #116
• Prefetch related packages for Resources by @JonoYang in #119
• Api update by @JonoYang in #135
• Use README in RTD docs by @AyanSinhaMahapatra in #138

New Contributors

• @armintaenzertng made their first contribution in #16
• @35C4n0r made their first contribution in #19
• @Sahajj made their first contribution in #45
• @keshav-space made their first contribution in #79
• @AyanSinhaMahapatra made their first contribution in #138

Full Changelog: v2.0.0...v3.0.0

v2.0.0

What's Changed

• Streamline settings by @pombredanne in #1
• Add addtional visitors and mappers by @JonoYang in #6
• Add more visitors and mappers by @JonoYang in #7
• Prep PyPI release by @JonoYang in #8
• Add setup steps in README.rst by @JonoYang in #9

New Contributors

• @pombredanne made their first contribution in #1
• @JonoYang made their first contribution in #6

Full Changelog: https://github.com/nexB/purldb/commits/v2.0.0