Merge commoncode back into scancode-toolkit#5116
Open
AyanSinhaMahapatra wants to merge 803 commits into
Open
Conversation
- When you create a VirtualCodebase with multiple scans, we now prefix each
scan path with a codebase-1/, codebase-2/, etc. directory in addition to the
"virtual_root" shared root directory. Otherwise files data was overwritten
and inconsistent when each location "files" were sharing leading path
segments.
- When you create a VirtualCodebase with more than one Resource, we now recreate
the directory tree for any intermediary directory used in a path that is
otherwise missing from files path list.
In particular this behaviour changed when you create a VirtualCodebase from
a pervious Codebase created with a "full_root" argument. Previously, the
missing paths of a "full_root" Codebase were kept unchanged.
Noet that the VirtualCodebase has always ignored the "full_root" argument.
- The Resource has no rid (resource id) and no pid (parent id). Instead
we now use internally a simpler mapping of {path: Resource} object.
- The Codebase and VirtualCodebase are now iterable. Iterating on a codebase
is the same as a top-down walk.
- The Resource.path now never contains leading or trailing slash. We also
normalize the path everywhere. In particular this behaviour is visible when
you create a Codebase with a "full_root" argument. Previously, the paths of a
"full_root" Codebase were prefixed with a slash "/".
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
- When you create a VirtualCodebase with multiple scans, we now prefix each
scan path with a codebase-1/, codebase-2/, etc. directory in addition to the
"virtual_root" shared root directory. Otherwise files data was overwritten
and inconsistent when each location "files" were sharing leading path
segments.
- When you create a VirtualCodebase with more than one Resource, we now recreate
the directory tree for any intermediary directory used in a path that is
otherwise missing from files path list.
In particular this behaviour changed when you create a VirtualCodebase from
a pervious Codebase created with a "full_root" argument. Previously, the
missing paths of a "full_root" Codebase were kept unchanged.
Noet that the VirtualCodebase has always ignored the "full_root" argument.
- The Resource has no rid (resource id) and no pid (parent id). Instead
we now use internally a simpler mapping of {path: Resource} object.
- The Codebase and VirtualCodebase are now iterable. Iterating on a codebase
is the same as a top-down walk.
- The Resource.path now never contains leading or trailing slash. We also
normalize the path everywhere. In particular this behaviour is visible when
you create a Codebase with a "full_root" argument. Previously, the paths of a
"full_root" Codebase were prefixed with a slash "/".
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
This was removed in a previous commit. Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
The Codebase and VirtualCodebase no longer have a "full_root" and "strip_root" constructor arguments and attributes. These can still be passed but they will be ignored. These were needed only for path output and this is now were these arguments and code lives. - Resource.path is now always the plain path where the first segment is the last segment of the root location, e.g. the root fiename. - The Resource now has new "full_root_path" and "strip_root_path" properties that return the corresponding paths. - The Resource.to_dict and the new Codebase.to_list both have a new "full_root" and "strip_root" arguments - The Resource.get_path() method accepts "full_root" and "strip_root" arguments. - The Resource.create_child() method has been removed. - The "Codebase.original_location" attributed has been removed. No known users of commoncode used this. Also format code and organize imports. Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
The Codebase and VirtualCodebase no longer have a "full_root" and "strip_root" constructor arguments and attributes. These can still be passed but they will be ignored. These were needed only for path output and this is now were these arguments and code lives. - Resource.path is now always the plain path where the first segment is the last segment of the root location, e.g. the root fiename. - The Resource now has new "full_root_path" and "strip_root_path" properties that return the corresponding paths. - The Resource.to_dict and the new Codebase.to_list both have a new "full_root" and "strip_root" arguments - The Resource.get_path() method accepts "full_root" and "strip_root" arguments. - The Resource.create_child() method has been removed. - The "Codebase.original_location" attributed has been removed. No known users of commoncode used this. Also format code and organize imports. Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
The Codebase and VirtualCodebase no longer have a "full_root" and "strip_root" constructor arguments and attributes. These can still be passed but they will be ignored. These were needed only for path output and this is now were these arguments and code lives. - Resource.path is now always the plain path where the first segment is the last segment of the root location, e.g. the root fiename. - The Resource now has new "full_root_path" and "strip_root_path" properties that return the corresponding paths. - The Resource.to_dict and the new Codebase.to_list both have a new "full_root" and "strip_root" arguments - The Resource.get_path() method accepts "full_root" and "strip_root" arguments. - The Resource.create_child() method has been removed. - The "Codebase.original_location" attributed has been removed. No known users of commoncode used this. Also format code and organize imports. Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Jono Yang <jyang@nexb.com>
Use paths in codebase
Use paths in codebase
Use paths in codebase
Signed-off-by: Jono Yang <jyang@nexb.com>
Signed-off-by: Jono Yang <jyang@nexb.com>
Signed-off-by: Jono Yang <jyang@nexb.com>
* Test posix_only argument for safe_path
* Update CHANGELOG.rst
Signed-off-by: Jono Yang <jyang@nexb.com>
* Test posix_only argument for safe_path
* Update CHANGELOG.rst
Signed-off-by: Jono Yang <jyang@nexb.com>
* Test posix_only argument for safe_path
* Update CHANGELOG.rst
Signed-off-by: Jono Yang <jyang@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Make safe filename safe to use on POSIX
Make safe filename safe to use on POSIX
Make safe filename safe to use on POSIX
Signed-off-by: Ayan Sinha Mahapatra <asmahapatra@aboutcode.org>
Update system.py for py314
Update system.py for py314
Update system.py for py314
Remove API token from pypi publishing action
This is needed for python 3.14 support. Reference: aboutcode-org/commoncode#88 Signed-off-by: Ayan Sinha Mahapatra <asmahapatra@aboutcode.org>
This is needed for python 3.14 support. Reference: aboutcode-org/commoncode#88 Signed-off-by: Ayan Sinha Mahapatra <asmahapatra@aboutcode.org>
This is needed for python 3.14 support. Reference: aboutcode-org/commoncode#88 Signed-off-by: Ayan Sinha Mahapatra <asmahapatra@aboutcode.org>
Support setting the tarfile extract filter
Support setting the tarfile extract filter
Support setting the tarfile extract filter
Signed-off-by: Jono Yang <jyang@nexb.com>
Signed-off-by: Jono Yang <jyang@nexb.com>
Add sunos and haiku
Add sunos and haiku
Signed-off-by: Jono Yang <jyang@nexb.com>
Add openbsd to os_arch()
Signed-off-by: Ayan Sinha Mahapatra <asmahapatra@aboutcode.org>
Signed-off-by: Ayan Sinha Mahapatra <asmahapatra@aboutcode.org>
…plugin Signed-off-by: Ayan Sinha Mahapatra <asmahapatra@aboutcode.org>
Signed-off-by: Ayan Sinha Mahapatra <asmahapatra@aboutcode.org>
Signed-off-by: Ayan Sinha Mahapatra <asmahapatra@aboutcode.org>
…plugin Signed-off-by: Ayan Sinha Mahapatra <asmahapatra@aboutcode.org>
Signed-off-by: Ayan Sinha Mahapatra <asmahapatra@aboutcode.org>
Signed-off-by: Ayan Sinha Mahapatra <asmahapatra@aboutcode.org>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
There was a problem hiding this comment.
CodeQL found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.
Member
|
@AyanSinhaMahapatra Many of the .py files contain the comment:
If we are deprecating commoncode then we should remove that comment. |
JonoYang
approved these changes
Jun 5, 2026
Member
There was a problem hiding this comment.
@AyanSinhaMahapatra LGTM, though I think we should look into the CodeQL checks to have it not complain anytime we call open(). The macos tests failed at the git clone stage, so I am rerunning the tests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
16 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merge commoncode back with:
Tasks
Run tests locally to check for errors.