Compatibility Issue: ScanCode fails with Click 8.3.0 - "Secondary flag is not valid for non-boolean flag"

[karthiknew07]

Description

ScanCode 32.4.1 fails to start when Click 8.3.0 is installed, throwing a TypeError: Secondary flag is not valid for non-boolean flag. This appears to be a compatibility issue between ScanCode's CLI option definitions and stricter validation introduced in Click 8.3.0.

Error Message

Traceback (most recent call last):
  File "/tmp/scancode-env/bin/scancode", line 5, in <module>
    from scancode.cli import scancode
  File "/tmp/scancode-env/lib/python3.10/site-packages/scancode/cli.py", line 383, in <module>
    def scancode(
  File "/tmp/scancode-env/lib/python3.10/site-packages/click/decorators.py", line 374, in decorator
    _param_memo(f, cls(param_decls, **attrs))
  File "/tmp/scancode-env/lib/python3.10/site-packages/commoncode/cliutils.py", line 454, in __init__
    super(PluggableCommandLineOption, self).__init__(
  File "/tmp/scancode-env/lib/python3.10/site-packages/click/core.py", line 2793, in __init__
    raise TypeError("Secondary flag is not valid for non-boolean flag.")
TypeError: Secondary flag is not valid for non-boolean flag.

How To Reproduce

1. Install scancode-toolkit 32.4.1 via pip:

pip install scancode-toolkit==32.4.1

2. This will automatically install Click 8.3.0 (latest version)
3. Try to run any scancode command:

scancode --license <repo-path>

4. Observe the error above

Environment Information

• ScanCode Version: 32.4.1
• Click Version: 8.3.0 (problematic), 8.2.1 (works)
• Python Version: 3.10 (but likely affects other versions)
• OS: Linux x64 (but likely affects other platforms)
• Installation Method: pip install

Root Cause Analysis

The issue appears to be in commoncode/cliutils.pyline 454, where PluggableCommandLineOption is trying to create CLI options that Click 8.3.0 considers invalid. Specifically, Click 8.3.0 introduced stricter validation that prohibits secondary flags (like --flag/--no-flag) for non-boolean options.

Current Workaround

Downgrade Click to the previously working version:

pip install click==8.2.1 --upgrade

Suggested Solution

Update ScanCode's CLI option definitions to be compatible with Click 8.3.0's stricter validation rules. This likely involves:

1. Reviewing all CLI options defined in ```commoncode/cliutils.py``` and related files
2. Ensuring that only boolean flags use secondary flag patterns
3. Updating any non-boolean options that incorrectly use secondary flags
4. Testing with both Click 8.2.1 and 8.3.0 to ensure backward compatibility

Additional Context

• The pre-built ScanCode release archive (TAR) works because it includes its own venv with Click 8.2.1
• This affects users who install via pip, as they get the latest Click version automatically
• Multiple users have reported this issue independently

Dependency Version Information

Working combination (from release archive):

• scancode-toolkit: 32.4.1
• click: 8.2.1

Failing combination (from pip install):

• scancode-toolkit: 32.4.1
• click: 8.3.0

Request

Could the maintainers please:

1. Update the code to be compatible with Click 8.3.0
2. Consider pinning Click to a specific version range in requirements until compatibility is ensured
3. Update the installation documentation to mention this compatibility issue as a temporary workaround

Thank you for maintaining this valuable tool!

[danielthieleog]

Same as #4572 .

I get the same error in a similar call:

Traceback (most recent call last):
  File "/app/venv-scancode/bin/scancode", line 3, in <module>
    from scancode.cli import scancode
  File "/app/venv-scancode/lib/python3.13/site-packages/scancode/cli.py", line 376, in <module>
    @click.option(
     ~~~~~~~~~~~~^
        "--check-version/--no-check-version",
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<3 lines>...
        hidden=True,
        ^^^^^^^^^^^^
        help_group=cliutils.MISC_GROUP, sort_order=1000, cls=PluggableCommandLineOption)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/venv-scancode/lib/python3.13/site-packages/click/decorators.py", line 374, in decorator
    _param_memo(f, cls(param_decls, **attrs))
                   ~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/app/venv-scancode/lib/python3.13/site-packages/commoncode/cliutils.py", line 454, in __init__
    super(PluggableCommandLineOption, self).__init__(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        param_decls=param_decls,
        ^^^^^^^^^^^^^^^^^^^^^^^^
    ...<11 lines>...
        **kwargs,
        ^^^^^^^^^
    )
    ^
  File "/app/venv-scancode/lib/python3.13/site-packages/click/core.py", line 2793, in __init__
    raise TypeError("Secondary flag is not valid for non-boolean flag.")
TypeError: Secondary flag is not valid for non-boolean flag.

[karthiknew07]

Thanks for reporting this! I can confirm this is the same Python 3.13 compatibility issue as #4572.
I'm currently working on fixing this. The problem is that Click's validation has become stricter in Python 3.13 regarding boolean flags with the --option/--no-option syntax.
In the meantime, if you need an immediate workaround, you can:

• Use Python 3.12 instead of 3.13, or
• Wait for the fix which I'm preparing now

I'll update both this issue and #4572 once the fix is ready.

[danielthieleog]

Looks like the issue was fixed by commoncode-32.4.0, which contains a fix for click 8.3.0. At least my regression tests with a fresh installation using that library as a dependency are successful again.

[AyanSinhaMahapatra]

@karthiknew07 @danielthieleog thanks for the reports.

@danielthieleog this is indeed partially fixed with the new commoncode release, but with a few caveats that some plugins are not working and there are scan errors.

The main underlying issue was how https://github.com/pallets/click/releases/tag/8.3.0 changed the behaviour of default and flag_value, with a Sentinel value UNSET instead of None to differentiate None values set explicitly from not setting a value, and since we use a slightly modified version of click.option with PluggableCommandLineOption we were overriding this UNSET value with None and causing a whole bunch of resultant issues not limited to:

• TypeError: Secondary flag is not valid for non-boolean flag. for --check-version/--no-check-version options (which @karthiknew07 pointed as the root cause, but this was actually just a failure caused by the above issue)
• eager CLI options were not being evaluated first
• output CLI options were not working with file not found error
• failures because optional CLI options were being set without input from user, and this was triggering the conflicting options

The commoncode release fixed these issues, and I thought it would be sufficient, but we needed some more changes on the scancode side with #4591 which sets the default values for click options explicitly because otherwise we were:

• triggering optional CLI options without input (like the SPDX output options)
• failures from multiple options like --ignore

We would likely need a scancode release with the fixes for complete compatibility fixes, so I'm keeping the main issue #4572 open.

[karthiknew07]

Thank you @danielthieleog for testing and confirming the fix works in your environment, and thank you @AyanSinhaMahapatra for the detailed technical explanation!

I appreciate the comprehensive breakdown of the root cause. You're absolutely right - I had identified the symptom (the TypeError with boolean flags), but the underlying issue with Click 8.3.0's UNSET Sentinel value and how it interacts with PluggableCommandLineOption is the real culprit.

It's good to know that:

• commoncode-32.4.0 provides a partial fix
• PR Fix click compatibility issues #4591 addresses the remaining scancode-side issues
• A full scancode release will be needed for complete compatibility

I'll keep an eye on #4572 for the full resolution. Thanks for handling this so thoroughly!