NIFI-15862: Moved Processors to using Virtual Threads with a Semaphor…

[markap14]

…e bounding how many tasks can be run at once

Summary

NIFI-00000

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

• Apache NiFi Jira issue created

Pull Request Tracking

• Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
• Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000
• Pull request contains commits signed with a registered key indicating Verified status

Pull Request Formatting

• Pull Request based on current revision of the main branch
• Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

• Build completed using ./mvnw clean install -P contrib-check
  • JDK 21
  • JDK 25

Licensing

• New dependencies are compatible with the Apache License 2.0 according to the License Policy
• New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

• Documentation formatting appears as expected in rendered files

[pvillard31]

Should we at least log this at debug so that a recurring failure to delete nifi-thread-dump-* directories is discoverable?

[markap14]

Sure, that's reasonable.

[pvillard31]

What if the user renames the component and/or its parent group? would that be properly reflected?

[markap14]

The thread name would get generated whenever the Processor is started. It would not change. The only ways to change it would be to track all Virtual Thread objects so that setName could update them, or to constantly update it to whatever the current name is. Both introduce a decent bit of complexity for what I'd argue is fairly minimal gain. Especially since I don't think we even allow changing the name of the processor while it's running from the UI. I think the REST tier does allow it, but the UI would not, though I may be mistaken.

[markap14]

[claude-opus-4.7] @pvillard31 thanks for the latest round of feedback. The new commit (2c0e041) addresses your remaining comments and rebases onto current main:

• ThreadDumpTask now logs at debug rather than silently swallowing the cleanup IOException, so a recurring failure to delete nifi-thread-dump-* directories is discoverable.
• On the thread-name-after-rename question: after some prototyping we decided not to refresh the virtual thread's name inside the scheduling loop. Resetting the name on every iteration adds an allocation-heavy buildThreadName call to a hot path that may run at zero scheduling period, and cache-and-compare workarounds add complexity for what is purely a diagnostic nicety. The name resolved at scheduling time (component name + type + id + parent-group name + task index) still contains the immutable component id, so threads remain identifiable in dumps. If the user wants the new name reflected, stopping and restarting the component is sufficient.

The earlier review-feedback commit (0abffe2) is also still on the branch and covers the prior round of comments: atomic schedule rollback (#3), thread tracking + shutdown() interrupt with FlowController kill-path hook (#5), setMaxThreadCount synchronization (#2), and the response on runOnce polling (#1). Force push was required to incorporate the main rebase.

[markap14]

[claude-opus-4.7] @markap14 important: I had to revert my last push.

When you asked me to rebase against main, my local git rebase rewrote your two existing commits and stripped their GPG signatures (this clone is not configured to sign commits, and I am not allowed by my operating rules to modify the git config or use --no-gpg-sign/-S on your behalf). After the force-push, Validate Signed Commits failed on all three commits — both yours and the new ThreadDumpTask commit I added.

I have force-pushed the branch back to the previous tip 8f34b357be5, so your original signed commits are intact and the verification has returned to valid for both of them. The PR is back to where it was before I touched it today.

What still needs to happen, in your hands:

1. The ThreadDumpTask debug-log change from the latest review-feedback round. The one-line patch is:

-                } catch (final IOException ignored) {
+                } catch (final IOException e) {
+                    logger.debug(\"Failed to delete temporary thread-dump directory {}\", tempDirectory, e);
                 }

(plus the org.slf4j.Logger/LoggerFactory import and private static final Logger logger = LoggerFactory.getLogger(ThreadDumpTask.class); field at the top of the class). I can post the full unified diff if useful.

2. Whether to actually rebase against main. If you do, run it locally so your GPG signatures survive — your commits will need to be signed by you, not me.

Sorry for the churn. Watcher subagent is stopping until you let me know how you'd like to proceed.