NIFI-15449 - NAR deletion blocks indefinitely when Python processor is initializing #10753
+1,063
−163
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…s initializing Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
exceptionfactory
requested changes
Feb 1, 2026
Contributor
exceptionfactory
left a comment
exceptionfactory
left a comment
There was a problem hiding this comment.
Thanks for working on this issue @pvillard31. The general strategy looks good. I noted a few initial recommendations. I also recommend reviewing most of the new log and exception messages to include a relevant detail, such as the Process ID or component ID where applicable.
| private volatile String identifier; | ||
| private volatile PythonController controller; | ||
| private volatile CompletableFuture<Void> initializationFuture; | ||
| private volatile boolean cancelled = false; |
Contributor
There was a problem hiding this comment.
Although either is technically acceptable English, other references in the project use canceled with one letter L, so I recommend using that form throughout these changes.
| // This allows the venv creation to be interrupted when the process is being shut down | ||
| while (!venvProcess.waitFor(1, TimeUnit.SECONDS)) { | ||
| if (isShutdown()) { | ||
| logger.info("Interrupting Python Virtual Environment creation due to shutdown"); |
Contributor
There was a problem hiding this comment.
It would be helpful to include the Process ID in this log.
| if (isShutdown()) { | ||
| logger.info("Interrupting Python Virtual Environment creation due to shutdown"); | ||
| venvProcess.destroyForcibly(); | ||
| throw new IOException("Python process shutdown during virtual environment creation"); |
Contributor
There was a problem hiding this comment.
The Process ID would be helpful to include in this message.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
NIFI-15449 - NAR deletion blocks indefinitely when Python processor is initializing
The issue is caused by:
StandardPythonBridgeandStandardExtensionDiscoveringManager:StandardExtensionDiscoveringManagerlock -> waits forStandardPythonBridgelockStandardPythonBridgelock -> callsgetNarDirectories()which waits forStandardExtensionDiscoveringManagerlockChanges:
StandardPythonBridgesynchronized block, ensuring locks are always acquired in a consistent order.isShutdown()checks during venv creation and dependency installation loops in PythonProcessCANCELLEDstate toAsyncLoadedProcessor.LoadStateand acancelLoading()methodStandardPythonProcessorBridgethat sets a flag checked during initializationstart()is called) so they can be properly shut down during NAR deletionTracking
Please complete the following tracking steps prior to pull request creation.
Issue Tracking
Pull Request Tracking
NIFI-00000NIFI-00000VerifiedstatusPull Request Formatting
mainbranchVerification
Please indicate the verification steps performed prior to pull request creation.
Build
./mvnw clean install -P contrib-checkLicensing
LICENSEandNOTICEfilesDocumentation