Fix slow, fragile OpenClaw upgrade rollback in installer - #107
Merged
Conversation
The installer could hang for 20+ minutes and then fail while recovering an interrupted OpenClaw upgrade, leaving a permanent 'rollback-failed' state that blocked every subsequent install. Three fixes: 1. fsync read-only files + parallelize. _fsync_file opened files 'rb+', which fails with WinError 5 on the read-only files npm ships, aborting rollback. It now clears the read-only attribute, fsyncs, and restores it, so every file is still durably flushed. _fsync_payload_tree now fsyncs via a thread pool instead of a serial ~90 files/sec crawl. 2. Self-heal a failed rollback. Add OpenClawUpgradeTransaction.discard() and call it (instead of leaving rollback-failed) so a rollback that cannot complete drops its manifest, lock, and backup and releases the retained lock. The live install is left intact (staged copy + atomic rename) and later steps reinstall OpenClaw, so future installs are no longer permanently blocked by UpgradeInProgressError. 3. Show what the installer is doing. Add a progress_detail sub-status line under the progress bar, fed by backup/restore file operations (e.g. 'Restoring OpenClaw files (12,400/36,129 files)'), so long file ops no longer look frozen. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #108
Problem
Installing MicroClaw on a machine with a previously-interrupted OpenClaw upgrade could hang at ~18–20% for 20+ minutes and then fail, leaving a permanent dead-end that blocked every subsequent install.
Root causes, diagnosed with
py-spyon a frozen installer:_fsync_fileopened files with"rb+".os.fsync(→FlushFileBufferson Windows) requires write access, but npm packages ship many read-only files. Opening them for writing failed withPermissionError/WinError 5(access denied) and aborted the rollback.mark_rollback_failed()left the manifest inrollback-failed(which stays inRECOVERABLE_PHASES) and retained the upgrade lock, so the next install re-entered recovery and failed again withan OpenClaw upgrade is already in progress.Fixes
1. fsync read-only files + parallelize
_fsync_filenow catchesPermissionErroron Windows, temporarily clears the read-only attribute, fsyncs, and restores the original mode infinally. Every file is still durably flushed — no loss of crash-safety._fsync_payload_treefsyncs via aThreadPoolExecutor(fsync is a blocking C call that releases the GIL) instead of a serial crawl, cutting the multi-minute walk dramatically. Used by both backup and restore.2. Self-heal a failed rollback
OpenClawUpgradeTransaction.discard()drops the on-disk manifest, lock file, and backup tree, and releases the retained lock._rollback_openclaw_transactionnow discards (instead of leavingrollback-failed) when rollback can't complete. Because restores stage a copy and only swap the live tree with a final atomic rename, a failure during staging/fsync leaves the live installation intact; later steps reinstall OpenClaw. Future installs are no longer permanently blocked byUpgradeInProgressError.discard()'s tree removal tolerates read-only files (npm again) so cleanup itself can't fail withWinError 5.3. Show what the installer is doing
progress_detailsub-status line under the progress bar in the installer UI.Restoring OpenClaw files (12,400/36,129 files), so long file ops never look frozen.Testing
python -m unittest discover -s tests→ 107 tests pass.ruff check/ruff format --check→ clean._fsync_fileflushes a read-only file and restores its attribute._fsync_payload_treereports progress for every file.rollback-failedtransaction isdiscard()ed so a fresh install can acquire the lock and proceed.Related installer-robustness issues: #105 (MAX_PATH during backup), #93 (npm registry block, handled in #106).