Skip to content

Add novem --load tree sync with --dry-run#228

Merged
bjornars merged 1 commit into
novem-code:mainfrom
bjornars:bsn/load-dry-run-sync
Jun 3, 2026
Merged

Add novem --load tree sync with --dry-run#228
bjornars merged 1 commit into
novem-code:mainfrom
bjornars:bsn/load-dry-run-sync

Conversation

@bjornars
Copy link
Copy Markdown
Contributor

@bjornars bjornars commented Jun 3, 2026

Screenshot From 2026-06-03 15-23-14

What

Turns --load into a real tree sync and adds a --dry-run preview. --load/--dump let you round-trip a resource's file tree (plots, grids, docs, mails, jobs) to/from a local folder.

Previously --load walked the dumped folder and blindly PUT+POSTed every file — leaving stale remote files in place, rewriting unchanged ones, and reporting success regardless of what the server returned. Now it treats the local folder as the desired state:

  • create files that don't exist remotely
  • overwrite only files whose content differs (single-line values log the old -> new change instead of a byte count)
  • delete remote files no longer present locally
  • leave unchanged files untouched

--dry-run prints the planned actions without sending any request.

Notable behaviour

  • Only real file_content-backed files are synced. The directory listing flattens every leaf to type="file", so selection keys on the DELETE verb in each node's actions. Read-only files (no DELETE) and virtual/computed files like notifications (POST but no DELETE) are excluded — and virtual paths that exist remotely are tracked so a stale local copy is ignored rather than re-created on every run.
  • Shares (/shared/) round-trip as links created via PUT (they 405 on POST).
  • Tags (/tags/) are excluded — they're managed via the -t flag.
  • Each create/overwrite/delete is gated on its POST/DELETE status: failures print a FAILED line and are tallied under a N failed summary suffix instead of being silently counted as done.

Structure

The dump/load logic is shared by the vis and job APIs, so it lives in a NovemTreeSync mixin in novem/sync.py; each class supplies a base-path and a label hook. Filesystem code uses pathlib.

Testing

  • New tests/test_load_sync.py covers create/overwrite/delete/unchanged, dry-run, failure reporting, single-line diffs, and the virtual/shares/tags handling.
  • Full suite green (659 passed, 4 skipped); flake8 + mypy clean.
  • Manually verified end-to-end against the dev backend: dump → no-op round-trip, create/overwrite/delete, share creation via PUT, and idempotency on a stale dump.

`--load` previously walked a dumped folder and blindly PUT+POSTed every
file, leaving stale remote files in place, rewriting unchanged ones, and
reporting success regardless of the server response. Make it a real sync
that treats the local folder as the desired state of the resource:

  * create files that don't exist remotely
  * overwrite only files whose content differs (single-line values log
    the old -> new change instead of a byte count)
  * delete remote files no longer present locally
  * leave unchanged files untouched

`--dry-run` prints the planned actions without sending any request.

Only real file_content-backed files are synced. The directory listing
flattens every leaf to type="file", so selection keys on the DELETE verb
in each node's `actions`: read-only files (no DELETE) and virtual/computed
files like `notifications` (POST but no DELETE) are excluded. Virtual
paths that exist remotely are tracked so a stale local copy is ignored
rather than re-created every run. Shares (`/shared/`) round-trip as
links created via PUT (POST 405s on them); tags (`/tags/`) are excluded
entirely as they're managed via the -t flag.

Each create/overwrite/delete is gated on its POST/DELETE status: failures
print a FAILED line and are tallied under a `N failed` summary suffix
instead of being counted as done.

The dump/load logic is shared by the vis and job APIs, so it lives in a
NovemTreeSync mixin in novem/sync.py; each class supplies a base-path and
a label hook. Covered by tests/test_load_sync.py.
@bjornars bjornars force-pushed the bsn/load-dry-run-sync branch from 2d50ef2 to a0e28b8 Compare June 3, 2026 13:30
@myme
Copy link
Copy Markdown
Collaborator

myme commented Jun 3, 2026

Could've considered a --delete flag similar to rsync. But I guess this is also fine.

@bjornars bjornars merged commit 58b69d3 into novem-code:main Jun 3, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants