Skip to content

fix(server): Install rclone on servers created from an image - #7253

Open
regdocs wants to merge 2 commits into
developfrom
fix/install-rclone-on-new-servers
Open

fix(server): Install rclone on servers created from an image#7253
regdocs wants to merge 2 commits into
developfrom
fix/install-rclone-on-new-servers

Conversation

@regdocs

@regdocs regdocs commented Aug 18, 2026

Copy link
Copy Markdown
Member

Why

Install Rclone was never running on servers created from the dashboard.

The install_rclone role is only wired into the full setup playbooks (press/playbooks/server.yml, unified_server.yml), which are played by Server._setup_server() / _setup_unified_server(). Dashboard-created servers never take that path: api/server.py:new()Cluster.create_server()VirtualMachine.create_server(), and since those VMs boot from a prebaked virtual_machine_image, the server doc is inserted with is_server_setup already set to true, so server.yml is skipped entirely.

The post-boot work for those servers is done by the Create Server press job. Server.setup_rclone() has existed since cc10819 but had no automatic caller, so it only ever ran if someone invoked it by hand.

What changed

A dedicated, synchronous rclone step in the Create Server job. Rather than adding setup_rclone() to set_additional_config(), which only enqueues its plays, the job gets its own @task:

@task(queue="long", timeout=1200)
def enable_backup_streaming(self):
    if self.server_type != "Server":
        return

    self.server_doc.enable_backup_streaming()

It runs ahead of set_additional_config() for two reasons: provisioning has to actually wait for rclone before anything advertises streaming, and that step enqueues filebeat, wazuh, earlyoom, ncdu and cadvisor — several of which use apt, so running rclone inside that pool would have it contending for the dpkg lock. The server_type != "Server" guard also covers unified servers, since those are Server docs.

stream_backups is set by that step, not by a field default. The flag is a consequence of a successful install:

def enable_backup_streaming(self):
    """Install rclone, then let this server stream offsite backups.

    The agent rejects a streamed backup when rclone is missing, so the flag
    must not be set until the play has actually succeeded.
    """
    play = self._setup_rclone()
    if play and play.status == "Success":
        self.db_set("stream_backups", True)

Defaulting the field to 1 would have been wrong: the doc is inserted with streaming on while rclone is still absent, so an offsite backup landing in that window — or on a server where the play failed — advertises streaming and gets rejected by the agent for a missing binary. _setup_rclone() now returns its Ansible Play instead of discarding it; Ansible.run() folds task failures into the play's status rather than raising (runner.py:252-266), so status == "Success" is the real signal, and the play and guard covers the exception path, which still logs as before.

A failed play leaves streaming off rather than failing the job. Raising would fail the press job and block is_provisioning_press_job_completed, marking a healthy server un-provisioned over an optional backup optimization. The failed Ansible Play record is the signal instead.

setup_rclone() gets an explicit queue and timeout (second commit). It inherited enqueue_doc's defaults — queue="default", timeout=300 (background_jobs.py:218) — so a play that has to SSH in, fetch a deb from GitHub and run apt got 300s on the queue that also carries agent job polling. Now queue="long", timeout=1200, matching the other playbook enqueues in the file (_install_nginx, _setup_auditd, _set_docker_mtu) and the ceiling on the new press job task.

Tests

Three in TestServer, following the existing test_install_marks_wazuh_agent_installed_on_successful_play pattern: streaming enabled after a successful play, and staying disabled both when the play returns Failure and when Ansible.run raises.

Notes for review

  • Servers provisioned through server.yml still won't stream. That path plays the install_rclone role but never sets the flag, so bootstrap and manual "Setup Server" servers stay opted out until someone sets it. Left alone deliberately — the reported bug is the dashboard/image path. Happy to have _setup_server()'s success branch set it too if that's wanted.
  • Existing servers are untouched. They keep stream_backups = 0 and have no rclone. Enabling streaming on any of them needs _setup_rclone() run first, and there's currently no desk button for it (server.js exposes no action for setup_rclone or setup_ncdu), so today it's a console call. A button and/or a backfill patch would be worth adding separately.
  • Two unrelated lines touched. settings.wazuh_api_password = "pass" in test_server.py got a # pragma: allowlist secret. It's a pre-existing false positive that detect-secrets only surfaces once the file is staged, and it blocks committing any change to that file.

🤖 Generated with Claude Code

@regdocs
regdocs requested a review from adityahase as a code owner August 18, 2026 07:38
@greptile-apps

greptile-apps Bot commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Reviews (3): Last reviewed commit: "fix(server): Enqueue rclone install on t..." | Re-trigger Greptile

Comment thread press/press/doctype/server/server.py Outdated
@codecov-commenter

codecov-commenter commented Aug 18, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 89.47368% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 53.91%. Comparing base (739ee64) to head (66d990f).
⚠️ Report is 5 commits behind head on develop.

Files with missing lines Patch % Lines
press/press/doctype/server/server.py 75.00% 2 Missing ⚠️
press/press/doctype/server/test_server.py 91.66% 2 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #7253      +/-   ##
===========================================
- Coverage    59.43%   53.91%   -5.52%     
===========================================
  Files         1024     1024              
  Lines        94103    94168      +65     
  Branches      1216     1067     -149     
===========================================
- Hits         55929    50772    -5157     
- Misses       38151    43373    +5222     
  Partials        23       23              
Flag Coverage Δ
dashboard 64.72% <ø> (-22.61%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Dashboard-created servers boot from a prebaked VMI, so they're inserted
with is_server_setup already true and never play server.yml — where the
install_rclone role lives. Give the Create Server press job its own rclone
step, ahead of set_additional_config so it doesn't contend for the dpkg
lock with the apt plays that step enqueues.

That step also sets stream_backups, rather than the field defaulting to 1:
the agent rejects a streamed backup when rclone is missing, so the flag
can't be set before the play succeeds. A failed play leaves streaming off
instead of failing the job, since a healthy server shouldn't be marked
un-provisioned over an optional backup path.

setup_rclone() has had no automatic caller since cc10819.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@regdocs
regdocs force-pushed the fix/install-rclone-on-new-servers branch from 62d1849 to f7cd871 Compare August 18, 2026 07:52
setup_rclone() inherited enqueue_doc's defaults, so a play that has to SSH
in, fetch a deb from GitHub and run apt got 300s on the queue that also
carries agent job polling. Matches the other playbook enqueues here
(_install_nginx, _setup_auditd, _set_docker_mtu).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants