Skip to content

feat(migration): move to new my#1148

Draft
edospadoni wants to merge 1 commit into
mainfrom
feat/backup-my-cutover
Draft

feat(migration): move to new my#1148
edospadoni wants to merge 1 commit into
mainfrom
feat/backup-my-cutover

Conversation

@edospadoni
Copy link
Copy Markdown
Member

@edospadoni edospadoni commented Apr 23, 2026

Summary

Final migration PR that takes the ns8 cluster off backupd.nethesis.it, off my-old (/api/, /isa/) and off the my.nethesis.it/proxy/* translation layer, and moves it onto the my collect API with its own native credentials. nscom clusters keep using the legacy my.nethserver.com / backupd.nethesis.it infrastructure — that is explicitly out of scope for the my migration.

What it does

  • New /var/lib/nethserver/cluster/bin/migrate-to-my (bash, idempotent, nsent-only). Runs at the top of every send-cluster-backup / send-heartbeat / send-inventory invocation.

    1. Calls the translation proxy's /proxy/credentials with the legacy Basic-Auth pair.
    2. Atomically HSETs cluster/subscription in Redis with the new system_key / system_secret.
    3. Preserves the legacy pair under legacy_system_id / legacy_auth_token, asserts collect_url, stamps migrated_at and sets migrated='1'.
  • set-subscription.subscribe_nsent now POSTs my.nethesis.it/backend/api/systems/register, stores the returned system_key as system_id, writes collect_url + migrated='1' so a fresh subscription lands natively on the new my without any rotation.

  • set-subscription.terminate_nsent routes /api/Utils/freekey through the preserved legacy_* pair when available, so migrated clusters can still release their slot on my-old at unregister time.

  • get-subscription.fetch_subscription_info_nsent reads from collect /info with the rotated credentials and synthesises the legacy envelope the UI consumes — plan / expiration fall back to the organization name / null because the new my data model no longer tracks a subscription plan per system. Pre-migration clusters get a "pending" snapshot instead of a 500.

  • send-cluster-backup, send-heartbeat, send-inventory: single POST per script to $collect_url/{backups,heartbeat,inventory} on the nsent branch. nscom branches are byte-for-byte untouched.

  • Migration fingerprint in phonehome. print-phonehome now emits a facts.migration block with:

    • from_legacy_system_id: the pre-rotation UUID when the cluster was migrated through the proxy, null on fresh natively-subscribed clusters.
    • migrated_at: ISO 8601 timestamp of the rotation, null on fresh clusters.

    This lets my count how many legacy clusters have completed the rotation and decide when backupd.nethesis.it and the translation proxy can be decommissioned.

Resilience

A /proxy/credentials outage during an nsent cluster's upgrade window leaves the cluster on legacy credentials against collect, which returns 401. migrate-to-my is re-invoked every time one of the send-* services fires, so the cluster recovers automatically on the first successful rotation once the proxy is back up. Accepted trade-off: no dual-mode in the scripts, the simpler single-send path is preferred.

Blocking / coordination

Warning

Do not merge this PR until my is in production.
Once this ships, nsent clusters stop talking to my-old and backupd entirely.

Requires in order:

  1. feat(backup): configuration backup service (collect ingest + backend UI API) my#81 merged and deployed to production.
  2. feat(backup): dual-send cluster backup to my-new proxy #1146 (dual-send on ns8-core) merged and released.
  3. This PR merged and released.

Matching pair on the nethsecurity side: NethServer/nethsecurity#1609 (cutover) + NethServer/nethsecurity-ui#746 (UI consumption).

Post-merge operational follow-up

  • Monitor /proxy/credentials success rate during the cluster upgrade rollout.
  • On the my admin side, count inventories where facts.migration.from_legacy_system_id != null. When this count equals the rows in proxy_mappings, every legacy cluster has rotated and backupd.nethesis.it + the translation proxy can be decommissioned.

Tracking issue: NethServer/my#83.

@edospadoni edospadoni changed the title feat(migration): cut cluster over to my collect feat(migration): move to new my Apr 23, 2026
@edospadoni edospadoni force-pushed the feat/backup-my-cutover branch 2 times, most recently from acdff1f to a41552a Compare April 24, 2026 07:27
Final migration PR for ns8-core, mirror of the nethsecurity cutover
on NethServer/nethsecurity#1609. After this commit, nsent
clusters talk directly to the my collect API with native my
credentials; the legacy my.nethesis.it /api/ and /isa/ endpoints
and the /proxy/* translation routes are gone from the hot path.
nscom clusters keep using the legacy my.nethserver.com /
backupd.nethesis.it infrastructure — that is explicitly out of
scope for the my migration.

Credential rotation (existing nsent clusters):

- New /var/lib/nethserver/cluster/bin/migrate-to-my: idempotent
  bash one-shot, gated on provider=nsent. Calls the translation
  proxy's /proxy/credentials with the legacy Basic-Auth pair, reads
  back the mapped my system_key/system_secret and atomically
  HSETs cluster/subscription in Redis. Preserves the legacy pair
  under legacy_system_id / legacy_auth_token for audit and manual
  rollback, (re)asserts collect_url, and sets the migrated='1'
  marker that stops the helper from running again. A single HSET
  guarantees no half-migrated state.
- send-cluster-backup / send-heartbeat / send-inventory invoke
  migrate-to-my up front on the nsent branch so the first
  successful cron/timer tick flips a pre-migration cluster over.

Native my registration (fresh nsent subscriptions):

- set-subscription subscribe_nsent now POSTs
  my.nethesis.it/backend/api/systems/register with
  {system_secret: <pasted>} and stores the returned system_key as
  cluster/subscription system_id. collect_url is written alongside
  the VPN metadata and migrated='1' is set so migrate-to-my is a
  no-op. Community subscribe is untouched; it keeps using the
  dartagnan endpoint on my.nethserver.com.
- terminate_nsent routes /api/Utils/freekey through the preserved
  legacy_system_id / legacy_auth_token pair when available, so
  migrated clusters can still release their slot on my-old at
  unregister time. On pre-migration clusters behaviour is unchanged.
- get-subscription fetch_subscription_info_nsent queries collect
  /info with the rotated credentials and synthesises the legacy
  envelope the UI consumes (system_url, plan_name, expires,
  expire_date, status, with_remote_support). The new my data
  model no longer tracks a subscription plan at the system level,
  so plan_name falls back to the organization name and
  expires/expire_date to an "unbounded" default — the UI keeps
  rendering the same row layout without a KeyError. A
  pre-migration cluster falls back to a "pending" snapshot
  instead of raising, so the subscription page stays usable
  during the first rotation window.

Single-path send scripts:

- send-heartbeat nsent: POST $collect_url/heartbeat with native
  Basic-Auth. The primary my.nethesis.it/isa/heartbeats/store and
  the proxy shadow are gone; nscom continues on
  ${dartagnan_url}/machine/heartbeats/store.
- send-inventory nsent: POST $collect_url/inventory with a
  phonehome payload. The primary my.nethesis.it/isa/inventory
  path, the my-old /api/systems/info registration-date refresh
  and the proxy shadow are gone; nscom continues on
  ${dartagnan_url}/machine/inventories/store.
- send-cluster-backup: POST $collect_url/backups for nsent,
  $TYPE-less community path preserved via a dedicated nscom
  branch (backupd.nethesis.it/community/api/v2/backup/).

Failure mode:

- A /proxy/credentials outage during an nsent cluster's upgrade
  window leaves the cluster on legacy credentials against collect,
  which returns 401. migrate-to-my is re-invoked every time one
  of the send-* services fires, so the cluster recovers
  automatically once the proxy is back up. Accepted trade-off:
  no dual-mode in the scripts; the simpler single-send path is
  preferred.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant