feat(migration): move to new my#1148
Draft
edospadoni wants to merge 1 commit into
Draft
Conversation
acdff1f to
a41552a
Compare
Final migration PR for ns8-core, mirror of the nethsecurity cutover on NethServer/nethsecurity#1609. After this commit, nsent clusters talk directly to the my collect API with native my credentials; the legacy my.nethesis.it /api/ and /isa/ endpoints and the /proxy/* translation routes are gone from the hot path. nscom clusters keep using the legacy my.nethserver.com / backupd.nethesis.it infrastructure — that is explicitly out of scope for the my migration. Credential rotation (existing nsent clusters): - New /var/lib/nethserver/cluster/bin/migrate-to-my: idempotent bash one-shot, gated on provider=nsent. Calls the translation proxy's /proxy/credentials with the legacy Basic-Auth pair, reads back the mapped my system_key/system_secret and atomically HSETs cluster/subscription in Redis. Preserves the legacy pair under legacy_system_id / legacy_auth_token for audit and manual rollback, (re)asserts collect_url, and sets the migrated='1' marker that stops the helper from running again. A single HSET guarantees no half-migrated state. - send-cluster-backup / send-heartbeat / send-inventory invoke migrate-to-my up front on the nsent branch so the first successful cron/timer tick flips a pre-migration cluster over. Native my registration (fresh nsent subscriptions): - set-subscription subscribe_nsent now POSTs my.nethesis.it/backend/api/systems/register with {system_secret: <pasted>} and stores the returned system_key as cluster/subscription system_id. collect_url is written alongside the VPN metadata and migrated='1' is set so migrate-to-my is a no-op. Community subscribe is untouched; it keeps using the dartagnan endpoint on my.nethserver.com. - terminate_nsent routes /api/Utils/freekey through the preserved legacy_system_id / legacy_auth_token pair when available, so migrated clusters can still release their slot on my-old at unregister time. On pre-migration clusters behaviour is unchanged. - get-subscription fetch_subscription_info_nsent queries collect /info with the rotated credentials and synthesises the legacy envelope the UI consumes (system_url, plan_name, expires, expire_date, status, with_remote_support). The new my data model no longer tracks a subscription plan at the system level, so plan_name falls back to the organization name and expires/expire_date to an "unbounded" default — the UI keeps rendering the same row layout without a KeyError. A pre-migration cluster falls back to a "pending" snapshot instead of raising, so the subscription page stays usable during the first rotation window. Single-path send scripts: - send-heartbeat nsent: POST $collect_url/heartbeat with native Basic-Auth. The primary my.nethesis.it/isa/heartbeats/store and the proxy shadow are gone; nscom continues on ${dartagnan_url}/machine/heartbeats/store. - send-inventory nsent: POST $collect_url/inventory with a phonehome payload. The primary my.nethesis.it/isa/inventory path, the my-old /api/systems/info registration-date refresh and the proxy shadow are gone; nscom continues on ${dartagnan_url}/machine/inventories/store. - send-cluster-backup: POST $collect_url/backups for nsent, $TYPE-less community path preserved via a dedicated nscom branch (backupd.nethesis.it/community/api/v2/backup/). Failure mode: - A /proxy/credentials outage during an nsent cluster's upgrade window leaves the cluster on legacy credentials against collect, which returns 401. migrate-to-my is re-invoked every time one of the send-* services fires, so the cluster recovers automatically once the proxy is back up. Accepted trade-off: no dual-mode in the scripts; the simpler single-send path is preferred.
a41552a to
5afaa91
Compare
20 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Final migration PR that takes the ns8 cluster off
backupd.nethesis.it, off my-old (/api/,/isa/) and off themy.nethesis.it/proxy/*translation layer, and moves it onto themycollect API with its own native credentials.nscomclusters keep using the legacymy.nethserver.com/backupd.nethesis.itinfrastructure — that is explicitly out of scope for the my migration.What it does
New
/var/lib/nethserver/cluster/bin/migrate-to-my(bash, idempotent,nsent-only). Runs at the top of everysend-cluster-backup/send-heartbeat/send-inventoryinvocation./proxy/credentialswith the legacy Basic-Auth pair.cluster/subscriptionin Redis with the newsystem_key/system_secret.legacy_system_id/legacy_auth_token, assertscollect_url, stampsmigrated_atand setsmigrated='1'.set-subscription.subscribe_nsentnow POSTsmy.nethesis.it/backend/api/systems/register, stores the returnedsystem_keyassystem_id, writescollect_url+migrated='1'so a fresh subscription lands natively on the new my without any rotation.set-subscription.terminate_nsentroutes/api/Utils/freekeythrough the preservedlegacy_*pair when available, so migrated clusters can still release their slot on my-old at unregister time.get-subscription.fetch_subscription_info_nsentreads from collect/infowith the rotated credentials and synthesises the legacy envelope the UI consumes — plan / expiration fall back to the organization name / null because the new my data model no longer tracks a subscription plan per system. Pre-migration clusters get a "pending" snapshot instead of a 500.send-cluster-backup,send-heartbeat,send-inventory: single POST per script to$collect_url/{backups,heartbeat,inventory}on thensentbranch.nscombranches are byte-for-byte untouched.Migration fingerprint in phonehome.
print-phonehomenow emits afacts.migrationblock with:from_legacy_system_id: the pre-rotation UUID when the cluster was migrated through the proxy,nullon fresh natively-subscribed clusters.migrated_at: ISO 8601 timestamp of the rotation,nullon fresh clusters.This lets
mycount how many legacy clusters have completed the rotation and decide whenbackupd.nethesis.itand the translation proxy can be decommissioned.Resilience
A
/proxy/credentialsoutage during annsentcluster's upgrade window leaves the cluster on legacy credentials against collect, which returns401.migrate-to-myis re-invoked every time one of the send-* services fires, so the cluster recovers automatically on the first successful rotation once the proxy is back up. Accepted trade-off: no dual-mode in the scripts, the simpler single-send path is preferred.Blocking / coordination
Warning
Do not merge this PR until
myis in production.Once this ships,
nsentclusters stop talking tomy-oldandbackupdentirely.Requires in order:
Matching pair on the nethsecurity side: NethServer/nethsecurity#1609 (cutover) + NethServer/nethsecurity-ui#746 (UI consumption).
Post-merge operational follow-up
/proxy/credentialssuccess rate during the cluster upgrade rollout.facts.migration.from_legacy_system_id != null. When this count equals the rows inproxy_mappings, every legacy cluster has rotated andbackupd.nethesis.it+ the translation proxy can be decommissioned.Tracking issue: NethServer/my#83.