Ensure node is fully removed instead of partially.

[crypticC0der]

Previously on removing the node failing, which happens if the node is not in the cluster, the function exits. Which means the trust store isn't cleaned. This stops an issue wherin a prejoin hook could fail and cause the trust store to remain in a broken state which is not good.

Fixes: #784

[roosterfish]

Thanks for this! One remark from my side.

Also does this actually fix #784 or does it rather allow cleanly removing the member from the cluster after you hit the consistency error?

[roosterfish]

Can you please omit the part after the ,? Just logging Failed to remove cluster member from dqlite should be enough.
Also can you please add slog.Bool("force", force) and maybe also a slog.String("name", name) to give a bit more context which member couldn't be deleted?

[crypticC0der]

DeleteClusterMember is called with force on

microcluster/internal/rest/resources/control.go

Line 133 in a2735b4

err = internalClient.DeleteClusterMember(context.Background(), client, req.Name, "", true)

so this does fix it when it fails with prejoin, it also means deleting a cluster member will 100% fix the trust store if its somehow still broken

[Copilot]

Pull request overview

Fixes forced cluster-member removal so that cleanup continues even if the dqlite removal step fails, preventing leftover truststore state after an incomplete join/remove flow (per issue #784).

Changes:

• Allow clusterMemberDelete to proceed with subsequent cleanup when leader.Remove() fails during a forced removal.
• Add an error log when leader.Remove() fails but removal continues.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[roosterfish]

DeleteClusterMember is called with force on

microcluster/internal/rest/resources/control.go

Line 133 in a2735b4

err = internalClient.DeleteClusterMember(context.Background(), client, req.Name, "", true)

so this does fix it when it fails with prejoin, it also means deleting a cluster member will 100% fix the trust store if its somehow still broken

I see thanks. Please also append the package name as prefix to the commit message. For example see 065316a.

Also would it be feasible to extend any of the tests in example/test to simulate and test this? We could maybe modify the PreJoin hook in there to have a different outcome based on something we can set external to Microcluster like the existence of a file or an env variable being set like FAIL_PREJOIN=1.