Skip to content

Fix ClusterMigrateWrite race condition: wait for gossip propagation after Meet()#1609

Closed
Copilot wants to merge 2 commits intovazois/mmrt-devfrom
copilot/sub-pr-1556
Closed

Fix ClusterMigrateWrite race condition: wait for gossip propagation after Meet()#1609
Copilot wants to merge 2 commits intovazois/mmrt-devfrom
copilot/sub-pr-1556

Conversation

Copy link
Contributor

Copilot AI commented Mar 5, 2026

ClusterMigrateTests(False).ClusterMigrateWrite was intermittently failing with "ERR I don't know about node <sourceNodeId>" because SetSlot IMPORTING was issued on the target node before gossip had propagated the source node's identity to it.

Fix

Added WaitUntilNodeIsKnown(targetNodeIndex, sourceNodeIndex) between Meet() and the subsequent SetSlot calls — consistent with the pattern already used in ClusterMigrateSlotWalk and similar tests.

context.clusterTestUtils.Meet(sourceNodeIndex, targetNodeIndex, logger: context.logger);
// Wait for target to learn about source via gossip before issuing SetSlot IMPORTING
context.clusterTestUtils.WaitUntilNodeIsKnown(targetNodeIndex, sourceNodeIndex, logger: context.logger);

var sourceNodeId = context.clusterTestUtils.GetNodeIdFromNode(sourceNodeIndex, context.logger);

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Co-authored-by: vazois <96085550+vazois@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Mar 5, 2026

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • this-hostname-will-never-resolve.invalid
    • Triggering command: /usr/share/dotnet/dotnet /usr/share/dotnet/dotnet exec --runtimeconfig /home/REDACTED/work/garnet/garnet/test/Garnet.test.cluster/bin/Release/net10.0/Garnet.test.cluster.runtimeconfig.json --depsfile /home/REDACTED/work/garnet/garnet/test/Garnet.test.cluster/bin/Release/net10.0/Garnet.test.cluster.deps.json /home/REDACTED/work/garnet/garnet/test/Garnet.test.cluster/bin/Release/net10.0/testhost.dll --port 45497 --endpoint 127.0.0.1:045497 --role client --parentprocessid 4466 --telemetryoptedin false (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI changed the title [WIP] Add support for multi-log parallel replication Fix ClusterMigrateWrite race condition: wait for gossip propagation after Meet() Mar 5, 2026
@vazois
Copy link
Contributor

vazois commented Mar 5, 2026

@copilot, run ClusterMigrateWrite test multiple iterations to validate that the test did not regress or any new issues appeared.

Copy link
Contributor Author

Copilot AI commented Mar 5, 2026

@copilot, run ClusterMigrateWrite test multiple iterations to validate that the test did not regress or any new issues appeared.

Ran ClusterMigrateWrite 10 consecutive times — all passed (10/10). Also ran the full ClusterMigrateTests fixture (24 tests) with no failures or regressions.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • this-hostname-will-never-resolve.invalid
    • Triggering command: /usr/share/dotnet/dotnet /usr/share/dotnet/dotnet exec --runtimeconfig /home/REDACTED/work/garnet/garnet/test/Garnet.test.cluster/bin/Release/net10.0/Garnet.test.cluster.runtimeconfig.json --depsfile /home/REDACTED/work/garnet/garnet/test/Garnet.test.cluster/bin/Release/net10.0/Garnet.test.cluster.deps.json /home/REDACTED/work/garnet/garnet/test/Garnet.test.cluster/bin/Release/net10.0/testhost.dll --port 34229 --endpoint 127.0.0.1:034229 --role client --parentprocessid 6566 --telemetryoptedin false (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

@vazois vazois closed this Mar 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants