Skip to content

fix(mysql): fix connectors getting stuck in "connecting" state#15909

Closed
savonarola wants to merge 8 commits intoemqx:release-586from
savonarola:250912-fix-mysql-getting-stuck
Closed

fix(mysql): fix connectors getting stuck in "connecting" state#15909
savonarola wants to merge 8 commits intoemqx:release-586from
savonarola:250912-fix-mysql-getting-stuck

Conversation

@savonarola
Copy link
Copy Markdown
Contributor

No description provided.

@savonarola savonarola changed the base branch from release-58 to release-586 September 12, 2025 10:59
end;
false ->
?status_connecting
{error, timeout} ->
Copy link
Copy Markdown
Contributor Author

@savonarola savonarola Sep 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, in EMQX, MySQL driver is configured with infinity timeout.
So, if a tcp connection breaks ungracefully, any request (including health checks) just makes workers stuck.
The subsequent healthcheck requests also fail with timeout.
So in case of healthcheck timeout we report disconnected status to trigger reconnection.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Strange that the mysql driver does not attempt to reconnect or die if the tcp connection breaks 🤔

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its just a guess and an attempt to fix a problem of a concrete customer.

The thing is that the connection may break without any fin/rst packets, traffic just stops to arrive. A reconnect solves the problem, but the connector stucks in connecting state till the traffic unblocks.

@savonarola savonarola closed this Sep 12, 2025
@savonarola savonarola reopened this Sep 12, 2025
Comment thread rebar.config
{grpc, {git, "https://github.com/emqx/grpc-erl", {tag, "0.6.12"}}},
{minirest, {git, "https://github.com/emqx/minirest", {tag, "1.4.4"}}},
{ecpool, {git, "https://github.com/emqx/ecpool", {tag, "0.6.1"}}},
{ecpool, {git, "https://github.com/emqx/ecpool", {tag, "0.6.1-hotfix"}}},
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@savonarola savonarola force-pushed the 250912-fix-mysql-getting-stuck branch from 790524c to c8eef2b Compare September 12, 2025 11:18
@savonarola savonarola force-pushed the 250912-fix-mysql-getting-stuck branch from e81194b to e7af46d Compare September 12, 2025 12:38
@zmstone zmstone deleted the branch emqx:release-586 January 7, 2026 15:14
@zmstone zmstone closed this Jan 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants