fix(kafka source): reduce flakiness in rebalance acknowledgement tests

[thomasqueirozb]

Summary

CI run https://github.com/vectordotdev/vector/actions/runs/28872076555/job/85639356773 hit two failures in the kafka rebalance integration tests:

• drains_acknowledgements_during_rebalance_default_assignments: events3 was 0 because the first two consumers drained the whole 125k-message backlog before the third joined at KAFKA_CONSUMER_DELAY * 2. Bumped the default KAFKA_SEND_COUNT from 125,000 to 500,000 to restore a safety margin against faster CI runners.

• drains_acknowledgements_during_rebalance_sticky_assignments: total exceeded expect_count by 15-30% across the failing runs (e.g. 153328 vs 125000). This is expected at-least-once duplicate delivery from a partition being revoked and reassigned mid-rebalance before the prior offset commit lands, not lost data.

  The strict assert_eq!(total, expect_count) couldn't tell a legitimate duplicate apart from an actual dropped message, since a drop could be masked by an unrelated duplicate landing in another consumer's batch. Replaced it with two checks: total >= expect_count (no net loss), plus an exact check that every message index (each sent message carries a unique key) was seen by at least one consumer, which can't be fooled by duplicates.

Vector configuration

NA

How did you test this PR?

Ran scripts/run-integration-test.sh int kafka locally; both previously-flaky tests pass along with the rest of the kafka integration suite.

Change Type

• Bug fix
• New feature
• Dependencies
• Non-functional (chore, refactoring, docs)
• Performance

Is this a breaking change?

• Yes
• No

Does this PR include user facing changes?

• Yes. Please add a changelog fragment based on our guidelines.
• No. A maintainer will apply the no-changelog label to this PR.

References

• https://github.com/vectordotdev/vector/actions/runs/28872076555/job/85639356773

[datadog-vectordotdev]

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: b846c02 | Docs | Give us feedback!}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9d458c5489

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b93269ffad

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".