Skip to content

refactor: switch diskNormalized strategy to be more tune-able#19663

Merged
jtuglu1 merged 1 commit into
apache:masterfrom
jtuglu1:tune-disk-normalized-cost-balancer
Jul 8, 2026
Merged

refactor: switch diskNormalized strategy to be more tune-able#19663
jtuglu1 merged 1 commit into
apache:masterfrom
jtuglu1:tune-disk-normalized-cost-balancer

Conversation

@jtuglu1

@jtuglu1 jtuglu1 commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Description

Goal here is to make the diskNormalized strategy more tunable. Given the new, working version has not yet been included in a major release I've done some refactoring to make things easier to tune based on experience of running this in production.

The primary changes are making the utilizationThreshold more intuitive: increasing the threshold increases the "tolerance" of the strategy, while decreasing pushes nodes' disk utilization closer together. Prior to these changes, the tuning of the threshold config wasn't as straightforward and could result in under/over-aggressive segment movement. With these changes, you can expect the config value to be approximately equal to the absolute difference between the min/max disk utilized servers' utilization within a tier.

Release note

Make the diskNormalized strategy more tunable. The primary changes are making the utilizationThreshold more intuitive: increasing the threshold increases the "tolerance" of the strategy, while decreasing pushes nodes' disk utilization closer together.


This PR has:

  • been self-reviewed.
  • added documentation for new or modified features or behaviors.
  • a release note entry in the PR description.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added or updated version, license, or notice information in licenses.yaml
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • added integration tests.
  • been tested in a test Druid cluster.

@jtuglu1 jtuglu1 added this to the 38.0.0 milestone Jul 7, 2026
@jtuglu1 jtuglu1 requested a review from kfaraz July 8, 2026 02:04

@FrankChen021 FrankChen021 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have reviewed the code for correctness, edge cases, concurrency, and integration risks; no issues found.

Reviewed 7 of 7 changed files.


This is an automated review by Codex GPT-5.5

@jtuglu1 jtuglu1 requested review from clintropolis, gianm and maytasm July 8, 2026 18:01

@maytasm maytasm left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jtuglu1 jtuglu1 merged commit 31019ad into apache:master Jul 8, 2026
69 of 72 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants