Skip to content

[fix][PES][datasource] fix datasource oom issue in mdq service partition statistic#1040

Merged
casionone merged 1 commit into
dev-2.0.0-webankfrom
dev-2.0.0-fix-datasource-oom
Jun 3, 2026
Merged

[fix][PES][datasource] fix datasource oom issue in mdq service partition statistic#1040
casionone merged 1 commit into
dev-2.0.0-webankfrom
dev-2.0.0-fix-datasource-oom

Conversation

@v-kkhuang

@v-kkhuang v-kkhuang commented Jun 1, 2026

Copy link
Copy Markdown

What is the purpose of the change

Background/Problem:
The partition statistic method in MdqServiceImpl uses forEach with Lambda expressions, which can cause OOM (Out of Memory) issues when processing large partitions. The functional programming style creates multiple intermediate objects that increase memory pressure.

Purpose of Change:
To address this OOM problem, this PR refactors the getMdqTablePartitionStatisticInfoVO method by replacing forEach + Lambda expressions with traditional for-each loops, reducing object creation and improving memory efficiency.

Value/Impact:
After the change, the system can handle large partition lists without OOM errors, improving stability and reliability in production environments.

Related issues/PRs

Related issues: close #1039
Related pr:none

Brief change log

  • Replace forEach + Lambda expressions with traditional for-each loops in getMdqTablePartitionStatisticInfoVO
  • Add null pointer filter for subPartitions to improve robustness
  • Optimize memory usage by reducing intermediate object creation

Checklist

  • I have read the Contributing Guidelines on pull requests.
  • I have explained the need for this PR and the problem it solves
  • I have explained the changes or the new features added to this PR
  • I have added tests corresponding to this change
  • I have updated the documentation to reflect this change
  • I have verified that this change is backward compatible
  • If this is a code change: I have written unit tests to fully verify the new behavior.

@v-kkhuang v-kkhuang changed the title #AI commit# refactor:修复datasource oom异常 [fix][PES][datasource] fix datasource oom issue in mdq service partition statistic Jun 1, 2026

@casionone casionone left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@casionone casionone merged commit 65fddab into dev-2.0.0-webank Jun 3, 2026
18 of 22 checks passed
@v-kkhuang v-kkhuang deleted the dev-2.0.0-fix-datasource-oom branch June 3, 2026 08:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants