Skip to content

Optimized the overall performance of IoTDB#17664

Open
Caideyipi wants to merge 4 commits into
masterfrom
performance
Open

Optimized the overall performance of IoTDB#17664
Caideyipi wants to merge 4 commits into
masterfrom
performance

Conversation

@Caideyipi
Copy link
Copy Markdown
Collaborator

@Caideyipi Caideyipi commented May 14, 2026

Description

  1. Removed the unnecessary copies in QueryDataSetUtils & PartialPath.
  2. Cached the deletion ranges to speedup the mod checks
  3. Removed some unnecessary copies for partial paths
  4. Removed some boxings on hot paths
  5. Optimized some data structures (List::contains -> Set::contains, LinkedList -> ArrayList)

This PR has:

  • been self-reviewed.
    • concurrent read
    • concurrent write
    • concurrent read and write
  • added documentation for new or modified features or behaviors.
  • added Javadocs for most classes and all non-trivial methods.
  • added or updated version, license, or notice information
  • added comments explaining the "why" and the intent of the code wherever would not be obvious
    for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold
    for code coverage.
  • added integration tests.
  • been tested in a test IoTDB cluster.

Key changed/added classes (or packages if there are too many classes) in this PR

@Caideyipi Caideyipi changed the title Optimized the overall performance of Io Optimized the overall performance of IoTDB May 14, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented May 14, 2026

Codecov Report

❌ Patch coverage is 54.58515% with 104 lines in your changes missing coverage. Please review.
✅ Project coverage is 40.41%. Comparing base (a2c22ac) to head (828d3e8).
⚠️ Report is 8 commits behind head on master.

Files with missing lines Patch % Lines
...read/filescan/impl/UnclosedFileScanHandleImpl.java 0.00% 38 Missing ⚠️
...in/java/org/apache/iotdb/rpc/IoTDBJDBCDataSet.java 0.00% 19 Missing ⚠️
...rc/main/java/org/apache/iotdb/session/Session.java 85.91% 10 Missing ⚠️
...n/read/filescan/impl/ClosedFileScanHandleImpl.java 0.00% 9 Missing ⚠️
.../apache/iotdb/rpc/stmt/PreparedParameterSerde.java 12.50% 7 Missing ⚠️
...a/org/apache/iotdb/db/utils/QueryDataSetUtils.java 63.15% 7 Missing ⚠️
...org/apache/iotdb/db/schemaengine/SchemaEngine.java 0.00% 4 Missing ⚠️
...g/apache/iotdb/db/storageengine/StorageEngine.java 33.33% 4 Missing ⚠️
...metadata/fetcher/cache/TableDeviceSchemaCache.java 0.00% 3 Missing ⚠️
...ava/org/apache/iotdb/commons/path/PartialPath.java 84.61% 2 Missing ⚠️
... and 1 more
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #17664      +/-   ##
============================================
+ Coverage     40.37%   40.41%   +0.03%     
  Complexity     2574     2574              
============================================
  Files          5178     5179       +1     
  Lines        349089   349247     +158     
  Branches      44655    44676      +21     
============================================
+ Hits         140959   141143     +184     
+ Misses       208130   208104      -26     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment on lines +92 to +99
List<TimeRange> timeRangeList = deviceToDeletionRanges.get(deviceID);
if (timeRangeList == null) {
timeRangeList =
getMergedTimeRanges(queryContext.getPathModifications(curFileModEntries, deviceID));
deviceToDeletionRanges.put(deviceID, timeRangeList);
}
return ModificationUtils.isPointDeleted(timestamp, timeRangeList);
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ComputeIfAbsent?
Is the additional memory footprint risky?

Comment on lines 1089 to 1095
final java.util.Collection<Integer> targetDataRegionIds =
dataRegionIds.size() > 1 ? new java.util.HashSet<>(dataRegionIds) : dataRegionIds;
dataRegionMap.forEach(
(dataRegionId, dataRegion) -> {
if (dataRegionIds.contains(dataRegionId.getId())) {
if (targetDataRegionIds.contains(dataRegionId.getId())) {
dataRegionDisk.put(dataRegionId.getId(), dataRegion.countRegionDiskSize());
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for id : dataRegionIds
dataRegionMap.computeIfPresent?

@sonarqubecloud
Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants