Skip to content

[trees] improve tree initialization and rebuild performance #429

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
SlexAxton merged 2 commits into from
Mar 26, 2026
Merged

[trees] improve tree initialization and rebuild performance #429

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
de186e0
[trees] Add core tree primitives benchmark
SlexAxton Mar 26, 2026
a347b6a
[trees] Optimize large tree initialization (~66% faster)
SlexAxton Mar 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions packages/trees/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -206,6 +206,7 @@ From `packages/trees`:
```bash
bun test
bun run benchmark
bun run benchmark:core
bun run test:e2e
bun run tsc
bun run build
Expand Down Expand Up @@ -265,6 +266,41 @@ bun ws trees benchmark -- --case=linux --compare tmp/fileListToTree-baseline.jso
mismatches. That makes it useful both for performance regressions and for
catching accidental behavior changes while refactoring.

For core tree primitive profiling, use the dedicated benchmark runner:

```bash
bun ws trees benchmark:core
```

If you care most about large datasets, run a filtered large-shape subset:

```bash
bun ws trees benchmark:core -- --case=large-wide --case=large-monorepo --case=linux
```

This benchmark isolates core tree costs by preparing fixture-backed tree data up
front and timing only primitive calls. The `createTree` timing reflects the real
initialization path (`createTree` + `setMounted(true)` + initial `rebuildTree`).
`rebuildTree` can run either as unchanged hot rebuilds or as changed-state
rebuilds via `--rebuild-mode=expanded-copy`.

To better mirror the trees-dev virtualization workload, benchmark cases are
built with `sort: false` and `flattenEmptyDirectories: true`.

It also supports `--json`, `--compare`, and `--case` filters, plus:

- `--create-iterations` to batch multiple create+mount+initial-rebuild calls per
measured sample
- `--rebuild-iterations` to batch multiple `rebuildTree` calls per measured
sample
- `--rebuild-mode` to choose unchanged rebuilds or a changed-state mode
(`expanded-copy`) with stronger update-path signal
- `--feature-profile` to switch between `virtualized-card` realism,
`root-default`, and `minimal` core-only feature overhead

Those batching flags improve confidence for fast operations by reducing timer
jitter while still reporting per-call milliseconds.

# Credits and Acknolwedgements

The core of this library's underlying tree implementation started as a hard fork
Expand Down
2 changes: 2 additions & 0 deletions packages/trees/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@
"scripts": {
"build": "tsdown --clean",
"benchmark": "bun run ./scripts/benchmarkFileListToTree.ts",
"benchmark:file-list-to-tree": "bun run ./scripts/benchmarkFileListToTree.ts",
"benchmark:core": "bun run ./scripts/benchmarkTreeCorePrimitives.ts",
"dev": "echo 'Watching for changes…' && tsdown --watch --log-level error",
"test": "bun test",
"coverage": "bun test --coverage",
Expand Down
199 changes: 15 additions & 184 deletions packages/trees/scripts/benchmarkFileListToTree.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,16 +1,29 @@
import { readFileSync } from 'node:fs';
import { resolve } from 'node:path';

import type { FileTreeData } from '../src/types';
import {
benchmarkFileListToTreeStages,
type FileListToTreeStageName,
} from '../src/utils/fileListToTree';
import {
type BenchmarkEnvironment,
calculateDeltaPercent,
formatMs,
formatSignedMs,
formatSignedPercent,
getEnvironment,
parseNonNegativeInteger,
parsePositiveInteger,
printTable,
summarizeSamples,
type TimingSummary,
} from './lib/benchmarkUtils';
import {
type FileListToTreeBenchmarkCase,
filterBenchmarkCases,
getFileListToTreeBenchmarkCases,
} from './lib/fileListToTreeBenchmarkData';
import { checksumFileTreeData } from './lib/treeBenchmarkChecksums';

interface BenchmarkConfig {
runs: number;
Expand All @@ -20,22 +33,6 @@ interface BenchmarkConfig {
comparePath?: string;
}

interface BenchmarkEnvironment {
bunVersion: string;
platform: string;
arch: string;
}

interface TimingSummary {
runs: number;
meanMs: number;
medianMs: number;
p95Ms: number;
minMs: number;
maxMs: number;
stdDevMs: number;
}

interface CaseSummary extends TimingSummary {
name: string;
source: FileListToTreeBenchmarkCase['source'];
Expand Down Expand Up @@ -121,26 +118,6 @@ const STAGE_ORDER: FileListToTreeStageName[] = [
'hashTreeKeys',
];

function parsePositiveInteger(value: string, flagName: string): number {
const parsed = Number.parseInt(value, 10);
if (!Number.isFinite(parsed) || parsed <= 0) {
throw new Error(
`Invalid ${flagName} value '${value}'. Expected a positive integer.`
);
}
return parsed;
}

function parseNonNegativeInteger(value: string, flagName: string): number {
const parsed = Number.parseInt(value, 10);
if (!Number.isFinite(parsed) || parsed < 0) {
throw new Error(
`Invalid ${flagName} value '${value}'. Expected a non-negative integer.`
);
}
return parsed;
}

function printHelpAndExit(): never {
console.log('Usage: bun ws trees benchmark -- [options]');
console.log('');
Expand Down Expand Up @@ -209,136 +186,6 @@ function parseArgs(argv: string[]): BenchmarkConfig {
return config;
}

function percentile(sortedValues: number[], percentileRank: number): number {
if (sortedValues.length === 0) {
return 0;
}

const rank = (sortedValues.length - 1) * percentileRank;
const lowerIndex = Math.floor(rank);
const upperIndex = Math.ceil(rank);
const lower = sortedValues[lowerIndex] ?? sortedValues[0] ?? 0;
const upper =
sortedValues[upperIndex] ?? sortedValues[sortedValues.length - 1] ?? lower;
if (lowerIndex === upperIndex) {
return lower;
}

const interpolation = rank - lowerIndex;
return lower + (upper - lower) * interpolation;
}

function summarizeSamples(samples: number[]): TimingSummary {
if (samples.length === 0) {
return {
runs: 0,
meanMs: 0,
medianMs: 0,
p95Ms: 0,
minMs: 0,
maxMs: 0,
stdDevMs: 0,
};
}

const sortedSamples = [...samples].sort((left, right) => left - right);
const total = samples.reduce((sum, value) => sum + value, 0);
const mean = total / samples.length;
const variance =
samples.reduce((sum, value) => sum + (value - mean) ** 2, 0) /
samples.length;

return {
runs: samples.length,
meanMs: mean,
medianMs: percentile(sortedSamples, 0.5),
p95Ms: percentile(sortedSamples, 0.95),
minMs: sortedSamples[0] ?? 0,
maxMs: sortedSamples[sortedSamples.length - 1] ?? 0,
stdDevMs: Math.sqrt(variance),
};
}

function formatMs(value: number): string {
return value.toFixed(3);
}

function formatSignedMs(value: number): string {
const prefix = value > 0 ? '+' : '';
return `${prefix}${value.toFixed(3)}`;
}

function formatSignedPercent(value: number): string {
if (!Number.isFinite(value)) {
return value > 0 ? '+inf%' : value < 0 ? '-inf%' : '0.0%';
}

const prefix = value > 0 ? '+' : '';
return `${prefix}${value.toFixed(1)}%`;
}

function checksumTree(tree: FileTreeData): number {
let checksum = 0;

for (const [id, node] of Object.entries(tree)) {
checksum += id.length;
checksum += node.name.length;
checksum += node.path.length;

if (node.children != null) {
checksum += node.children.direct.length;
for (const child of node.children.direct) {
checksum += child.length;
}
if (node.children.flattened != null) {
checksum += node.children.flattened.length;
for (const child of node.children.flattened) {
checksum += child.length;
}
}
}

if (node.flattens != null) {
checksum += node.flattens.length;
for (const path of node.flattens) {
checksum += path.length;
}
}
}

return checksum;
}

function printTable(rows: Record<string, string>[], headers: string[]): void {
const widths = headers.map((header) => {
const valueWidth = rows.reduce(
(max, row) => Math.max(max, row[header]?.length ?? 0),
header.length
);
return valueWidth;
});

const formatRow = (row: Record<string, string>) =>
headers
.map((header, index) => (row[header] ?? '').padEnd(widths[index]))
.join(' ')
.trimEnd();

const headerRow = Object.fromEntries(
headers.map((header) => [header, header])
);
console.log(formatRow(headerRow));
console.log(
widths
.map((width) => '-'.repeat(width))
.join(' ')
.trimEnd()
);
for (const row of rows) {
console.log(formatRow(row));
}
}

function createStageSampleStorage(): Record<FileListToTreeStageName, number[]> {
return {
buildPathGraph: [],
Expand All @@ -348,22 +195,6 @@ function createStageSampleStorage(): Record<FileListToTreeStageName, number[]> {
};
}

function getEnvironment(): BenchmarkEnvironment {
return {
bunVersion: Bun.version,
platform: process.platform,
arch: process.arch,
};
}

function calculateDeltaPercent(current: number, baseline: number): number {
if (baseline === 0) {
return current === 0 ? 0 : Number.POSITIVE_INFINITY;
}

return ((current - baseline) / baseline) * 100;
}

// Benchmarks only stay comparable when the output payload has the same shape.
// Load and validate the previous JSON run up front so comparison failures are
// immediate instead of producing misleading deltas later on.
Expand Down Expand Up @@ -608,7 +439,7 @@ function main() {
const startTime = performance.now();
const result = benchmarkFileListToTreeStages(caseConfig.files);
const elapsedMs = performance.now() - startTime;
const resultChecksum = checksumTree(result.tree);
const resultChecksum = checksumFileTreeData(result.tree);
const existingChecksum = caseChecksums[caseIndex];

if (existingChecksum == null) {
Expand Down
Loading
Loading