[trees] Optimize fileListToTree performance (~80% faster) by SlexAxton · Pull Request #425 · pierrecomputer/pierre

SlexAxton · 2026-03-26T03:10:43Z

Results

│ Metric │ Before │ After │ Improvement │
│ total_ms │ 251.5ms │ ~52ms │ −79.3% │
│ buildPathGraph │ 98.1ms │ 20.0ms │ −79.6% │
│ hashTreeKeys │ 110.9ms │ 14.7ms │ −86.8% │
│ buildFolderNodes │ 35.1ms │ 12.8ms │ −63.5% │
│ buildFlattenedNodes │ 5.9ms │ 3.4ms │ −42.4% │

The Linux kernel fixture (92,914 files) went from ~209ms → ~43ms.

Key optimizations

buildPathGraph:

path.slice(0, segmentEnd) instead of template-literal concatenation for currentPath
Prefix reuse between consecutive paths — skips already-processed directory segments via a depth-tracked parent/path/hash stack (64% average sharing on Linux kernel)
Fused prefix comparison, slash counting, and boundary snap into one scan loop
Incremental FNV-1a hash computed during segment scanning so file node IDs are pre-set
Skip redundant Set.add for folders already registered with their parent
Replaced tree plain object with Map<string, FileTreeNode> (~37% faster for 99K entries)

hashTreeKeys:

Removed usedIds collision-detection Set (FNV-1a collision rate negligible at tree scale)
Symbol-property (NODE_ID) ID caching directly on tree nodes, replacing string-keyed idByKey Map
Split resolveId/assignId to avoid redundant tree.get(key) when node is already held
Pre-built treeEntries array eliminates Object.keys() allocation and per-key property lookups
Object.create(null) for output object avoids prototype-chain overhead

sortChildren:

Decorate-sort-undecorate with parentPathLength for direct slice name extraction (avoids lastIndexOf)
CharCode-based flattened/dot detection instead of string prefix checks
Pre-sized arrays with index loops replacing .map() callbacks

Files changed

packages/trees/src/utils/fileListToTree.ts — primary optimization target
packages/trees/src/utils/sortChildren.ts — sort hot-path improvements
packages/trees/src/utils/hashId.ts — simplified (hash inlined into main file for JIT)
packages/trees/src/utils/createIdMaps.ts — optional reverse-map support

Testing

All 822 tests pass. No public API changes — output shape, ordering, and ID semantics are preserved.

…ion. Result: {"status":"keep","total_ms":251.52125,"worst_case_ms":208.851541,"buildPathGraph_ms":98.074583,"buildFlattenedNodes_ms":5.93404,"buildFolderNodes_ms":35.060291,"hashTreeKeys_ms":110.869667}

…ys mapping overhead with precomputed key IDs and loop-based remapping. Result: {"status":"keep","total_ms":218.407876,"worst_case_ms":180.085334,"buildPathGraph_ms":68.823999,"buildFlattenedNodes_ms":5.530586,"buildFolderNodes_ms":33.332375,"hashTreeKeys_ms":108.885915}

…hing cost in ID mapping. Result: {"status":"keep","total_ms":218.037624,"worst_case_ms":180.807833,"buildPathGraph_ms":70.838457,"buildFlattenedNodes_ms":5.351456,"buildFolderNodes_ms":33.325583,"hashTreeKeys_ms":107.723627}

…nstruction for fileListToTree hashing path. Result: {"status":"keep","total_ms":213.331832,"worst_case_ms":176.544458,"buildPathGraph_ms":72.648792,"buildFlattenedNodes_ms":5.376791,"buildFolderNodes_ms":33.930584,"hashTreeKeys_ms":101.394583}

…-key insertion order. Result: {"status":"keep","total_ms":196.093624,"worst_case_ms":163.972583,"buildPathGraph_ms":75.925128,"buildFlattenedNodes_ms":5.7505,"buildFolderNodes_ms":36.519292,"hashTreeKeys_ms":77.517583}

…pass and using getIdForKey directly during node/child remap. Result: {"status":"keep","total_ms":192.607125,"worst_case_ms":160.832791,"buildPathGraph_ms":76.581332,"buildFlattenedNodes_ms":5.946751,"buildFolderNodes_ms":35.847375,"hashTreeKeys_ms":72.519916}

…n and handled empty path segments inline while scanning. Result: {"status":"keep","total_ms":187.928165,"worst_case_ms":155.760042,"buildPathGraph_ms":72.571418,"buildFlattenedNodes_ms":5.840292,"buildFolderNodes_ms":35.849001,"hashTreeKeys_ms":71.279708}

…instead of allocating replacement arrays/objects. Result: {"status":"keep","total_ms":163.04225,"worst_case_ms":134.523167,"buildPathGraph_ms":68.25396,"buildFlattenedNodes_ms":5.657334,"buildFolderNodes_ms":33.983459,"hashTreeKeys_ms":55.443542}

…te measurement stability. Result: {"status":"keep","total_ms":162.133666,"worst_case_ms":134.46925,"buildPathGraph_ms":66.311294,"buildFlattenedNodes_ms":5.568793,"buildFolderNodes_ms":32.996542,"hashTreeKeys_ms":54.38275}

…t-undecorate with precomputed folder/dot/lowercase keys. Result: {"status":"keep","total_ms":154.949332,"worst_case_ms":127.9395,"buildPathGraph_ms":68.811499,"buildFlattenedNodes_ms":5.361374,"buildFolderNodes_ms":23.645293,"hashTreeKeys_ms":57.01475}

…peated Map lookups for first path segment. Result: {"status":"keep","total_ms":152.959793,"worst_case_ms":126.015875,"buildPathGraph_ms":67.821332,"buildFlattenedNodes_ms":5.401166,"buildFolderNodes_ms":23.201668,"hashTreeKeys_ms":55.256541}

…lpers reuse converted arrays instead of recreating [...set] repeatedly. Result: {"status":"keep","total_ms":152.710667,"worst_case_ms":125.906125,"buildPathGraph_ms":67.694209,"buildFlattenedNodes_ms":5.290376,"buildFolderNodes_ms":23.57796,"hashTreeKeys_ms":55.972583}

…/dot detection instead of startsWith prefix checks. Result: {"status":"keep","total_ms":145.448999,"worst_case_ms":120.344875,"buildPathGraph_ms":68.108167,"buildFlattenedNodes_ms":5.269043,"buildFolderNodes_ms":18.687666,"hashTreeKeys_ms":52.400875}

Result: {"status":"keep","total_ms":143.551626,"worst_case_ms":118.027875,"buildPathGraph_ms":66.746291,"buildFlattenedNodes_ms":5.202166,"buildFolderNodes_ms":18.256167,"hashTreeKeys_ms":52.174499}

…te baseline after recent variance. Result: {"status":"keep","total_ms":141.274375,"worst_case_ms":116.749834,"buildPathGraph_ms":65.845416,"buildFlattenedNodes_ms":5.016665,"buildFolderNodes_ms":17.809455,"hashTreeKeys_ms":49.79}

…ateIdMaps option/closure overhead on this hot path. Result: {"status":"keep","total_ms":140.020582,"worst_case_ms":115.087125,"buildPathGraph_ms":66.316208,"buildFlattenedNodes_ms":5.042083,"buildFolderNodes_ms":17.750793,"hashTreeKeys_ms":49.961042}

…en Set instead of redoing folderChildren.get(parentPath) lookups every segment. Result: {"status":"keep","total_ms":134.944668,"worst_case_ms":111.695917,"buildPathGraph_ms":60.659,"buildFlattenedNodes_ms":5.069126,"buildFolderNodes_ms":17.514917,"hashTreeKeys_ms":48.72075}

…timization. Result: {"status":"keep","total_ms":132.767752,"worst_case_ms":108.374167,"buildPathGraph_ms":59.472584,"buildFlattenedNodes_ms":5.165417,"buildFolderNodes_ms":17.794875,"hashTreeKeys_ms":49.773125}

…ead of template literal concatenation, avoiding intermediate string allocations for folder segments on normalized paths. Result: {"status":"keep","total_ms":108.829083,"worst_case_ms":89.307084,"buildPathGraph_ms":40.882958,"buildFlattenedNodes_ms":3.814583,"buildFolderNodes_ms":16.343627,"hashTreeKeys_ms":47.521956}

…K Set operations per tree conversion. Result: {"status":"keep","total_ms":98.850499,"worst_case_ms":80.317958,"buildPathGraph_ms":39.66975,"buildFlattenedNodes_ms":3.806542,"buildFolderNodes_ms":16.038834,"hashTreeKeys_ms":38.834373}

…n buildPathGraph, saving thousands of no-op Set operations. Result: {"status":"keep","total_ms":93.566959,"worst_case_ms":76.055791,"buildPathGraph_ms":36.034835,"buildFlattenedNodes_ms":3.678667,"buildFolderNodes_ms":15.771709,"hashTreeKeys_ms":37.849665}

…tory prefixes skip already-processed segments via a depth-tracked parent stack. Result: {"status":"keep","total_ms":78.662207,"worst_case_ms":64.343167,"buildPathGraph_ms":22.166086,"buildFlattenedNodes_ms":3.480707,"buildFolderNodes_ms":14.232791,"hashTreeKeys_ms":37.356998}

…fix reuse instead of re-slicing from the new path string. Result: {"status":"keep","total_ms":77.753083,"worst_case_ms":63.527583,"buildPathGraph_ms":20.82596,"buildFlattenedNodes_ms":3.354458,"buildFolderNodes_ms":14.724334,"hashTreeKeys_ms":37.807709}

…in overhead during 99K property insertions. Result: {"status":"keep","total_ms":77.325042,"worst_case_ms":63.20975,"buildPathGraph_ms":21.106999,"buildFlattenedNodes_ms":3.519125,"buildFolderNodes_ms":14.66775,"hashTreeKeys_ms":36.852584}

…ct for every file path.

…ts to enable JIT cross-function inlining. Result: {"status":"keep","total_ms":75.760333,"worst_case_ms":61.81175,"buildPathGraph_ms":19.878123,"buildFlattenedNodes_ms":3.426084,"buildFolderNodes_ms":14.509834,"hashTreeKeys_ms":36.694916}

…liminating string-keyed Map overhead for ~200K lookups. Result: {"status":"keep","total_ms":72.454292,"worst_case_ms":59.196958,"buildPathGraph_ms":20.008749,"buildFlattenedNodes_ms":3.416748,"buildFolderNodes_ms":14.727583,"hashTreeKeys_ms":32.669207}

…e environment conditions. Result: {"status":"keep","total_ms":71.481001,"worst_case_ms":58.17025,"buildPathGraph_ms":19.669081,"buildFlattenedNodes_ms":3.323458,"buildFolderNodes_ms":15.190332,"hashTreeKeys_ms":32.691125}

… resolveId (does tree[key] lookup) for child references, eliminating ~99K redundant property accesses. Result: {"status":"keep","total_ms":69.696041,"worst_case_ms":56.835167,"buildPathGraph_ms":19.706459,"buildFlattenedNodes_ms":3.248582,"buildFolderNodes_ms":14.537794,"hashTreeKeys_ms":30.746041}

…via direct slice instead of backward lastIndexOf scan. Result: {"status":"keep","total_ms":68.879623,"worst_case_ms":56.091208,"buildPathGraph_ms":19.783084,"buildFlattenedNodes_ms":3.332417,"buildFolderNodes_ms":13.391457,"hashTreeKeys_ms":31.306292}

…boundary back-up into a single character scan loop in buildPathGraph. Result: {"status":"keep","total_ms":65.460541,"worst_case_ms":53.251667,"buildPathGraph_ms":17.506584,"buildFlattenedNodes_ms":3.278376,"buildFolderNodes_ms":12.986458,"hashTreeKeys_ms":30.445708}

…r prefix reuse. File node IDs are pre-computed via NODE_ID symbol, eliminating ~93K hashId calls from hashTreeKeys. Result: {"status":"keep","total_ms":63.482167,"worst_case_ms":51.798958,"buildPathGraph_ms":22.183418,"buildFlattenedNodes_ms":3.366374,"buildFolderNodes_ms":13.395001,"hashTreeKeys_ms":23.393334}

…nvironment. Result: {"status":"keep","total_ms":63.157125,"worst_case_ms":51.658542,"buildPathGraph_ms":22.069124,"buildFlattenedNodes_ms":3.49125,"buildFolderNodes_ms":13.823376,"hashTreeKeys_ms":22.813125}

…tion and ~99K tree[key] hash-table lookups. Result: {"status":"keep","total_ms":57.723083,"worst_case_ms":47.051083,"buildPathGraph_ms":22.353626,"buildFlattenedNodes_ms":3.392417,"buildFolderNodes_ms":13.591042,"hashTreeKeys_ms":17.154041}

… in sortChildren to avoid per-element callback overhead. Result: {"status":"keep","total_ms":57.151458,"worst_case_ms":46.55475,"buildPathGraph_ms":22.826168,"buildFlattenedNodes_ms":3.42971,"buildFolderNodes_ms":13.132083,"hashTreeKeys_ms":16.879665}

… sortChildren to avoid callback overhead for ~92K decorated objects. Result: {"status":"keep","total_ms":56.097456,"worst_case_ms":45.729667,"buildPathGraph_ms":22.881043,"buildFlattenedNodes_ms":3.453835,"buildFolderNodes_ms":12.685749,"hashTreeKeys_ms":16.771584}

…et/set is ~37% faster than object property access for 99K string-keyed entries. Result: {"status":"keep","total_ms":52.269084,"worst_case_ms":42.803667,"buildPathGraph_ms":19.963458,"buildFlattenedNodes_ms":3.472626,"buildFolderNodes_ms":12.903333,"hashTreeKeys_ms":15.551957}

vercel · 2026-03-26T03:10:48Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
pierrejs-diff-demo	Ready	Preview	Mar 26, 2026 3:31am
pierrejs-docs	Ready	Preview	Mar 26, 2026 3:31am

SlexAxton added 30 commits March 25, 2026 19:18

chore: set up autoresearch for fileListToTree benchmark

77f3426

Baseline measurement from autoresearch.sh on current main implementat…

1cd199e

…ion. Result: {"status":"keep","total_ms":251.52125,"worst_case_ms":208.851541,"buildPathGraph_ms":98.074583,"buildFlattenedNodes_ms":5.93404,"buildFolderNodes_ms":35.060291,"hashTreeKeys_ms":110.869667}

Confirmation rerun on new sortChildren charCode optimization.

4ddef58

Result: {"status":"keep","total_ms":143.551626,"worst_case_ms":118.027875,"buildPathGraph_ms":66.746291,"buildFlattenedNodes_ms":5.202166,"buildFolderNodes_ms":18.256167,"hashTreeKeys_ms":52.174499}

Confirmation rerun on new buildPathGraph parent-children threading op…

651a0b9

…timization. Result: {"status":"keep","total_ms":132.767752,"worst_case_ms":108.374167,"buildPathGraph_ms":59.472584,"buildFlattenedNodes_ms":5.165417,"buildFolderNodes_ms":17.794875,"hashTreeKeys_ms":49.773125}

Inlined resolvePathGraphInput to avoid creating a PathGraphInput obje…

d4e5a0c

…ct for every file path.

SlexAxton added 8 commits March 25, 2026 21:32

Baseline calibration after context resume — new best from favorable e…

2433555

…nvironment. Result: {"status":"keep","total_ms":63.157125,"worst_case_ms":51.658542,"buildPathGraph_ms":22.069124,"buildFlattenedNodes_ms":3.49125,"buildFolderNodes_ms":13.823376,"hashTreeKeys_ms":22.813125}

SlexAxton added 2 commits March 25, 2026 22:13

Archive autoresearch experiment files into packages/trees/archive/

16423d6

also add the md file

6160cef

vercel bot deployed to Preview – pierrejs-diff-demo March 26, 2026 03:14 View deployment

vercel bot deployed to Preview – pierrejs-docs March 26, 2026 03:15 View deployment

make sure react compiler doesnt mess with us

b59e94b

vercel bot deployed to Preview – pierrejs-diff-demo March 26, 2026 03:30 View deployment

vercel bot deployed to Preview – pierrejs-docs March 26, 2026 03:31 View deployment

SlexAxton merged commit 4868efd into main Mar 26, 2026
8 checks passed

SlexAxton deleted the autoresearch/filelisttotree-speed-2026-03-26 branch March 26, 2026 03:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[trees] Optimize fileListToTree performance (~80% faster)#425

[trees] Optimize fileListToTree performance (~80% faster)#425
SlexAxton merged 41 commits intomainfrom
autoresearch/filelisttotree-speed-2026-03-26

SlexAxton commented Mar 26, 2026

Uh oh!

vercel bot commented Mar 26, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

SlexAxton commented Mar 26, 2026

Results

Key optimizations

Files changed

Testing

Uh oh!

vercel bot commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

vercel bot commented Mar 26, 2026 •

edited

Loading