Skip to content

fix(zero-cache): correctly return sqlite iterators in table-source#5696

Merged
tantaman merged 3 commits intomainfrom
mlaw/return
Apr 6, 2026
Merged

fix(zero-cache): correctly return sqlite iterators in table-source#5696
tantaman merged 3 commits intomainfrom
mlaw/return

Conversation

@tantaman
Copy link
Copy Markdown
Contributor

@tantaman tantaman commented Mar 26, 2026

#mapFromSQLiteTypes used to return the iterator via its own finally block, but it is possible that wrappers over mapFromSQLiteTypes return before we ever call into mapFromSQLiteTypes!

When that happens, we leak the SQLite iterator!

The work on cap found this. Added a test to reproduce the issue.

@vercel
Copy link
Copy Markdown

vercel bot commented Mar 26, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
replicache-docs Ready Ready Preview, Comment Apr 6, 2026 2:45pm
zbugs Ready Ready Preview, Comment Apr 6, 2026 2:45pm

Request Review

@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 26, 2026

🐰 Bencher Report

Branchmlaw/return
TestbedLinux
Click to view all benchmark results
BenchmarkFile SizeBenchmark Result
kilobytes (KB)
(Result Δ%)
Upper Boundary
kilobytes (KB)
(Limit %)
zero-package.tgz📈 view plot
🚷 view threshold
1,947.00 KB
(+0.00%)Baseline: 1,946.95 KB
1,985.89 KB
(98.04%)
zero.js📈 view plot
🚷 view threshold
269.32 KB
(0.00%)Baseline: 269.32 KB
274.71 KB
(98.04%)
zero.js.br📈 view plot
🚷 view threshold
71.30 KB
(0.00%)Baseline: 71.30 KB
72.73 KB
(98.04%)
🐰 View full continuous benchmarking report in Bencher

@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 26, 2026

🐰 Bencher Report

Branchmlaw/return
Testbedself-hosted-metal
Click to view all benchmark results
BenchmarkThroughputBenchmark Result
operations / second (ops/s)
(Result Δ%)
Lower Boundary
operations / second (ops/s)
(Limit %)
src/db/pg-copy.bench.ts > pg-copy benchmark > copy📈 view plot
🚷 view threshold
23.46 ops/s
(-1.16%)Baseline: 23.73 ops/s
20.17 ops/s
(86.00%)
🐰 View full continuous benchmarking report in Bencher

@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 26, 2026

🐰 Bencher Report

Branchmlaw/return
Testbedself-hosted-metal
Click to view all benchmark results
BenchmarkThroughputBenchmark Result
operations / second (ops/s) x 1e3
(Result Δ%)
Lower Boundary
operations / second (ops/s) x 1e3
(Limit %)
src/client/custom.bench.ts > big schema📈 view plot
🚷 view threshold
104.80 ops/s x 1e3
(-4.76%)Baseline: 110.04 ops/s x 1e3
93.53 ops/s x 1e3
(89.25%)
src/client/zero.bench.ts > basics > All 1000 rows x 10 columns (numbers)📈 view plot
🚷 view threshold
1.36 ops/s x 1e3
(+6.94%)Baseline: 1.27 ops/s x 1e3
1.08 ops/s x 1e3
(79.49%)
src/client/zero.bench.ts > pk compare > pk = N📈 view plot
🚷 view threshold
37.77 ops/s x 1e3
(+3.22%)Baseline: 36.59 ops/s x 1e3
31.10 ops/s x 1e3
(82.35%)
src/client/zero.bench.ts > with filter > Lower rows 500 x 10 columns (numbers)📈 view plot
🚷 view threshold
1.95 ops/s x 1e3
(-1.46%)Baseline: 1.98 ops/s x 1e3
1.68 ops/s x 1e3
(86.26%)
🐰 View full continuous benchmarking report in Bencher

@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 26, 2026

🐰 Bencher Report

Branchmlaw/return
Testbedself-hosted-metal
Click to view all benchmark results
BenchmarkThroughputoperations / second (ops/s)
src/chinook-hydration.bench.ts > all playlists > zpg: all playlists📈 view plot
🚷 view threshold
4.56 ops/s
src/chinook-hydration.bench.ts > all playlists > zql: all playlists📈 view plot
🚷 view threshold
5.41 ops/s
src/chinook-hydration.bench.ts > all playlists > zqlite: all playlists📈 view plot
🚷 view threshold
1.35 ops/s
src/chinook-manual-cases.bench.ts > tracks with artist name > flipped📈 view plot
🚷 view threshold
4,326.90 ops/s
src/chinook-manual-cases.bench.ts > tracks with artist name > not flipped📈 view plot
🚷 view threshold
30.72 ops/s
src/chinook-push.bench.ts > edit for limited query, inside the bound > zql: edit for limited query, inside the bound📈 view plot
🚷 view threshold
136,814.86 ops/s
src/chinook-push.bench.ts > edit for limited query, inside the bound > zqlite: edit for limited query, inside the bound📈 view plot
🚷 view threshold
50,139.91 ops/s
src/chinook-push.bench.ts > edit for limited query, outside the bound > zql: edit for limited query, outside the bound📈 view plot
🚷 view threshold
142,532.02 ops/s
src/chinook-push.bench.ts > edit for limited query, outside the bound > zqlite: edit for limited query, outside the bound📈 view plot
🚷 view threshold
50,085.15 ops/s
src/chinook-push.bench.ts > push into limited query, inside the bound > zql: push into limited query, inside the bound📈 view plot
🚷 view threshold
76,541.55 ops/s
src/chinook-push.bench.ts > push into limited query, inside the bound > zqlite: push into limited query, inside the bound📈 view plot
🚷 view threshold
2,909.12 ops/s
src/chinook-push.bench.ts > push into limited query, outside the bound > zql: push into limited query, outside the bound📈 view plot
🚷 view threshold
265,033.02 ops/s
src/chinook-push.bench.ts > push into limited query, outside the bound > zqlite: push into limited query, outside the bound📈 view plot
🚷 view threshold
55,226.28 ops/s
src/chinook-push.bench.ts > push into unlimited query > zql: push into unlimited query📈 view plot
🚷 view threshold
219,909.55 ops/s
src/chinook-push.bench.ts > push into unlimited query > zqlite: push into unlimited query📈 view plot
🚷 view threshold
80,294.88 ops/s
src/planner-cost.bench.ts > planner cost > 1 exists: track.exists(album)📈 view plot
🚷 view threshold
8,346.38 ops/s
src/planner-cost.bench.ts > planner cost > 10 exists (AND)📈 view plot
🚷 view threshold
156,227.66 ops/s
src/planner-cost.bench.ts > planner cost > 10 exists (OR)📈 view plot
🚷 view threshold
2,771.43 ops/s
src/planner-cost.bench.ts > planner cost > 12 exists (AND)📈 view plot
🚷 view threshold
138,790.07 ops/s
src/planner-cost.bench.ts > planner cost > 12 exists (OR)📈 view plot
🚷 view threshold
2,358.79 ops/s
src/planner-cost.bench.ts > planner cost > 12 level nesting📈 view plot
🚷 view threshold
2,104.85 ops/s
src/planner-cost.bench.ts > planner cost > 2 exists (AND): track.exists(album).exists(genre)📈 view plot
🚷 view threshold
3,585.13 ops/s
src/planner-cost.bench.ts > planner cost > 3 exists (AND)📈 view plot
🚷 view threshold
1,390.89 ops/s
src/planner-cost.bench.ts > planner cost > 3 exists (OR)📈 view plot
🚷 view threshold
711.47 ops/s
src/planner-cost.bench.ts > planner cost > 5 exists (AND)📈 view plot
🚷 view threshold
221.58 ops/s
src/planner-cost.bench.ts > planner cost > 5 exists (OR)📈 view plot
🚷 view threshold
115.50 ops/s
src/planner-cost.bench.ts > planner cost > Nested 2 levels: track > album > artist📈 view plot
🚷 view threshold
3,121.80 ops/s
src/planner-cost.bench.ts > planner cost > Nested 4 levels: playlist > tracks > album > artist📈 view plot
🚷 view threshold
513.71 ops/s
src/planner-cost.bench.ts > planner cost > Nested with filters: track > album > artist (filtered)📈 view plot
🚷 view threshold
2,554.62 ops/s
src/planner-hydration.bench.ts > playlist.exists(tracks) > planned: playlist.exists(tracks)📈 view plot
🚷 view threshold
454.23 ops/s
src/planner-hydration.bench.ts > playlist.exists(tracks) > unplanned: playlist.exists(tracks)📈 view plot
🚷 view threshold
445.47 ops/s
src/planner-hydration.bench.ts > track.exists(album) OR exists(genre) > planned: track.exists(album) OR exists(genre)📈 view plot
🚷 view threshold
115.25 ops/s
src/planner-hydration.bench.ts > track.exists(album) OR exists(genre) > unplanned: track.exists(album) OR exists(genre)📈 view plot
🚷 view threshold
32.20 ops/s
src/planner-hydration.bench.ts > track.exists(album) where title="Big Ones" > planned: track.exists(album) where title="Big Ones"📈 view plot
🚷 view threshold
5,902.80 ops/s
src/planner-hydration.bench.ts > track.exists(album) where title="Big Ones" > unplanned: track.exists(album) where title="Big Ones"📈 view plot
🚷 view threshold
35.91 ops/s
src/planner-hydration.bench.ts > track.exists(album).exists(genre) > planned: track.exists(album).exists(genre)📈 view plot
🚷 view threshold
27.72 ops/s
src/planner-hydration.bench.ts > track.exists(album).exists(genre) > unplanned: track.exists(album).exists(genre)📈 view plot
🚷 view threshold
27.81 ops/s
src/planner-hydration.bench.ts > track.exists(album).exists(genre) with filters > planned: track.exists(album).exists(genre) with filters📈 view plot
🚷 view threshold
4,260.70 ops/s
src/planner-hydration.bench.ts > track.exists(album).exists(genre) with filters > unplanned: track.exists(album).exists(genre) with filters📈 view plot
🚷 view threshold
38.63 ops/s
src/planner-hydration.bench.ts > track.exists(playlists) > planned: track.exists(playlists)📈 view plot
🚷 view threshold
2.92 ops/s
src/planner-hydration.bench.ts > track.exists(playlists) > unplanned: track.exists(playlists)📈 view plot
🚷 view threshold
2.92 ops/s
🐰 View full continuous benchmarking report in Bencher

@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 26, 2026

🐰 Bencher Report

Branchmlaw/return
Testbedself-hosted-metal

🚨 3 Alerts

BenchmarkMeasure
Units
ViewBenchmark Result
(Result Δ%)
Lower Boundary
(Limit %)
src/btree-set.bench.ts > BTreeSet iterators > [Symbol.iterator]() full scanThroughput
operations / second (ops/s) x 1e3
📈 plot
🚷 threshold
🚨 alert (🔔)
129.04 ops/s x 1e3
(-16.37%)Baseline: 154.30 ops/s x 1e3
131.15 ops/s x 1e3
(101.64%)

src/tdigest.bench.ts > TDigest Benchmarks > merge > addCentroidThroughput
operations / second (ops/s) x 1e3
📈 plot
🚷 threshold
🚨 alert (🔔)
15.43 ops/s x 1e3
(-48.68%)Baseline: 30.07 ops/s x 1e3
25.56 ops/s x 1e3
(165.61%)

src/tdigest.bench.ts > TDigest Benchmarks > merge > mergeThroughput
operations / second (ops/s) x 1e3
📈 plot
🚷 threshold
🚨 alert (🔔)
16.86 ops/s x 1e3
(-45.33%)Baseline: 30.84 ops/s x 1e3
26.21 ops/s x 1e3
(155.49%)

Click to view all benchmark results
BenchmarkThroughputBenchmark Result
operations / second (ops/s)
(Result Δ%)
Lower Boundary
operations / second (ops/s)
(Limit %)
src/btree-set.bench.ts > BTreeSet iterator next() in isolation > forward iterator next()📈 view plot
🚷 view threshold
132,284.92 ops/s
(-5.99%)Baseline: 140,716.28 ops/s
119,608.84 ops/s
(90.42%)
src/btree-set.bench.ts > BTreeSet iterator next() in isolation > forward iterator next() from mid📈 view plot
🚷 view threshold
290,995.60 ops/s
(+4.17%)Baseline: 279,341.37 ops/s
237,440.16 ops/s
(81.60%)
src/btree-set.bench.ts > BTreeSet iterator next() in isolation > reverse iterator next()📈 view plot
🚷 view threshold
146,541.01 ops/s
(+1.85%)Baseline: 143,884.99 ops/s
122,302.25 ops/s
(83.46%)
src/btree-set.bench.ts > BTreeSet iterator next() in isolation > reverse iterator next() from mid📈 view plot
🚷 view threshold
288,597.19 ops/s
(+4.44%)Baseline: 276,317.36 ops/s
234,869.75 ops/s
(81.38%)
src/btree-set.bench.ts > BTreeSet iterators > [Symbol.iterator]() full scan📈 view plot
🚷 view threshold
🚨 view alert (🔔)
129,042.88 ops/s
(-16.37%)Baseline: 154,297.66 ops/s
131,153.01 ops/s
(101.64%)

src/btree-set.bench.ts > BTreeSet iterators > values() full scan📈 view plot
🚷 view threshold
186,318.85 ops/s
(-0.42%)Baseline: 187,102.18 ops/s
159,036.86 ops/s
(85.36%)
src/btree-set.bench.ts > BTreeSet iterators > valuesFrom() from mid📈 view plot
🚷 view threshold
314,570.74 ops/s
(-6.80%)Baseline: 337,518.89 ops/s
286,891.05 ops/s
(91.20%)
src/btree-set.bench.ts > BTreeSet iterators > valuesFromReversed() from mid📈 view plot
🚷 view threshold
328,892.44 ops/s
(+8.62%)Baseline: 302,785.79 ops/s
257,367.92 ops/s
(78.25%)
src/btree-set.bench.ts > BTreeSet iterators > valuesReversed() full scan📈 view plot
🚷 view threshold
167,300.57 ops/s
(-2.78%)Baseline: 172,084.93 ops/s
146,272.19 ops/s
(87.43%)
src/size-of-value.bench.ts > getSizeOfValue performance > arrays > large array (100 items)📈 view plot
🚷 view threshold
1,535,382.78 ops/s
(+2.36%)Baseline: 1,499,943.29 ops/s
1,274,951.79 ops/s
(83.04%)
src/size-of-value.bench.ts > getSizeOfValue performance > arrays > small array (10 items)📈 view plot
🚷 view threshold
12,757,825.19 ops/s
(+2.87%)Baseline: 12,401,334.14 ops/s
10,541,134.02 ops/s
(82.62%)
src/size-of-value.bench.ts > getSizeOfValue performance > datasets > large dataset (100x512B)📈 view plot
🚷 view threshold
39,522.58 ops/s
(+4.19%)Baseline: 37,934.31 ops/s
32,244.17 ops/s
(81.58%)
src/size-of-value.bench.ts > getSizeOfValue performance > datasets > small dataset (10x256B)📈 view plot
🚷 view threshold
387,605.59 ops/s
(+0.76%)Baseline: 384,672.52 ops/s
326,971.64 ops/s
(84.36%)
src/size-of-value.bench.ts > getSizeOfValue performance > objects > nested object📈 view plot
🚷 view threshold
3,383,177.90 ops/s
(+1.96%)Baseline: 3,318,140.43 ops/s
2,820,419.37 ops/s
(83.37%)
src/size-of-value.bench.ts > getSizeOfValue performance > objects > structured object (1KB)📈 view plot
🚷 view threshold
3,946,946.97 ops/s
(+3.46%)Baseline: 3,815,082.70 ops/s
3,242,820.29 ops/s
(82.16%)
src/size-of-value.bench.ts > getSizeOfValue performance > objects > structured object (256B)📈 view plot
🚷 view threshold
3,897,153.37 ops/s
(+2.08%)Baseline: 3,817,831.50 ops/s
3,245,156.78 ops/s
(83.27%)
src/size-of-value.bench.ts > getSizeOfValue performance > primitives > boolean📈 view plot
🚷 view threshold
73,409,449.25 ops/s
(+5.43%)Baseline: 69,631,617.01 ops/s
59,186,874.46 ops/s
(80.63%)
src/size-of-value.bench.ts > getSizeOfValue performance > primitives > integer📈 view plot
🚷 view threshold
73,635,196.46 ops/s
(+6.18%)Baseline: 69,349,998.28 ops/s
58,947,498.54 ops/s
(80.05%)
src/size-of-value.bench.ts > getSizeOfValue performance > primitives > null📈 view plot
🚷 view threshold
73,480,381.72 ops/s
(+5.70%)Baseline: 69,521,118.80 ops/s
59,092,950.98 ops/s
(80.42%)
src/size-of-value.bench.ts > getSizeOfValue performance > primitives > string (100 chars)📈 view plot
🚷 view threshold
791,503.26 ops/s
(+1.75%)Baseline: 777,880.14 ops/s
661,198.12 ops/s
(83.54%)
src/tdigest.bench.ts > TDigest Benchmarks > add📈 view plot
🚷 view threshold
2.39 ops/s
(-0.27%)Baseline: 2.39 ops/s
2.03 ops/s
(85.23%)
src/tdigest.bench.ts > TDigest Benchmarks > addCentroid📈 view plot
🚷 view threshold
1.89 ops/s
(+1.05%)Baseline: 1.87 ops/s
1.59 ops/s
(84.11%)
src/tdigest.bench.ts > TDigest Benchmarks > addCentroidList📈 view plot
🚷 view threshold
1.90 ops/s
(-0.65%)Baseline: 1.91 ops/s
1.62 ops/s
(85.56%)
src/tdigest.bench.ts > TDigest Benchmarks > merge > addCentroid📈 view plot
🚷 view threshold
🚨 view alert (🔔)
15,433.11 ops/s
(-48.68%)Baseline: 30,069.56 ops/s
25,559.13 ops/s
(165.61%)

src/tdigest.bench.ts > TDigest Benchmarks > merge > merge📈 view plot
🚷 view threshold
🚨 view alert (🔔)
16,858.79 ops/s
(-45.33%)Baseline: 30,839.54 ops/s
26,213.61 ops/s
(155.49%)

src/tdigest.bench.ts > TDigest Benchmarks > quantile📈 view plot
🚷 view threshold
2.34 ops/s
(+0.20%)Baseline: 2.34 ops/s
1.99 ops/s
(84.83%)
🐰 View full continuous benchmarking report in Bencher

@tantaman tantaman changed the title fix(zero-cache): move return of iter to where iter is created fix(zero-cache): correctly return sqlite iterators in table-source Mar 26, 2026
@tantaman tantaman requested review from arv and grgbkr March 26, 2026 16:45
@tantaman tantaman enabled auto-merge March 26, 2026 16:46
Copy link
Copy Markdown
Contributor

@arv arv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tantaman tantaman added this pull request to the merge queue Mar 26, 2026
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to a conflict with the base branch Mar 26, 2026
@tantaman tantaman enabled auto-merge April 6, 2026 14:43
@tantaman tantaman added this pull request to the merge queue Apr 6, 2026
Merged via the queue into main with commit 9d9153e Apr 6, 2026
22 of 26 checks passed
@tantaman tantaman deleted the mlaw/return branch April 6, 2026 15:04
arv pushed a commit that referenced this pull request Apr 17, 2026
…5696)

#mapFromSQLiteTypes used to return the iterator via its own finally
block, but it is possible that wrappers over `mapFromSQLiteTypes` return
before we ever call into `mapFromSQLiteTypes`!

When that happens, we leak the SQLite iterator!

The work on `cap` found this. Added a test to reproduce the issue.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants