test(verify): consolidate TradingView trade-list fragments before pairing#54
Merged
Merged
Conversation
…ring TradingView's "List of Trades" (and the engine mirroring it) splits one entry FILL into multiple "Trade #" rows: a tiny qty_step rounding remainder sharing the SAME entry time+price, or FIFO partial-close lots of a grid bot. The greedy entry-time matcher in verify_corpus then cross-pairs same-entry lots, producing spurious count + exit-price-p90 deltas (the ~90% qty-p90 is the fingerprint) even though the engine's fills are trade-for-trade price-exact. Add consolidate_fragments(): group rows by an EXACT (entry_time, entry_price, direction) key and merge each group into one logical trade (sum qty/pnl, keep the shared entry, represent the exit by the shared price or qty-weighted final close). Applied symmetrically to the TV and engine lists before alignment (mirrored in regen_validation_report.py). Because the key is compared exactly, two rows merge only when they are the same fill event — a distinct trade lands on a different bar or price level and keeps its own key, so real divergences are never masked. Corpus byte-identical: verify_corpus.py --all stays excellent=251 / anomaly=1 / fail=0 with ZERO tier changes (4 corpus strategies that legitimately fragment consolidate symmetrically and stay excellent — the key is correct, not inert). Scraped targets: tomukasss weak->excellent (count 31%->0%, qty-p90 51%->0.2%); xlm/xau grids' fragment artifact cleared (qty-p90 ~90%->~0.5%, count->~0) while their REAL residual exit-p90 ~6% (TV holds lots days longer) correctly remains. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
TradingView's "List of Trades" (and the engine mirroring it) splits one entry fill into multiple "Trade #" rows — a tiny
qty_steprounding remainder sharing the same entry time+price, or FIFO partial-close lots of a grid bot.verify_corpus's greedy entry-time matcher then cross-pairs same-entry lots → spurious count + exit-p90 deltas (the ~90% qty-p90 is the fingerprint), even though the engine's fills are trade-for-trade price-exact.Fix
consolidate_fragments()groups rows by an exact(entry_time, entry_price, direction)key and merges each group into one logical trade (sum qty/pnl, keep the shared entry, represent the exit by the shared price or qty-weighted final close). Applied symmetrically to the TV and engine lists before alignment (mirrored inregen_validation_report.py).Non-masking guarantee: the key is compared exactly, so two rows merge only when they are the same fill event; a distinct trade lands on a different bar or price level and keeps its own key. Real divergences are never merged.
Validation
90%→0.5%); their real residual exit-p90 ~6% (TV holds lots days longer) correctly remains surfacedScripts-only change (no
src//include/); added a doctest for the helper.🤖 Generated with Claude Code