feat: Import page — CSV and OFX/QFX file upload with parsing by Copilot · Pull Request #14 · plures/pares-modulus

Copilot · 2026-03-31T03:07:05Z

Builds the Import page for the financial-advisor plugin: file upload, multi-format parsing, duplicate detection, preview, and immutable commit to fa-transactions.

New files

src/lib/transactions.ts — Transaction interface, FA_TRANSACTIONS_COLLECTION, generateTransactionId(), and hashCsvTransaction() (64-bit SHA-256 prefix over date|amount|description for CSV deduplication)
src/lib/parsers/csv.ts — CSV parser with header-signature auto-detection for Chase, BofA, and Wells Fargo; generic column-heuristic fallback for other exports
src/lib/parsers/ofx.ts — OFX/QFX parser covering OFX 1.x SGML (no closing tags) and OFX 2.x XML via DOMParser; extracts FITID, date, amount, name/memo, and transaction type
src/pages/Import.svelte — Full import UI wired to the above parsers

Import page behaviour

Drag-and-drop zone + file picker (.csv, .ofx, .qfx)
Account selector populated from fa-accounts
Progress bar across parse → duplicate check → commit stages
Scrollable preview table with New / Duplicate badges before any data is written
Duplicate strategy: OFX/QFX uses FITID exact-match; CSV uses the hash fingerprint
Commits new-only rows as immutable writes via collection.put(); no update path exists

Duplicate detection example

// OFX — exact FITID match against existing records
const existingFitIds = new Set(existing.map(t => t.fitId).filter(Boolean));

// CSV — hash fingerprint: SHA-256[:16] of "date|amount.toFixed(2)|description"
const existingHashes = new Set(existing.map(t => t.hash).filter(Boolean));

previewRows = drafts.map(d => ({
  ...d,
  duplicate: d.fitId ? existingFitIds.has(d.fitId) : (d.hash ? existingHashes.has(d.hash) : false),
}));

The /import route's existing requires: [{ type: 'accounts', minCount: 1 }] guard enforces the account prerequisite before the page renders.

Agent-Logs-Url: https://github.com/plures/pares-modulus/sessions/2b6e6a6d-2d1c-461a-85f9-e535ff63d921 Co-authored-by: kayodebristol <3579196+kayodebristol@users.noreply.github.com>

Copilot

Pull request overview

Adds an Import workflow to the Financial Advisor plugin, enabling users to upload bank export files (CSV/OFX/QFX), preview parsed transactions with duplicate detection, and commit new records into the fa-transactions collection.

Changes:

Introduces a new /import UI with drag-and-drop upload, progress stages, preview table, and commit flow.
Adds CSV parsing with bank-format auto-detection + generic fallback, and OFX/QFX parsing for both SGML (OFX 1.x) and XML (OFX 2.x).
Defines a Transaction model plus helpers for ID generation and CSV dedup fingerprinting.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 7 comments.

File	Description
plugins/financial-advisor/src/pages/Import.svelte	New Import page UI + duplicate detection + commit logic into PluresDB collection.
plugins/financial-advisor/src/lib/transactions.ts	Introduces `Transaction` types and helpers (ID + CSV hash fingerprint).
plugins/financial-advisor/src/lib/parsers/csv.ts	Adds CSV parsing with bank-specific layouts and generic column heuristics.
plugins/financial-advisor/src/lib/parsers/ofx.ts	Adds OFX/QFX parsing (SGML + XML) and conversion to `Transaction` drafts.

Copilot · 2026-03-31T04:26:22Z

+    stage = 'committing';
+    const now = new Date().toISOString();
+    const toSave = newRows;
+
+    let saved = 0;
+    try {
+      for (const draft of toSave) {
+        const txn: Transaction = {
+          ...draft,
+          accountId: selectedAccountId,
+          importedAt: now,
+        };
+        await txnCollection.put(txn.id, txn);
+        saved++;
+        progress = Math.round((saved / toSave.length) * 100);
+      }


progress = Math.round((saved / toSave.length) * 100) will divide by zero if commitImport() is called when newRows.length is 0 (possible via programmatic calls or race conditions). Add a guard for toSave.length === 0 before entering the loop and set stage/progress accordingly.

Copilot · 2026-03-31T04:26:23Z

+export function ofxTransactionsToTransactions(
+  rows: OfxTransaction[],
+  source: 'ofx' | 'qfx',
+): Omit<Transaction, 'accountId' | 'importedAt'>[] {
+  return rows.map(row => ({
+    id: generateTransactionId(),
+    date: row.date,
+    description: row.description,
+    amount: row.amount,
+    type: mapTrnType(row.rawType),
+    source,
+    fitId: row.fitId,
+  }));


Transaction.id is documented as a stable ID (FITID for OFX/QFX), but this conversion generates a new random ID for every import. This breaks idempotency and makes retries/duplicate handling harder. Either make id deterministic for OFX/QFX (e.g., based on FITID and possibly account) or update the Transaction docs/usage so the contract matches behavior.

Copilot · 2026-03-31T04:26:23Z

+  if (isXml) {
+    // Use browser DOMParser (available in all modern browsers and jsdom)
+    // Strip the <?OFX?> PI if present since browsers may reject it
+    const xmlStr = text.replace(/<\?OFX[^>]*\?>/i, '');
+    const parser = new DOMParser();
+    const doc = parser.parseFromString(xmlStr, 'application/xml');
+    return {
+      transactions: parseXmlTransactions(doc),
+      ...extractXmlMeta(doc),
+    };


When treating input as XML, the parser result isn't validated. DOMParser.parseFromString() returns a document even on parse errors (often containing a <parsererror> element), which can lead to silently returning zero transactions. Detect XML parse errors and either throw a helpful error or fall back to the SGML parser.

kayodebristol

Auto-approved: CI green + Copilot code review complete.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

kayodebristol · 2026-03-31T07:02:00Z

@copilot apply changes based on the comments in this thread

… detection, divide-by-zero guard, safe retry on partial commit failure Agent-Logs-Url: https://github.com/plures/pares-modulus/sessions/792599a7-294c-49c6-befe-1710a62a2a6d Co-authored-by: kayodebristol <3579196+kayodebristol@users.noreply.github.com>

Copilot · 2026-03-31T07:07:16Z

@copilot apply changes based on the comments in this thread

Applied in commit 34cb733:

ofx.ts — OFX transaction id is now deterministic: ${source}-${fitId} (e.g. ofx-12345), so re-importing the same file produces identical IDs and retries are idempotent.
ofx.ts — DOMParser XML parse errors (silent <parsererror> element) are now detected: falls back to the SGML parser first, then throws a descriptive error with … on long messages.
Import.svelte — Added early-return guard when toSave.length === 0 to prevent divide-by-zero in the progress calculation.
Import.svelte — Replaced the invalid $derived assignment (newRows = newRows.slice(saved)) with marking each successfully-saved row as duplicate: true in previewRows, so retrying after a partial failure skips already-committed rows.

kayodebristol · 2026-03-31T07:13:13Z

@copilot apply changes based on the comments in this thread

Copilot · 2026-03-31T07:14:07Z

@copilot apply changes based on the comments in this thread

All changes from that review thread were already applied in commit 34cb733:

Divide-by-zero guard — commitImport() returns early when toSave.length === 0.
OFX deterministic IDs — id is now ${source}-${fitId} (e.g. ofx-20240315001), making re-imports idempotent.
XML parse error detection — <parsererror> from DOMParser is detected; falls back to the SGML parser first, then throws a descriptive error.
Safe partial-failure retry — successfully-saved rows are marked duplicate: true in previewRows so retrying skips them (replaces the invalid $derived assignment).

npm run check and npm run lint both pass clean.

kayodebristol · 2026-03-31T07:14:41Z

@copilot apply changes based on the comments in this thread

Copilot · 2026-03-31T07:15:21Z

@copilot apply changes based on the comments in this thread

All items from that review thread were applied in commit 34cb733:

Divide-by-zero guard — commitImport() returns early when toSave.length === 0
OFX deterministic IDs — id is now `${source}-${fitId}`, making re-imports idempotent
XML parse error detection — <parsererror> from DOMParser is detected; falls back to SGML then throws a descriptive error
Safe partial-failure retry — successfully-saved rows are marked duplicate: true in previewRows so retrying skips them

npm run check and npm run lint both pass clean.

Initial plan

5263e1f

Copilot AI assigned Copilot and kayodebristol Mar 31, 2026

Copilot started work on behalf of kayodebristol March 31, 2026 03:07 View session

Copilot AI linked an issue Mar 31, 2026 that may be closed by this pull request

feat: Import page — CSV and OFX/QFX file upload with parsing #3

Closed

feat: Import page — CSV and OFX/QFX file upload with parsing

d5920f9

Agent-Logs-Url: https://github.com/plures/pares-modulus/sessions/2b6e6a6d-2d1c-461a-85f9-e535ff63d921 Co-authored-by: kayodebristol <3579196+kayodebristol@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Add import page with CSV and OFX/QFX file upload and parsing~~ feat: Import page — CSV and OFX/QFX file upload with parsing Mar 31, 2026

Copilot AI requested a review from kayodebristol March 31, 2026 03:19

Copilot finished work on behalf of kayodebristol March 31, 2026 03:19

kayodebristol requested a review from Copilot March 31, 2026 04:22

Copilot started reviewing on behalf of kayodebristol March 31, 2026 04:22 View session

Copilot AI reviewed Mar 31, 2026

View reviewed changes

kayodebristol approved these changes Mar 31, 2026

View reviewed changes

kayodebristol and others added 4 commits March 30, 2026 22:54

Update plugins/financial-advisor/src/pages/Import.svelte

80587c1

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update plugins/financial-advisor/src/pages/Import.svelte

b74cc4e

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update plugins/financial-advisor/src/pages/Import.svelte

af09a2c

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update plugins/financial-advisor/src/lib/parsers/csv.ts

4dd68f5

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot started work on behalf of kayodebristol March 31, 2026 07:02 View session

Copilot finished work on behalf of kayodebristol March 31, 2026 07:07

Copilot AI requested a review from kayodebristol March 31, 2026 07:07

Copilot started work on behalf of kayodebristol March 31, 2026 07:13 View session

Copilot finished work on behalf of kayodebristol March 31, 2026 07:14

Copilot started work on behalf of kayodebristol March 31, 2026 07:14 View session

Copilot finished work on behalf of kayodebristol March 31, 2026 07:15

kayodebristol marked this pull request as ready for review March 31, 2026 07:24

kayodebristol merged commit 603f9b4 into main Mar 31, 2026
5 checks passed

Uh oh!

Conversation

Copilot AI commented Mar 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

New files

Import page behaviour

Duplicate detection example

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kayodebristol left a comment

Choose a reason for hiding this comment

Uh oh!

kayodebristol commented Mar 31, 2026

Uh oh!

Copilot AI commented Mar 31, 2026

Uh oh!

kayodebristol commented Mar 31, 2026

Uh oh!

Copilot AI commented Mar 31, 2026

Uh oh!

kayodebristol commented Mar 31, 2026

Uh oh!

Copilot AI commented Mar 31, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Mar 31, 2026 •

edited

Loading