Skip to content

feat: client mode improvements — sync, HA export, scheduler, display fixes#110

Open
jrozelle wants to merge 9 commits intoMyElectricalData:mainfrom
jrozelle:main
Open

feat: client mode improvements — sync, HA export, scheduler, display fixes#110
jrozelle wants to merge 9 commits intoMyElectricalData:mainfrom
jrozelle:main

Conversation

@jrozelle
Copy link

@jrozelle jrozelle commented Feb 21, 2026 •
edited
Loading

Summary

Client mode — Sync & data pipeline

  • Sync off-by-one fix: <= instead of < in _find_missing_ranges (sync.py + local_data.py) so yesterday's data is properly detected as missing and synced
  • NULLS NOT DISTINCT: New migration for PostgreSQL 15+ — fixes ON CONFLICT never firing for daily records (interval_start is NULL)
  • MaxPowerData model + migration: New table for daily max power (W) with interval_start
  • Max power sync fix: end_date = yesterday — Enedis API rejects end dates >= today
  • Sync improvements: deduplication, placeholder handling, MaxPower sync, better error handling/retry
  • enedis_client: local-first cache, dedup, backoff logic for detailed data chunks
  • Adapter: use_cache=true parameter on API calls

Home Assistant export — content-card-linky compatibility

  • Rich attributes: dailyweek, dailyweek_HC, dailyweek_HP, dailyweek_cost, dailyweek_MP, dailyweek_MP_time, dailyweek_Tempo, evolution percentages (yearly/monthly/weekly/yesterday), HP/HC breakdown, costs, peak/offpeak percent
  • Prefix-aware MQTT topics: uses self.prefix from config (not hardcoded myelectricaldata_) — allows v1 and v2 instances to coexist without topic collision
  • Statistics DETAILED fallback: get_day_total tries DAILY first, falls back to summing DETAILED records (W/2 = Wh)

Scheduler (~20 API calls/day)

  • Startup sync (immediate on container restart)
  • Morning sync window: 6h–9h every 30min (~8 calls)
  • Daytime checks: 12h + 18h (2 calls)
  • Export after sync: _run_sync() triggers _run_all_exports() automatically
  • Export scheduler: checks every 1 minute for due exports
  • Tempo: 7h + 11h (2 calls, aligned with RTE publication times)
  • EcoWatt: 17h daily + Friday 12h15 + fallback 8h30/20h30 if J+3 incomplete (~4 max)
  • France consumption: 8h/14h/20h (3 calls)
  • Generation forecast: 9h/21h (2 calls)

Frontend fixes

  • endDate off-by-one: all fetch hooks and pages use today (not yesterday) as exclusive end bound
  • queryKey mismatch: consumptionDaily/productionDaily keys for proper cache invalidation
  • Tempo chart dedup: keeps most complete year entry (not first occurrence)
  • DetailedCurve: improved weekly navigation
  • HomeAssistant.tsx: prefix-aware entity preview
  • MQTT.tsx / VictoriaMetrics.tsx: runtime env defaults via window.__ENV__
  • PDLDetails.tsx: HTML entity escaping (XSS protection)
  • vite-env.d.ts + entrypoint.sh: VITE_DEFAULT_* env vars for runtime config

Other

  • Remove TrustedHostMiddleware (caused issues behind reverse proxy)
  • Remove Synology @eadir files + .gitignore entry
  • MQTT exporter: TempoDay import fix + defensive enum handling
  • Export router: schedule fields + model_fields_set logic
  • Ecowatt router: return format with created/updated counts

Test plan

  • Deploy client mode, verify sync fetches yesterday's data correctly
  • Verify content-card-linky v2 card displays all attributes (HC/HP, costs, tempo, evolution %)
  • Verify v1 and v2 HA instances don't collide (different MQTT prefixes)
  • Verify scheduler logs show ~20 API calls/day pattern
  • Verify Tempo chart shows complete curves for all years
  • Verify max power sync works (end_date = yesterday)
  • Verify git pull no longer fails with Synology @eadir errors

🤖 Generated with Claude Code

jrozelle and others added 5 commits February 21, 2026 22:37
@jrozelle
remove middleware
27ef2cb
@jrozelle @claude
fix: keep most complete year entry in Tempo chart deduplication
da0d23f 
When a Tempo year (e.g. 2024-2025) appeared both as "previous" of a
newer block and as "current" of its own block, the deduplication kept
the first (incomplete) version, causing gaps in the chart curve.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@jrozelle @claude
chore: remove Synology @eadir files and add to .gitignore
0912f66 
These metadata files created by Synology NAS were polluting the repo
and causing git pull errors (bad object refs/@eaDir/heads@SynoEAStream).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@jrozelle @claude
fix: resolve HA export issues (sync off-by-one, MQTT topic collision,…
a2bdb89 
… stats fallback)

- Fix off-by-one in _find_missing_ranges: use <= instead of < so yesterday's
  data is properly detected as missing and synced from Enedis
- Add DETAILED granularity fallback in get_day_total when DAILY data is missing
- Use self.prefix instead of hardcoded "myelectricaldata_" in all MQTT topics,
  unique_ids, and subscription patterns to prevent v2 overwriting v1 sensors
- Add queryClient.invalidateQueries after HA export mutation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@jrozelle @claude
fix: restore rich HA export attributes + NULLS NOT DISTINCT constraint
ad0a102 
Previous session accidentally replaced the entire home_assistant.py and
sync.py files instead of making targeted prefix edits, losing ~600 lines
of content-card-linky compatible attributes (HC/HP breakdown, costs,
max power, tempo colors, weekly/monthly/yearly stats, evolution %).

This commit:
- Restores home_assistant.py from e73fbb4 (with rich attributes)
- Restores sync.py from e73fbb4 (with deduplication, MaxPower, placeholder handling)
- Restores HomeAssistant.tsx from e73fbb4 (with prefix-aware entity preview)
- Keeps the <= fix in _find_missing_ranges (applied on top of restored sync.py)
- Keeps statistics.py DETAILED fallback and simplified queries (no duplicates
  possible now thanks to NULLS NOT DISTINCT)
- Adds NULLS NOT DISTINCT migration for consumption_data and production_data
  unique constraints (fixes ON CONFLICT never firing for daily records)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
jrozelle and others added 3 commits February 23, 2026 12:39
@jrozelle @claude
fix: restore all regressed files from e73fbb4
e0411b4 
Previous session accidentally overwrote multiple files, losing:
- MaxPowerData model + migration
- TrustedHost security middleware
- Adapter cache parameters
- enedis_client local-first cache, dedup, backoff logic
- MQTT exporter model imports and defensive coding
- Scheduler startup sync + export-after-sync
- Export router schedule fields
- PDLDetails XSS protection
- MQTT/VictoriaMetrics env-based defaults
- Ecowatt return format handling

All files restored from e73fbb4, with only intentional changes kept:
- sync.py: <= fix for date boundaries
- statistics.py: DETAILED fallback + simplified queries
- NULLS NOT DISTINCT migration (new)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@jrozelle @claude
fix: calm scheduler (~20 API calls/day) and remove TrustedHost middle…
d00e7d7 
…ware

Scheduler: replace aggressive polling (24 Tempo/day, 6h intervals) with
targeted cron jobs aligned to data publication times.
Remove TrustedHostMiddleware that caused issues behind reverse proxy.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@jrozelle @claude
fix: re-apply display fixes from 03b3616 (endDate off-by-one, queryKe…
d2d9eb2 
…y mismatch)

Cherry-pick of 03b3616 that was lost when files were restored from e73fbb4:
- local_data.py: inclusive end_date (<=), MaxPower queries, placeholder handling
- Frontend hooks: endDate = today (not yesterday) for inclusive backend queries
- Frontend pages: same endDate fix in ConsumptionEuro, ConsumptionKwh, Production
- invalidateQueries keys: consumptionDaily/productionDaily (not consumption/production)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@jrozelle jrozelle changed the title fix: Tempo chart gaps + cleanup Synology files feat: client mode improvements — sync, HA export, scheduler, display fixes Feb 23, 2026
@jrozelle
Copy link
Author

jrozelle commented Feb 23, 2026

J'ai modifié la PR car elle m'a écrasé des commit précédents...

@jrozelle @claude
fix: max_power sync end_date must be yesterday (Enedis rejects end > …
45a8c66 
…today)

Enedis API returns error "end date must be earlier than current date"
when end_date is today+1. Since _find_missing_power_ranges adds +1 day
for exclusive end bounds, using date.today() resulted in API calls with
end=tomorrow, which Enedis always rejected with data=None.

Fix: use yesterday as end_date since today's max_power isn't published yet.
Also remove debug logging from previous commits.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@jrozelle