Supernote-KOReader Harmonizer

Sync your KOReader highlights to Supernote Digest. Read on your e-reader, review in Supernote.

Why Use This?

The Problem: You use KOReader on an e-ink device for reading, but want your highlights accessible in Supernote's Digest for review, export, or handwritten notes.

The Solution: This tool syncs KOReader highlights directly into Supernote's database, so they appear in your Digest alongside native Supernote highlights.

Key Features

• KOReader → Supernote: Highlights appear in Digest with correct book links
• Location in Notes: Since page numbers don't display, location is embedded in notes (e.g., [Loc: Chapter 5, p.42])
• Round-trip Note Editing: Edit notes in either app - changes sync back with timestamp-based conflict resolution
• Hash-based Matching: Books are matched by content hash, not filename, so renamed files still sync correctly
• Safe Dry-run Mode: Preview all changes before applying

What Works and What Doesn't

Direction	What Works	What Doesn't
KOReader → Supernote	Text, notes, book links, location in notes	Page numbers in UI, position navigation
Supernote → KOReader	Note changes for KOReader-originated highlights	Supernote-originated highlights don't appear in KOReader

Bottom line: Create highlights in KOReader, review them in Supernote Digest. Note edits sync both ways.

Requirements

• Python 3.10+
• Access to KOReader's hashdocsettings directory (synced via Syncthing or similar)
• Access to Supernote Personal Cloud's MariaDB database (Docker)
• mysql-connector-python package
• Local copies of your ebook files (for hash-based matching)

Data Sync Setup

KOReader metadata (hashdocsettings/): Sync this folder from your e-reader using Syncthing. KOReader stores all highlights and reading progress here.

To enable hash-based storage in KOReader:

1. Tap the gear icon (⚙️) → Settings → Document → Save document settings
2. Select hash-based per-document settings in hashdocsettings folder

This stores metadata in hashdocsettings/{hash}.sdr/ instead of next to each book file.

Ebook files: The tool needs local access to your ebook files for hash calculation. Options:

• Calibre / Calibre-Web-Automated: Use your existing Calibre library
• Syncthing: Sync your book library from your e-reader or Supernote
• Manual copy: Copy books to a local folder

Configure the path mapping in .env to tell the tool where to find local copies of files referenced in Supernote.

Installation

# Clone or navigate to the project
cd /path/to/supernote-koreader-harmonizer

# Create virtual environment
python3 -m venv venv
source venv/bin/activate

# Install dependencies
pip install mysql-connector-python

Configuration

1. Copy the example environment file:

   cp .env.example .env

2. Edit .env with your settings:

   # Required: Path to KOReader's hashdocsettings
   KOREADER_SETTINGS_PATH=/path/to/hashdocsettings
   
   # Required: MariaDB connection (Supernote Personal Cloud)
   MARIADB_HOST=192.168.1.100
   MARIADB_PORT=3306
   MARIADB_USER=supernote
   MARIADB_PASSWORD=your_password
   MARIADB_DATABASE=supernotedb
   
   # Optional: Map Supernote paths to local paths for hash calculation
   LOCAL_FILE_MAP_1=Document/=/path/to/local/supernote/Document/

Finding the MariaDB IP

If the Supernote container is on a Docker network:

docker inspect supernote-mariadb | grep "IPAddress"

Usage

# Activate virtual environment
source venv/bin/activate

# Dry run - see what would be synced (default)
python sync_highlights.py

# Dry run for a specific book
python sync_highlights.py --book "Amber"
python sync_highlights.py --book "Zelazny"

# Apply changes
python sync_highlights.py --apply

# Apply for specific book only
python sync_highlights.py --book "Amber" --apply

# Create KOReader metadata for books only in Supernote
# (requires local file access for hash calculation)
python sync_highlights.py --apply --create-missing

How It Works

Data Sources

KOReader stores annotations in:

hashdocsettings/{hash_prefix}/{full_hash}.sdr/metadata.{epub|pdf}.lua

Each annotation contains:

• text: The highlighted text
• note: Optional user note
• datetime / datetime_updated: Timestamps
• chapter, pageno: Location info
• pos0, pos1: XPath positions (for EPUBs)

Supernote stores digests in the t_summary table:

• content: The highlighted text
• comment_str: User notes
• source_path: Document path
• creation_time, last_modified_time: Timestamps (milliseconds)
• metadata: JSON with position data

Matching Algorithm

1. Book Matching: Documents are matched using multiple strategies:

   • Hash matching (preferred): Calculates KOReader's partial MD5 hash for reliable identification
   • Path matching: Falls back to exact or basename path matching
   • Title similarity: Last resort with warning (less reliable)

2. Highlight Matching: Uses fuzzy text matching with:

   • 85% similarity threshold
   • Punctuation normalization (handles spacing differences)
   • Substring matching (handles partial selections)

3. Conflict Resolution: When the same highlight has different notes:

   • Compare timestamps
   • Newer timestamp wins
   • Update the older system

Sync Directions

• KOReader → Supernote: New KOReader highlights are inserted into t_summary
• Supernote → KOReader: New Supernote digests are added to the Lua metadata
• Conflicts: Note differences are resolved by timestamp (newest wins)

File Structure

supernote-koreader-harmonizer/
├── sync_highlights.py      # Main synchronization script
├── koreader_hash.py        # KOReader hash algorithm implementation
├── .env.example            # Configuration template
├── .env                    # Your configuration (not tracked)
├── hash_cache.json         # Cached file hashes (auto-generated, not tracked)
├── hashdocsettings/        # KOReader metadata (symlink or copy, not tracked)
│   └── {hash[:2]}/{hash}.sdr/metadata.*.lua
└── venv/                   # Python virtual environment (not tracked)

KOReader Hash Algorithm

KOReader uses a partial MD5 checksum to identify documents. The koreader_hash.py module implements this algorithm:

• Reads 1024-byte chunks at exponentially increasing offsets: 1024 << (2*i) for i = -1 to 10
• Offsets: 256, 1024, 4096, 16384, 65536, 262144, 1048576, 4194304, 16777216, 67108864, 268435456, 1073741824 bytes
• MD5 hash of all chunks concatenated
• Folder structure: hashdocsettings/{hash[:2]}/{hash}.sdr/metadata.{ext}.lua

Position Data Mapping

KOReader and Supernote use different coordinate systems:

KOReader XPath positions:

/body/DocFragment[16]/body/p[2]/text().179

• DocFragment[N] → chapter/section index
• .N suffix → character offset within text node

Supernote location data:

{"chapter": 16, "page": 301, "startPosition": 179, "endPosition": 250}

The sync script extracts chapter index and character positions from KOReader's XPath format to populate Supernote's metadata. However, position-based navigation does not currently work for synced highlights - the location is embedded in notes as a workaround.

Limitations

• Currently single-user (uses first user in u_user table)
• KOReader file writes may need manual sync back to device
• Supernote .mark files (handwritten annotations) are not modified
• Text extraction differences between apps may cause matching failures for edge cases

Sync Direction Limitations

KOReader → Supernote:

• ✅ Highlight text and notes sync correctly
• ✅ Links to the correct book file
• ✅ Location info embedded in notes as workaround (e.g., [Loc: Chapter 5, p.42])
• ❌ Page numbers do not display in Supernote Digest UI
• ❌ Tapping a highlight opens the book but doesn't navigate to the exact position

Supernote → KOReader:

• ✅ Round-trip only: Note changes sync back correctly for highlights that originated in KOReader
• ❌ Highlights that originated in Supernote do NOT appear in KOReader at all
• KOReader requires XPath position data (pos0/pos1) to display highlights, which Supernote doesn't provide

Location Embedding

Since Supernote doesn't display page numbers for synced highlights, the tool automatically adds location information to the note field:

• No existing note: Creates a note like [Loc: Letter 23, p.301]
• Existing note: Appends location on a new line

This is applied to both systems simultaneously to prevent sync loops.

Troubleshooting

"Can't connect to MySQL server"

• Check Docker container is running: docker ps | grep mariadb
• Verify IP address: docker inspect supernote-mariadb | grep IPAddress
• Ensure you're on the same network as the container

Highlights not matching

• Check text similarity in dry-run output
• Lower TEXT_SIMILARITY_THRESHOLD if needed (default: 0.85)
• Verify books have matching paths in both systems

Permission errors on KOReader files

• Ensure write access to hashdocsettings directory
• Check file ownership matches your user

License

MIT