Attachments overhaul

[nickchomey]

Note: I opened a previous PR, but something broke with my git and the PR got closed. This is a new one.

This PR changes RxDB's attachments to be handled and saved as Blobs, rather than Base64.

It also adds support for including multiple attachments in the insert and upsert functions, and also adds a putAttachments function for adding multiple attachments in one operation.

Attachments will be compressed if compression is enabled, but they will only be selectively compressed baased on the mime type provided when the attachment was created.

Encryption is also done on the binary data rather than base64.

The RxStorage implementations were modified as-needed to support binary data (eg Foundationdb now handles binary).

Localstorage and websocket remain base64. Websocket could, in theory, be changed to use binary data, but that seems like something that would be a larger breaking change for users of that plugin.

Various functions have been tidied up, removed, etc accordingly.

Tests were created for all functionality, and pass on the fast, node, bun and deno variations. MongoDB was not tested, as I couldn't figure out how to run it. Some bypasses were added for test:deno:dexie as denoland/deno#12067 (comment). This should be irrelevant, since dexie would never realistically be used on the server - Node and Bun work for testing.

This satisfies this Premium Task, and these backlog issues:

• https://github.com/pubkey/rxdb/blob/master/orga/BACKLOG.md#add-plugin-for-attachments-compression
• https://github.com/pubkey/rxdb/blob/master/orga/BACKLOG.md#upsert-atomic-with-attachments

Please let me know if you have any questions, concerns, etc...!

[Copilot]

Pull request overview

This PR overhauls RxDB attachments to use Blob as the internal canonical data type (instead of base64 strings), extends collection/document APIs to support multiple/inline attachments on insert/upsert, and updates storage wrappers/implementations (encryption, compression, remote transports) accordingly.

Changes:

• Switch attachment write/read paths to Blob (including hashing on binary data and RxStorageInstance.getAttachmentData(): Promise<Blob>).
• Add multi-attachment APIs (putAttachments) and inline attachments support for insert/upsert flows (with attachment merge/replace options).
• Update compression/encryption/storage backends and expand tests/docs to reflect new behavior (including selective MIME-type-based compression).

Reviewed changes

Copilot reviewed 35 out of 40 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
test/unit/util.test.ts	Adds coverage for hashing Blob inputs and clone() behavior with Blob.
test/unit/rx-storage-implementations.test.ts	Updates attachment tests to use Blob and adds Deno+dexie skip for structuredClone issues.
test/unit/rx-database.test.ts	Adjusts custom hash function typing to accept binary inputs.
test/unit/replication.test.ts	Skips attachment replication tests for Deno+dexie due to Blob structuredClone limitations.
test/unit/replication-protocol.test.ts	Updates protocol tests to use Blob attachments; adds Deno+dexie skip.
test/unit/migration-schema.test.ts	Adds Deno+dexie skip for attachment migration tests due to Blob clone issues.
test/unit/attachments.test.ts	Updates attachment tests to Blob and adds extensive coverage for inline attachments + encryption/compression interactions.
test/unit/attachments-compression.test.ts	Migrates compression tests to Blob and adds selective MIME-type compression tests.
src/types/util.d.ts	Expands HashFunction input type to `string
src/types/rx-storage.interface.d.ts	Updates storage contract/comments for structured-cloneable data and getAttachmentData(): Promise<Blob>.
src/types/rx-storage.d.ts	Changes RxAttachmentWriteData.data to Blob and updates rationale/docs in typings.
src/types/rx-schema.d.ts	Adds compressibleTypes?: string[] to attachment schema options.
src/types/rx-document.d.ts	Adds putAttachments() API typing.
src/rx-storage-helper.ts	Removes base64 attachment-size helper and uses Blob.size for length.
src/rx-collection.ts	Normalizes inline attachments during insert/upsert; adds upsert attachment merge/replace option.
src/rx-collection-helper.ts	Adds attachments alias support and introduces normalizeInlineAttachments() for inline attachment normalization.
src/plugins/utils/utils-object.ts	Updates deep clone to preserve Blob instances by reference.
src/plugins/utils/utils-hash.ts	Updates nativeSha256 to hash `string
src/plugins/utils/utils-blob.ts	Removes redundant getBlobSize() helper.
src/plugins/storage-sqlite/sqlite-storage-instance.ts	Updates interface type of getAttachmentData() to return Blob (still unimplemented).
src/plugins/storage-remote/rx-storage-remote.ts	Updates remote storage instance typing for getAttachmentData(): Promise<Blob>.
src/plugins/storage-remote-websocket/index.ts	Adds JSON-boundary base64<->Blob conversion for websocket transport.
src/plugins/storage-mongodb/rx-storage-instance-mongodb.ts	Updates return type of getAttachmentData() to Blob (still unimplemented).
src/plugins/storage-memory/rx-storage-instance-memory.ts	Updates memory storage getAttachmentData() signature to return Blob.
src/plugins/storage-localstorage/rx-storage-instance-localstorage.ts	Serializes Blob to base64 for localStorage storage and returns Blob on read.
src/plugins/storage-foundationdb/rx-storage-instance-foundationdb.ts	Stores attachment payloads as raw binary buffers; returns Blob on read.
src/plugins/storage-foundationdb/foundationdb-types.ts	Updates attachments DB typing from JSON to Buffer.
src/plugins/storage-dexie/rx-storage-instance-dexie.ts	Updates dexie attachment storage and getAttachmentData() to use Blob.
src/plugins/storage-denokv/rx-storage-instance-denokv.ts	Updates signature to return Blob (still unimplemented).
src/plugins/migration-storage/index.ts	Migrates attachments by pulling Blob from old storage and writing Blob into new storage.
src/plugins/encryption-crypto-js/index.ts	Moves attachment encryption/decryption to binary/base64-at-boundary while keeping Blob in storage pipeline.
src/plugins/dev-mode/error-messages.ts	Adds new error codes/messages for inline attachment validation.
src/plugins/dev-mode/check-document.ts	Adjusts structured-clone/JSON check to strip attachment Blob payloads before JSON roundtrip validation.
src/plugins/attachments/index.ts	Reworks attachment read/write to use Blob; adds putAttachments().
src/plugins/attachments/attachments-utils.ts	Updates attachment-data filling to pull Blob from storage when needed.
src/plugins/attachments-compression/index.ts	Switches compression to operate on Blob streams and adds selective MIME-type-based compression.
src/plugin-helpers.ts	Updates wrapper hook signature for attachment transformation to operate on Blob.
docs-src/docs/rx-collection.md	Documents upsert attachment merge/replace behavior and options.
docs-src/docs/rx-attachment.md	Documents Blob-based attachments, putAttachments(), inline attachments, and MIME-type-aware compression.
config/karma.conf.cjs	Adds MOCHA_GREP support in Karma config for faster test iteration.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[pubkey]

This runs in a hot path so microseconds performance here is important and await is expensive:

• Skip it if attachments is not enabled in the schema.
• Skip per document if attachments array is empty.

[nickchomey]

added the if checks.

Is there a better non-promise way to do it?

[Copilot]

Pull request overview

Copilot reviewed 39 out of 44 changed files in this pull request and generated 3 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[nickchomey]

Looks like all the tests are passing now. Strange - there were random NATS and browser:remote errors earlier. Probably just CI issues...

what do you want me to do about the docs? The mime type injection works locally with npm run docs:build and npm run docs:serve. Do i need to provide built versions in the PR or do they get rebuilt in CI?

[Copilot]

Pull request overview

Copilot reviewed 39 out of 44 changed files in this pull request and generated 4 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 39 out of 44 changed files in this pull request and generated 5 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.