Move resource-intensive cryptographic operations off the main thread using Isolates

[HamdaanAliQuatil]

Several cryptographic operations are computationally expensive and block the main thread, causing UI Janks and a poor user experience.

The goal is to identify such operations and isolate the intensive computation to background threads.
Use dart:isolates to create cheap isolates.

[jonasfj]

We need to determine what is worth moving off the main thread and what isn't.

I think a clear example of something definitely worth moving is RSA key generation.

• Not much data to copy between isolates
• Finding large primes is expensive!

I clear example of something not worth moving is AES key generation.

• Generation of random data
• Only a few bytes at most
• Dirt cheap, cost of starting an isolate and communicating between isolates greatly exceeds generating a few random bytes.

Unless, of course the source of secure random bytes is empty and the thread is blocked while reading from /dev/srandom. But that's kind of an extreme corner case -- I think once BoringSSL is initialized with an initial seed it has an infinite source of random data. So let's not optimize for extreme cases like this, where moving the mouse generates the randomness that wasn't there.

---

I don't know the exact criteria for moving an operation off the main thread. So let's begin that discussion here.

One thing worth keeping in mind is that even filling a large array with random data, or computing the hash for a large blob, is probably extremely cheap. And we could do it in 4MB increments with await Future.delayed(Duration(0)) in between each increment. Ensuring that there is time for other operations to run.

That doesn't offload the operation to another isolate. It merely splits large operations into multiple microtasks running on the same isolate. But by splitting into separate microtasks we avoid blocking UI rendering, etc.

So we have 3 options for an operation:

• (a) Do it on the main thread synchronously (what we currently do).
• (b) Do it on the main thread asynchronously, by doing it in small increments interleaved with other microtasks.
• (c) Do it on an background isolate, copying the data to said isolate.

While (c) is probably least blocking, it also carries the most overhead. Imagine processing a stream, we'd have to copy event block we get from the stream to the background isolate. Or at least pass an FFI pointer to the block to the isolate. That's a lot of isolate messages going back and forth. Because we'd also have to support a reasonable level of bacl-pressure on the stream, hence, we'd also need to send ack messages back for every block.

I don't know how efficient, isolate messaging is. But I imagine that if we did it for sha256, we'd only be making the process slower overall. So again (and this is just my intuition that says this) perhaps (a) or (b) is the best option for hashing bytes and streams with sha256.

Anyways, I'm open to ideas on deciding which operations are worth moving and which aren't.

Perhaps we can say that of the smallest microtask in an implementation using (b) exceeds x milliseconds then we'll move it to an isolate. I'm not sure what x is. And perhaps we can tolerate dropping a single frame, IDK? Anyone who really cares about perfection can always move parts of their logic to a background isolate (which is probably a better option when doing many relatively cheap crypto operations in a single method).

Suggestions for x and what hardware we'd what to measure it on are welcomed.

[HamdaanAliQuatil]

Updates on this task:

For ECDH, offloading costs (spawn/IPC/export) exceed doing the work on the main isolate. We’ll not offload ECDH. Closing #201

Users shouldn’t have to guess when we offload. We’ll only auto-offload known-slow classes (prime finding, key-strengthening). Everything else stays on the calling isolate - no per-device heuristics.

We'll evaluate offloading PBKDF2, perhaps past a documented threshold.