Background loading

Background loader with throttled retrieval

This document was created as part of the work of the new document format using a table-of-contents.

Motivation

Reading files from a high latency network drive is a major reason for an unresponsive user interface. Moving I/O operations out of the main thread can greatly improve this. Background loading is already implemented for document scanning when building the tree model for the document mode. However I/O operations are currently executed in the main thread for loading thumbnails, scenes and media assets. Moving the I/O operations out of the main thread could greatly improve user experience.

With the introduction of the TOC this gets even more important. Especially during the transition phase it is necessary to scan complete documents, i.e. to read all scenes of a document in order to identify referenced media assets. This scanning should be done in background without blocking of the UI.

To achieve this, we introduce a background loader which can read files on other threads and provides the results asynchronously.

Requirements

Single files or list of many files

The background loader should be suitable to load a list of many files (e.g. all scenes of a document). But in other cases it might also be necessary to load just a single file in background, i.e. when loading the media assets for a scene or when preloading a neighboring scene.

Allow throttling to align with the processing speed of the results

When the files reside on a local fast disk, then background loading should align to the processing speed of the result in order to avoid too many pending results occupying memory, which are eventually unused in the end.

Abort running pipeline

It might also be necessary to abort a running background loader, e.g. when the user selects another document in document mode before all thumbnails for the currently selected document are loaded.

Architecture

The QtConcurrent framework already contains the base for the functionality needed for background loading:

• Distribute workload to multiple threads.
• Emit signals when results are available.
• Throttle worker threads according to pending unprocessed results.
• Abort running pipeline.

For throttling, we have however one additional requirement which is not fulfilled out of the box by QtConcurrent.

Throttling using setPendingResultsLimit() only works if resultReadyAt() blocks. However we are processing the scene files on the main thread in several small steps (see issue about loading scenes in background), so the slot receiving the file contents does not block. In order to solve this we propose the following:

• Add a blocking buffer for the results.
• This buffer runs in a separate thread to be able to make it blocking.
• This buffer receives the signals from the QFutureWatcher.
• Run the QFutureWatcher in a separate thread to allow blocking on emitting a signal.
• The buffer emits a signal when a new result was added.
• When processing in main thread is complete, it unblocks the buffer again for the next result.
• Use a separate thread pool with a small number of threads so that tasks from one mapping are not blocked by tasks from another mapping.

Consider

• we also need a separate thread for the QFutureWatcher, because it is blocked by the buffer
• careful design the connections, ownership and lifetime of the threads
• QThread can run an event loop, but doesn't have to
• event loop is necessary when receiving queued signals from another thread

Solution

The following diagram sketches a solution with three separate threads:

• The Main thread of the application running an event loop
• A thread running a blocking buffer with no event loop
• A thread running the FutureWatcher running an event loop

sequenceDiagram
  participant Main thread
  participant Blocking buffer
  participant FutureWatcher
  FutureWatcher ->> Blocking buffer: resultReadyAt<br/>[direct]
  activate Blocking buffer
  Blocking buffer ->> Main thread: resultAvailable<br/>[queued]
  Main thread ->> Blocking buffer: resultProcessed<br/>[direct]
  deactivate Blocking buffer

Loading

Note that we're not running an event loop for the Blocking buffer. This means that all connections to slots on this thread must be direct, because queued connections need an event loop to deliver the signals.

When a result is ready, the FutureWatcher emits a resultReadyAt signal, which is connected to the blocking buffer in a direct call. The Blocking buffer does not return from that slot until it receives the resultProcessed signal later. This indicates to the FutureWatcher that this result is still processed until then.

The Blocking buffer signals the availability of a result to the Main thread with the resultAvailable signal.

The Main thread processes the result in many small steps, interwoven to other activities. When processing is complete, it indicates this to the Blocking buffer using the resultProcessed signal.

The Blocking buffer then releases the block, causing the invocation of the resultReadyAt signal to return. The FutureWatcher then knows that processing of this result is complete and can adjust the throttling accordingly.

Future thoughts, TBD

QFutureWatcher

• create a QThread
• move QFutureWatcher to the thread
• run the event loop of this thread

Blocking Buffer

• no need for an event loop
• override run to fetch results from the queue
• emit signal when result is available and buffer is unblocked
• connect QFutureWatcher::resultReadyAt to the buffer using a direct connection
• connect consumer on main thread using a queued connection (default)

Starting threads

• create and starts the thread for the QFutureWatcher
• create the QFutureWatcher and move it to this thread
• create the blocking buffer as derived class of QThread
• connect it to the QFutureWatcher (direct)
• connect it to the background loader (queued)
• start the blocking buffer

Terminating threads

I think termination must mainly be triggered from the destructor of the background loader

• delete the QFutureWatcher
• quit the event loop
• join the thread
• terminate the blocking queue, making sure it gets unblocked
• join the thread
• both threads could be child objects of the background loader and destroyed automatically