Expose event.getTargetRanges() in oncompositionstart (and possibly oncompositionend as well)

[michael]

Problem: On Android-Chrome there's no way to capture initial composition target range (event.getTargetRanges()) when using dictation. There may be other situations like this.

Only compositionstart and compositionend are fired, no beforeinput, which I would need to determine selection (event.getTargetRanges) right before the composition started.

Reproduce:

• On a Samsung-Android phone, set cursor inside a word
• activate dictation (word gets underlined → this is the selection I'm after)
• stop dictation immediately (before making any voice inputs)

IMG_4350.mov

Observations:

• as soon as dictation is enabled a compositionstart event is fired (now waiting for voice inputs)
• if there are no voice inputs and the dictation is cancelled by the user, still a compositionend event is fired. Assuming that at the beginning the cursor was inside the word "hello" event.data will have "hello"

Problem:

• at compositionend there's no way for me to set the selection to what should be replaced (the word that gets underlined initially)
• in my editor this means i have to assume a collapsed selection (the cursor before dictation was enabled), which will insert the word from which the composition started again at the current cursor position.

Solution ideas:

• Expose event.getTargetRanges in compositionstart event. Then i can capture the initial range that should be replaced (the word that has been underlined)
• Change implementation so event.data is "" when nothing has been dictated
• Only fire compositionstart event when something actually has been dictated.

[michael]

Here's how a reasoanable API (without breaking backwards compatibility) could look like, so I can handle this properly.

let before_input_selection;

function onbeforeinput(event) {
  if (event.isComposing) {
    // skip: handled by oncompositionstart/end
  } else {
    // handle any other inputs, including replacements using event.getTargetRanges()
  }
}

function oncompositionstart(event) {
  before_input_selection = __getModelSelectionFromTargetRange(event.getTargetRanges()[0]);
}

function oncompositionend(event) {
  // Step 1 - I need a way to roll back the DOM changes to have a clean slate for step 3
  // NOTE: I tried document.execCommand('undo') here, which would basically do the
  // same thing, it works in Chrome but breaks in weird ways in Safari/Firefox.
  event.resetDOM(); 
  // Step 2 - Set the model selection to the state right after the composition started 
  // (where no changes were applied yet - e.g. to the word the composition started from)
  doc.selection = before_input_selection;
  // Step 3 - Apply the final composition to (this will replace the contents of before_input_selection)
  const tr = doc.tr;
  tr.insert_text(event.data);
  // this applies the change to the model, and triggers an incremental rerender of the affected part of the document
  doc.apply(tr);
  // Now my incremental reactive rendering can happen reliably, knowing that the DOM is in the same state as before the composition started.
}

My approach here treats the complete composition as a single input to be applied to my internal model. My model would not reflect the in-between inputs during composition.

[masayuki-nakano]

The event source of composition events are IME and DOM Selection. On the other hand, the event source of beforeinput event is the builtin editor and its managing DOM tree and DOM Selection. Therefore, I don't think it's reasonable to add getTargetRanges to composition events.

On the other hand, if IME wants to replace existing text, I think browsers should select the range with DOM Selection before compositionstart. I think Firefox does it. The other browsers do not do that? If they do it, DOM Selection range which you can check from a compositionstart is what you want.

[johanneswilm]

The point of getTargetRanges() is to get the ranges that an event covers over - which may not always be the same as the selection. It was first introduced for input events as a new way to get static ranges in a way that is more memory efficient than dynamic selection objects. beforeinput and compositionstart are two different types of events and the meaning of their target ranges would therefore be somewhat different, but I don't see why it would be wrong to also expose target ranges for compositionstart even if its meaning is different.

The question then is if in the case of compositionstart, the target ranges are always exactly the same as the selection. If one can be sure that they are always the exact same thing, then there is arguably not a need for another method to additionally also get the target ranges.

@michael In the video example above, what is the selection at the time of compositionstart?

[michael]

@michael In the video example above, what is the selection at the time of compositionstart?

The selection is the same collapsed cursor as before you start the composition (turn dictation on). However the underlined part (whole word) is the targetrange that the composition will affect, which is why I'd like to capture it at oncompositionstart

@masayuki-nakano in most browsers I believe the selection will be the same as the composition target range at oncompositionstart, however on Samsung-Android this is different. E.g. when you start typing at the end of the word, it will start a composition that includes the whole word (so also here selection !== composition target range at compositionstart).

There are two reasons I'd like to capture event.getTargetRanges() in oncompositionstart:

1. More intuitive: It avoids adding logic to onbeforeinput to capture the target range before the composition starts (see relevant code in my editor)
2. Also covers the special case where there's oncompositionstart followed by oncompositionend directly, with no onbeforeinput in between, like in the Samsung Android Dictation On/Off scenario. Here's a relevant workaround to avoid the problem shown in the video.

[masayuki-nakano]

@michael

in most browsers I believe the selection will be the same as the composition target range at oncompositionstart, however on Samsung-Android this is different. E.g. when you start typing at the end of the word, it will start a composition that includes the whole word (so also here selection !== composition target range at compositionstart).

In my understanding, when browsers dispatch compositionstart, the Selection should've already been updated for the range which will be replaced by the composition. And that must be enough for your case.

There are two reasons I'd like to capture event.getTargetRanges() in oncompositionstart:

1. More intuitive: It avoids adding logic to onbeforeinput to capture the target range before the composition starts (see relevant [code](https://github.com/michael/svedit/blob/ce56d875dc8d57a28e4aa413697a639c5d3e23d4/src/lib/Svedit.svelte#L108-L112) in my editor)

I'm not sure about this...

2. Also covers the special case where there's `oncompositionstart` followed by `oncompositionend` directly, with no `onbeforeinput` in between, like in the Samsung Android Dictation On/Off scenario. Here's a relevant  [workaround](https://github.com/michael/svedit/blob/ce56d875dc8d57a28e4aa413697a639c5d3e23d4/src/lib/Svedit.svelte#L239-L242) to avoid the problem shown in the video.

Well, sounds like that it'd be nicer if UI Events changes the definition of compositionstart. That's defined as:

A user agent MUST dispatch this event when a text composition system is enabled and a new composition session is about to begin (or has begun, depending on the text composition system) in preparation for composing a passage of text.

So, currently, compositionstart is exposed event of the native composition start.

However, in my experience, this does not work well if browsers do so strictly. Linux's Korean IME and Chinese IME may start composition immediately after an editable thing gets focus and ends composition at its blur. However, this must not be expected by web apps. So, I think browsers should dispatch compositionstart when the composition string becomes non-empty and compositionend when the composition string is either partially or fully committed. Then, the odd empty composition won't happen.

Cc: @garykac