Presence cursors

[curran]

Cursors that show the other users' positions and selections.

Maybe each participant gets their own color?

Here's how VSCode + LiveShare does it:

The cursors should probably disappear if a user is idle for a long time.

[curran]

There's a rabbit hole here that we need to go down which is presence support in JSON1.

My understanding:

• ShareDB does support "presence". It's an opt-in feature that needs to be explicitly enabled. More info here: https://share.github.io/sharedb/presence
• In order to use the ShareDB presence feature, the OT type we're using (in our case JSON1) needs to support its API.
• JSON1 as yet does not have support for presence.
• However, there is an open issue in JSON1 with lots of ideas: Presence #9.
• There is also an open pull request on JSON1 that implements presence: Proposal – Add support for presences #25

Our next step here is probably to try using the work from that open PR. Since it's not merged, the easiest way to do that would probably be to take that branch of JSON1 and copy the files to a subdirectory of our project for initial testing. If it works, We'd probably want to maintain a fork JSON1 that has presence (I'm up for this).

That JSON1 PR has a chance of being merged I think. Our work here may enable that, since it seems to me the main blocker for merging that PR is a working open source example.

[curran]

I created an updated PR in JSON1 with the Presence PR merged with the latest upstream:

ottypes/json1#47

I'm realizing our setup is a bit problematic, because we're using sharedb-client-browser, which bundles JSON1 into it. So swapping out the JSON1 implementation requires some changes there I think. Anyway that should be updated to the latest ShareDB too, I created an issue for that as well vizhub-studio/sharedb-client-browser#2

[curran]

OK this spurred a bunch of activity:

• Vite example share/sharedb#598 - I added a proposed Vite example to the ShareDB project, as motivation for them to provide a browser build.
• Browser build for client share/sharedb#499 - I reopened this old ShareDB issue for providing a nice browser build

But as all that is pending, I'm thinking we go with the following solution to test the JSON1 with presence:

• We continue to use the current browser build of the ShareDB client that we are using currently, which bundles JSON1 (without presence)
• We add the files in Presence updated ottypes/json1#47 in a subdirectory of our src tree
• (here's the key) We rename the OT type so that it doesn't collide with the pre-bundled JSON1 type
• We use that new version in both our client and server
• This will open the door to experimentation with the ShareDB presence API with JSON1
• An example of using the ShareDB presence API can be found here: https://github.com/datavis-tech/json0-presence-demo
• We need to adapt this to work with CodeMirror
• I think the thing to use is "decorations" documented here: https://codemirror.net/examples/decoration/

[curran]

Also dropped a line in the CodeMirror Forum to sanity check the APIs

https://discuss.codemirror.net/t/how-to-show-peers-cursors-on-cm6-collab-editor/3996/6.

Pasting that here:

I’m also interested in doing this. I have collaborative editing working, backed by the OT algorithm of ShareDB, and I would like to add “presence cursors”. ShareDB has a presence API which takes care of the information handling aspect including broadcasting presence and handling cursor transformations.

I’m curious where the best place to start would be in terms of the CodeMirror API. To implement presence cursors, we need the following:

• A way to listen for where the user’s cursor is and get updates when it moves around and when selections are made (presence tracks begin and end of a selection)
• A way to add a “widget” or “decoration” to the CodeMirror editor that is an indication that another person is there.
  The design would look something like this, a screenshot from the VSCode LiveShare plugin:

I’m guessing that for displaying the widget, this is the best place to start:

CodeMirror Decoration Example

Or maybe this one, I’m not sure:

CodeMirror Tooltip Example

In terms of listening for where the cursor is and when selections change, I have no clear idea on where to start. I would guess that we need to create a plugin for this that has a side effect of emitting events when the selection changes. This thread seems relevant to that quest:

https://discuss.codemirror.net/t/codemirror-6-proper-way-to-listen-for-changes/2395/9

which leads to

CodeMirror Reference Manual

which I notice has a property called selectionSet, which it seems should be true when the selection changes (I assume this also includes changes to just the cursor position).

I’ll attempt an implementation based on these ideas and report back if I can get it working! In the mean time, any guidance on the approach or the APIs to best use would be greatly appreciated. Thanks!

[curran]

If we could get a PoC working for any of these subproblems, that would be great progress:

CodeMirror subproblems:

• Listen for cursor/selection changes
• Display a presence widget at a random position

ShareDB subproblems:

• Get access to JSON1 with presence
• Save presence based on listening for cursor/selection changes
• Listen for changes in remote presence using the ShareDB API
• Display a presence widget for each actual remote presence

[curran]

Huge progress today:

Published these NPM packages:

• https://www.npmjs.com/package/ot-json1-presence - a fork of JSON1 with the presence PR merged and name changed
• https://www.npmjs.com/package/sharedb-client-browser - upgraded this to include a build containing the JSON1 with presence fork

Now we should be able to just upgrade the sharedb-client-browser import to start experimenting with the ShareDB presence API.

[curran]

The build was borked and I worked on it. This is the resulting PR: vizhub-studio/sharedb-client-browser#8

[curran]

Whoah it looks like this repo might have a full blown presence implementation for CodeMirror!

https://github.com/yjs/y-codemirror.next

[curran]

Upgraded dependencies of this package

vizhub-studio/codemirror-ot#37

which resulted in a PR against CodeMirror itself

codemirror/view#46