How do we add an item after/before another item or at a specified index of the collection?

[niieani]

Imagine a chat app which implements offline support:

• you can "send" messages and they're added to the queue and shown immediately (optimistic updates)
• when the client reconnects, it pulls messages that were sent by others: the Date of their messages can thus be before the messages the user has sent out
• we need to place those messages at their appropriate places in the list (e.g. in the middle), not append them.

This is just one use case, but there are many others (like reordering lists, where you'd need to remove an item and re-add it in a different place).
I'd imagine a lot of the times you don't know the index and you'd actually need to find after/before which specific item you want your new item to appear with some sort of a search function.

Not sure how a pure/functional API would look like for this case, but if it were a classic mutable state, it'd be something like:

collection.addAfter(newItem, item => item.date < newItem.date)

This would iterate on all the elements until the return value of the search is false, then insert the new item at the index of the first item that returned false.

How can we deal with this case using cycle-collection?

[Hypnosphi]

I think that allowing users to set the item's id manually would really help here.

What you can do now are some hacks like reimplementing the Collection.Pluck method to sort the items before rendering:

function pluckWithSorting (collection$, sinkProperty, sortProperty) {
  const sinks = {};

  function sink$ (item) {
    const key = `${item.name}.${item.id}.${sinkProperty}`;

    if (sinks[key] === undefined) {
      if (sinkProperty === 'DOM') {
        sinks[key] = item[sinkProperty].map(vtree => ({...vtree, key})).remember();
      } else {
        sinks[key] = item[sinkProperty].remember();
      }
    }

    return sinks[key];
  }

  return collection$
    .map(collection => collection.asArray()
        .sort(fst, snd => fst[sortProperty] - snd[sortProperty])
        .map(item => sink$(item)))
    .map(sinkStreams => xs.combine((...items) => items, ...sinkStreams))
    .flatten()
    .startWith([]);

But this shouldn't be really performant as your collection size grows. Alternatively, you can use flexbox with CSS order property for visual rearrangement.

[niieani]

@Hypnosphi Unfortunately flexbox ordering would be a hack not a solution - it would work only if the output is the DOM. If we'd like to use collections as sinks for other drivers, like Canvas, SVG, Audio or anything else for that matter, there usually won't be any equivalent hack available.
And as you say, sorting as a post-processing of the stream will incur bad performance, as it would be invoked every time the collection changes. Its better to do it once, as pre-processing of the stream, before adding the new item.

[Hypnosphi]

Btw, unrelated to this repository, API like collection.addAfter(newItem, compareFunction) could as well be a pure/non-mutating inside:

  addAfter(newItem, compareFunction) {
    let index = this.findIndex(item => !compareFunction(item, newItem));
    return this.slice().splice(index, 0, newItem);
  }

[Widdershin]

I have a weird thought. It's a common pattern that you want to apply a sort order to a collection. It's one thing to manually maintain a sort order by inserting at indexes, but what if we provided a collection.sort(compareFunction) that returned the collection sorted for pluck and asArray().

Now the trick part is that often you want to sort on an observable, right? I think the default case should be a reactive sort. So the signature would actually look like collection.sort(sinkName, compareFunction). We could also support sorting on multiple sinks.

What do y'all think?

[Hypnosphi]

This would be post-processing again. What niieani wants is to have some kind of addSorted(compareFunction), but I think it would be even better to have Collection(Item, sources, compareFunction) and always insert items in an order. In that case we can be more effective by using binary search, as we know that collection is always sorted.

[niieani]

@Hypnosphi this indeed makes a lot of sense! Passing a sorting function to the collection definition would ensure the collection is always sorted - there's something so lovably "functional" about that notion. I suppose this will be the most common case.

@Widdershin's proposition seems good too, for other use-cases - it could be that we want to sort a list "onClick" - or even to sort it differently on different observables (e.g. sortable DataGrid / Excel). So both features would be welcome additions.

[Widdershin]

It's a common use case that a user wants to be able to change the sorting of a collection after creation. If we use collection.sort(propertyName, compareFunction) then it can be changed inside of a fold. It also means you can sort the same collection in different ways and have them act as a view.

My problem with the non-reactive approach is what are you actually sorting on? If you're sorting on the properties of your items, it's convention for those to be Observable. And then the return value from that compareFunction would be observable. And since it would be nice to access the actual value in the sink, we want something like pluck with a compareFunction on top.

As for the whole optimization thing, I wonder if we can cross that bridge when we come to it. Who knows how fast xstream works out to be in practice? This might be a good test.

[niieani]

@Widdershin Are you saying that static data which comes from the server should be, as a convention, converted to Observables of Observables? Why? Is there any benefit in doing so if any updates to the data come in the form of new objects altogether?
I get that xstream might be fast, but that's still not the point - and that is: redundant computation. To me, this is just something that goes against the whole philosophy of functional programming, or Cycle.js.
Thinking of workarounds to the problem instead of real solutions is the reason traditional state management techniques are the mess they are to work with. I really hope there's an elegant way to solve this.

[Hypnosphi]

Items in a collection must have some static key anyway. Currently it's auto-generated but it could be provided by user as well. It's sorting by key what can be done instantly. All the sortings by dynamic properties should be done on the view stage, off course.

[Hypnosphi]

Is there any benefit in doing so if any updates to the data come in the form of new objects altogether?

You just turn the sequence of each item's states into a separate stream by mapping over a responses stream. You should do that if you want reactive updates.

[Widdershin]

All the sortings by dynamic properties should be done on the view stage, off course.

Totally with you now, all "reactive" sorting should be done on top of pluck.

I'm still not clear what exactly we would be sorting on for the sort on addition style.

Could someone provide a real world example?

I'm fine with the notion of providing a compareFunction when creating the collection, but as far I can see that would require the user to provide a key when adding an item. Is that what y'all are after?

[Hypnosphi]

Yes, it is. In the original case of messages timestamp would be the key. It's safe, as it doesn't seem to change in the future. You may be able to remove or even edit your messages, but not to change the sending time =)

[Widdershin]

@niieani: have you had the chance to try out Collection.gather? Does it solve your use case?