XSF Discussion - 2023-03-17

Since we're in XEP-0060 bug-fixing mode, there are a couple of other outstanding issues I'd like to fix

Well, at least one of them just requires some informational text I think

I'd like to have a section that just explains the data model. It took me years to realize it's an ordered key-value store. Once that became clear, everything started to make more sense.

The second problem is that the spec isn't clear enough about whether publishing an item with the same id as an existing item modifies the order or not

I brought it up on the list here: https://mail.jabber.org/pipermail/standards/2018-August/035311.html and the final word was here: https://mail.jabber.org/pipermail/standards/2019-January/035606.html

So I think we need to get this conclusion into the XEP

XEP-0413 solves the issue mostly for clients, but once we start talking about what happens to items when the node is full, it becomes important

Yes, ordering of items is unclear, and I think Movim (and probably Libervia) have been bitten by this before

Yes, also why I explicitely said "creation date", but I'm not exactly sure what to do here

In your mail you use the wording "oldest", but we need to specify what it means first then

Ah, I'd overlooked that detail

As far as I'm concerned, it already has a meaning that is implied throughout the XEP but unfortunately never explicitly stated

ralphm (author with most implementation experience) seems to agree with this

That does of course mean that XEP-0060 doesn't currently suit cases where you want to be able to modify items without affecting the order

Would be great to "clarify" ✏

Unless you ignore that and just use XEP-0413

Wait I missed the implied meaning of "oldest" then?

That's by creation date?

Which works as long as you always use it, and never engage the server's "expire oldest" functionality or whatever

First: don't say creation date, nothing about pubsub ordering is to do with dates/times :)

The order is the order in which items are published

That the only operations XEP-0060 really has: publishing and retracting

That's.. more or less what I meant. There's always a date even if it's not explicited :p

Ok

Well, if you look in Prosody's data store you won't find one

or maybe you will these days, I'm not sure

But even if it's there, we don't use it for ordering

and it would be the timestamp of when the item was published

Yes. Ok so that's the only ordering that's assumed in 0060. It'd be great to explicit it yeah.

And then clearup what happens when an item with an existing ID is published

Yeah, I think once we get the data model clear, it becomes easier to talk about how to make different use-cases fit

Right now we have implementations that accidentally differ in how XEP-0060 ordering works, and that just makes everything harder because we're starting from different places

So you're saying updates re-order the new item to the front? I guess that makes sense since anyone doing a catch-up fetch needs to get it to be aware of the update. Just means really REALLY no one should ever render a UI based on the returned order without locally sorting

That depends on what the UI is for, but many cases will require local sorting or XEP-0413 (server sorting) ✏

For a blog, for example, you don't want to bring a post to the front just because you fixed a typo

Order of items should be part of the client request. Different clients & use cases have different requirements and need of order. As has been mentioned already: publishing time, when ordering a timeline, or last update time, when reviewing what has changes since last. (If parameter omitted in request, a default value should be specified) ✏

Another option, instead of XEP-0413, would be to pursue per-node configuration of the server-side order

I don't think local sorting is a huge burden. We already generally need to do it even for chat messages due to delay etc, which has this same semantic in MAM

Well, whether it's a huge burden partly depends on the number of items in the node

Even for a blog, you do want subscribers to become aware of the edit, so bring to front is good. So long as they know to sort in client

Thanks för reference. Schema in XEP-0413 needs enumeration for possible order by values.

I assume that ejabberd (which does not bring items to the front) still sends notifications to subscribers

Yes, but pubsub notifications are not reliable

And not stored in MAM.

On purpose because you can always just fetch latest items to catch up

Then it depends whether you have a thin client or not

Yes. Thin client scenario is what 0413 is for I guess

A thin client rendering the blog would fetch items on demand and they'd just be up to date, but a client that synchronizes to a local store would indeed need to catch every change

Test

TOAST !