jdev - 2022-09-09

in the MAM Xep there is a stable flag the MAM Service can set

is there any client out there that does anything with that info

i mean the text basically says the results are useless

they can change in sequence and in *content*

i mean how bad can it get lol

basically as a client i would drop all messages from that query, what am i expected to do here?

display the user some content, but tell him, hey this can be all gone and totally changed in a minute ✏

also weird, it only tells me in the <fin> element

means i have to process all messages, only to find out they maybe in the wrong order or content is not correct

or messages are missing etc

ok wait the <fin> element is on each page

so thats fine

so my approach would be, get a page which is unstable, abort the query drop everything, show the user the archive has some problem and gajim will try to fetch messages in a minute again?

show the results in the chat view, but don't save them in the local database? like a "thin client"

why would i show the user possible incomplete content, with possibly wrong order and missing messages inbetween

i consider that worse than "hey there is a problem, wait a minute"

hey i fetched some content, but its incomplete, but here is a little teaser of what awaits you :D

lovetox: I understand your pain, but there are situations where this is needed. Throwing the results away if you don't think your client architecture is capable of handling this is certainly an option.

MattJ, there is no pain, im just in a whining mood :D

It came up recently in the context of the Matrix bridge

anyway, im really interested if *any* client implements that ✏

or does something special with it

On Matrix, rooms are distributed. Messages can happen simultaneously on two disconnected servers. When they finally federate successfully, the history is merged.

That's the extreme example, but the less extreme example happens every day: sometimes it is not disconnected, but just a little slow

It can lead to "interesting" results via the bridge

And it couldn't have used the local received order, with like delay stamps to indicate the time discrepancy?

Then it would have been just like when messages get delayed by stream management or stuff

no Zash, with have no order attribute with MAM

timestamp != order

the messages have to be in the correct sequence delivered from MAM

i dont see a use case other than "drop this request and wait"

except if you are a thin client, without database, i guess it doesnt matter

you display all messages, and a minute later clear the whole view and do the request again

but then again, why would i show an incomplete chat

i see no benefit

wouldn't it be better to show an incomplete chat instead of showing no chat?

not in my opinion

i see everyday how serious people treat chat

like its an absolute katastrophe if a message is missed

I'd rather have an incomplete chat than not being able to talk at all fwiw

im not sure how "talking" factors into that

im not saying i quit the application and let people not talk anymore

what i would do is showing a message, that the archive request failed and we try in a minute again

you free to talk of course :)

but thats maybe a bit of baggage from my work life

lovetox: FWIW I don't think the approach has to be changed, I'd take it as: I see the flag I check more often, I don't see it I just stuff it in the db with less to no invalidation. Deduplication eventually should sort all the rest.

(Because SIDs still have to remain a stable reference to start from) ✏

no deduplication does not fix it, because MAM archive convey order with the order the messages arrive at the client

the only way would be, to mark the messages in the db, then delete all of them on the query that works later

or not write it to the db at all and keep it in cache or something

Tbh I always found this stream order arrival strictness *very stupid* in XMPP

its not a design decision

as i remember it, there is one use case where you can build a cluster of xmpp servers

and have a distributed archive or something

Indeed

and then they didnt want to make the server have some global order mechanism

and now we have that :)

although probably 0,01% uses a xmpp cluster

although i dont really understand how such a cluster can restore the order, if he doesnt know it somehow

and if he knows the order he could also put it into the stanza id

ah i think i remember the cluster doesnt know the order

ejabberd has microseconds as stanza id or something

and they just hope that works good enough

well Matrix has 2 timestamp references, origin_server_ts which equals to the timestamp when the original server sent the event, age which is the time that lapsed in ms from when that event was actually processed (locally)

and that looks to be a rather *convenient* approach to deal with timeline ordering

yeah, timestamp sorting should be enough

if the timestamp has a good resolution

now im not sure if seconds is good enough

and there is nothing that says a server needs to provide a ms timestamp with mam messages

if we got ms resolution, then i guess its a 1 in a million that 2 messages have the same timestamp

and we could further say, those two messages are not a response to each other

so does it really matter in which order they are

this would really simplify the whole thing

i do currently request the messages from my db with timestamp sorted, but with the additional benefit that if timestamps are the same, then insert order in the db counts

and insert order depends on message received order

anyways even if that's controversial because re-ordering the archive indeedly breaks the RFC, I think the case that I hypothesized concerto/fugata possibly experienced is *really* cornerish and missing messages mostly depends on other factors e.g. spam filter or stringprep (which I now fixed) for example more than that

(I never had Gajim jinx me with it, and I do almost all testing with it) ✏