XSF Discussion - 2017-09-21

Zash, mhm thanks

not sure what you expect me to do with it though

jonasw: me neither

okay :)

then I’m just going to ignore that for now

Is there any way to mark a message as related to some other message, without specifying the exact type of relation? We have 0372 for explicit references, <thread>s, LMC, but they all imply a certain relationship

what is the use-case?

jonasw: MAM - when a given message is asked for, it is useful to provide all the related messages as well.

I’m not convinced

at least not as default behaviour

jonasw: some messages only make sense on top of others, e.g. reactions or LMC

sure

jonasw: it would be impractical to deliver only the addon message but not the original one

for reactions, I’d simply omit those

(in the view that is)

jonasw: if your client receives a reaction message with a dangling reference, you need to store the XML and process it later when the actual message is obtained.

yes.

I don’t necessarily see an issue with that

while handling out-of-order messages is more annoying

the new CSS breaks horizontal scrolling on some XEPs https://xmpp.org/extensions/xep-0385.html

scrolling works, but indeed it’s an issue that it’s required

edhelas, can you make an issue on the xeps repository?

385 is just too wide.

jonasw done

edhelas, thanks

Ge0rG, we can’t go over all XEPs to fix that

jonasw: we could define a common mechanism to cross-reference messages and use that?

well, we could, but ...

but.

Ge0rG, sure, yet-another-standard? :)

jonasw: stop spoiling the fun!

it's possible in CSS to break lines properly

edhelas, auto-breaking of lines in code examples sounds like a bad idea

I also have a question

Ge0rG, you’d want to rebase the existing use-cases (LMC etc.) on that, and especially with LMC I see a lot of breakage from that...

I'm planning to use MAM to retrieve MUC histories, is it possible to disable the default messages pushed by the server on login ?

edhelas: yes

oh ?

edhelas, https://xmpp.org/extensions/xep-0045.html#enter-managehistory

maxchars='0' is one example

(I’d set maxchars='0' *and* maxstanzas='0' to be sure; I guess not all services support maxchars)

oh, so I should explicitely disable them

thanks jonasw :)

np :)

jonasw: I think I only set maxchars and I never had problems

Ge0rG: https://xmpp.org/extensions/xep-0367.html

That was meant for things like that

daniel, "never", not even biboumi? ;)

Even though maxchar is a very weird metric now that I think about it. I guess back in the day is just copy pasted that from somewhere

This will probably break if you set something other than 0

agreed

jonasw: didn't know that biboumi was a problem. Never got reported to me

SamWhited: is there a reason for using message ID and not unique ID?

odd, I had something about that in the back of my head. maybe it was about MAM instead

jonasw: yes. Mam did break the stanza queue for sm

Ge0rG: no, I don't think anyone supports unique ID

XEP-0045 is full of quirks

But it doesn't matter to me

I'd really love to fix message IDs instead of inventing new unique IDs. But backward compatibility!1!

Oh, am I mixing those up? Whichever. I would be fine coupling it to the unique stable message ID thing, whichever that is

XEP-0359 (<https://xmpp.org/extensions/xep-0359.html>)

Right, that.

Yah, in that case the real reason this doesn't support it is that HipChat doesn't support it and I was never going to be able to convince them to let me ad even more stuff.

Ge0rG: Do you think you'd want to use it for something? If so I wouldn't mind updating it to use the unique ID stuff

SamWhited: some weeks ago, the topic of grouping messages when delivering MAM data came up here

just curious, how do you know messages are related?

moparisthebest: that's the challenge

like these might be in response to your stuff, in fact they are, but they might just be a random unrelated topic

Oh, this is for grouping messages sent in reply to another message? That seems different to me

that seems like an unsolvable challenge from an automated perspective, so a xep to support the grouping seems useless

actually, maybe not…

SamWhited: it's about finding out which messages are related to each other

I agree with half of what moparisthebest said; it seems unsolvable if you're trying to automate it. That doesn't mean you can't have a good UI that leads the user to do the work for you.

I don't think you can and that's why it just returns the last X or whatever

SamWhited, if I respond to you like this I guess you could call those grouped (by putting your name first)

the relation is already spelled out in the XML (LMC, Reactions...)

there’s just no unified way to detect it

(it’s, from my understanding, not about generic replies)

moparisthebest: That seems like a sensible guess

This seems more like what <thread> is for though

SamWhited: I'm not speaking about message that have content that relates to other messages

SamWhited: I'm speaking about technical dependencies, like "this message is a reaction to message XY"

so it doesn't make sense to parse / display the message without also having XY

oh yes, that's what I thought you meant originally, yah, that's totally what this is for

I think… you mean things like clients having a +1 button or a party hat button or something?

so it would make sense, in the context of MAM, to use unique IDs

SamWhited, yes

SamWhited: yeah

Cool, yah, moparisthebest was confusing me :)

That is indeed what this was made for

so let’s rebase xep-0367 on stable & unique message IDs and build a reactions XEP on top of that

jonasw: +1

Except...

you want me to do the work, right? ;P

You still don't know the unique message ID of outgoing messages.

I don't remember the MID spec, but it allows clients to assign one too, right?

SamWhited, indeed

¯\_(ツ)_/¯ problem solved

so called origin-id

But MAM operates on the server-assigned ID

That seems fine; MAM operates on server assigned ID, this operates on client assigned ones.

it would be convenient if the client could query MAM for the message if MAM doesn’t support grouping

(the proposed grouping that is)

alternatively, we could modify MAM to adopt the client-generated origin-id for sent messages iff it doesn’t conflict with any existing ID (and maybe reject delivery otherwise or something)

I'm sure somebody will pull the "omg security!1!" card on that.

Ge0rG‎: "this message is a reaction to message XY" -- how do you know that though?

moparisthebest, reaction as in "+1 button"

not as I react to your message right now

Ge0rG, how?

(and isn’t that usually your job?)

he said 'this message is a reaction' not 'I hit a +1' (is +1 a thing in xmpp or any clients?)

not yet, that’s the point

and those "+1 buttons" are usually called reactions

e.g. on github

jonasw: let's see... if the server is using a typical hash-table implementation, a client could DoS it by contructing specially-chosen UUIDs.

Ge0rG, meeeh :P

just don’t tell clients your hash seed?

This one! https://arstechnica.com/information-technology/2011/12/huge-portions-of-web-vulnerable-to-hashing-denial-of-service-attack/

that’s a general vector, and afaict choosing a random hash seed is sufficient to fix that

jonasw: I'm sure it's possible to extract the seed via side-channel attacks

Ge0rG, *sigh*

then we’ll have to operate on the client-generated ID. it’s saner anyways, because it works without MAM

MAM will have to interpret these IDs then and intrenally store their relatedness for queries

what is the correct request to get MAM history from MUC ?

edhelas, ask the MUC for history?

<iq xmlns="jabber:client" type="set" id="S8lR4X"><query xmlns="urn:xmpp:mam:1"><x xmlns="jabber:x:data" type="submit"><field var="FORM_TYPE"><value>urn:xmpp:mam:1</value></field><field var="with"><value>muc@conference.movim.eu</value></field></x><set xmlns="http://jabber.org/protocol/rsm"><max>300</max></set></query></iq>

(I don’t know of any non-biboumi MUCs which support that)

looks like I'm wrong somewhere

edhelas, also, this may be better suited in jdev@

jonasw: yeah. Because it's a good thing to have three different IDs on each message.

a true

sory

Ge0rG, m(

why exactly is a mam query a type='set' operation?!

oh, wait. those IDs are all insufficient in some ways. Let's create a new XEP with a fourth, universal ID!

Ge0rG, any other suggestion? other than getting a sent-carbon for your own message with the MAM ID >.>

hmm, I guess querying MAM for "a message and all of its reactions" would be sort of nice. I'm not sure if it's strictly necessary though.

jonasw: that was exactly what I proposed back then.

It seems to add another dimension of complication for something that could easily be lived without.

SamWhited: the problem is not "a message and all of its reactions" but rather "the message and all the messages you need to properly understand it"

Why would you ever actually need to do that? I don't dislike the idea of making sure that's queryable, it just doesn't seem like a common enough thing to worry about.

SamWhited: if you receive a reaction message with an unresolved reference ID, you need to delay processing until you know what it is a reaction to

I suspect 90% of people will just sync history and you'll see new reactions appearing as history gets updated

That seems fine

I tend to agree

I tend to process messages when they are received, but that doesn't work with missing references.

the argument Ge0rG makes is in the spirit of the "simple clients, complex servers" principle though

Does MAM not guarantee delivery order?

Oh, nevermind, stupid question

ok :)

If you have a reaction and *then* you sync history it doesn't matter what MAM does.

SamWhited: it only guarantees the order for the subset you request.

yup, yup

So in that case though you end up having to make a MAM query, then for each message make another query asking if there are any reactions you haven't received yet

if there are any messages for a given reaction.

SamWhited, no, the idea would be that the original MAM query would give you all messages to which there have been reactions in the result setd

(even though they may be outside the range of messages you requested)

jonasw: but that would violate message order, and lead to interesting results.

This seems like a lot of additional complexity when you could just store your reactions somewhere on the client and when you get a message if you have reactions in storage already display them.

Or when you get a reaction, if you have the message its linked to, also display it.

example: you request 50 messages before #103. you get #53-#102 and #40 because it was reacted to. Next, you request 50 messages before #40, because it is the earliest in your history. #41-#52 are lost

Ge0rG, all depends on the sanity of your implementation

SamWhited: that makes (limited) sense if you store the original message XML somewhere

Ge0rG, do you really need the XML? I don’t think so.

Ge0rG: I don't think the storage format matters

SamWhited: but then you end up searching your storage for a reference to each new message you receive.

Ge0rG: That seems fine

I pull the "use a proper storage backend" card

There is no client DoS card.

e.g. a simple sqlite database

just add a column "reaction_to" with a stable message ID thing

add an index to it

-> reasonable lookup time

you also only need to do that for messages you fetch from the archive

Flat XML files are probably fine, you are going to get a bunch of stuff to store, that's just how XMPP works. If you're worried about scans and lookup times, its your job to do something intelligent and make sure you don't dos yourself.

Maybe that means creating an "unknown references" file to make scanning quicker, maybe it means using a database. That's up to you.

I wish for the times when XMPP servers were complicated and XMPP clients were simple.

I agree with that in general, but this doesn't make the XMPP client complicated; it makes it a dumb display layer that doesn't have to have a ton of logic about doing crazy MAM queries.

It can just keep doing whatever it was already doing and process things as they come in. It might be processing the reaction, or it might be processing the message in terms of reactions it already has.

do clients generally do mam queries reversed like that?

conversations doesn't seem to, it seems to start loading from your last message and go forward

in which case everything *just works*

moparisthebest: I think there are two models of MAM usage: (a) query for everything since your last connection, (b) query a given channel / user history on demand when opening the tab

oh I was waiting for b until I just realized gajim made that into a beer...

"Clients and servers MUST NOT include an <attach-to/> element on messages with a non-messaging payload unless they are including it on an error which may be attached to the message that caused the error to be generated." What is a "non-messaging payload"?

No idea, no <body> I guess? That should probably be fixed.

Or just removed, I don't see the point in that restriction anymore.

SamWhited: for reactions, I'd go with a separate non-<body> element, because people in here demanded invisibility to incompatible clients

And because it will be awesome to have to change all the implementations that filter out body-less messages as "unimportant".

yah, just removing those lines from message attatching seems reasonable

muc.xmpp.org doesn't supports mam:1/mam:2 yet ?

No

-version xmpp.org

Zash: xmpp.org is running Prosody version 0.9.12 on Linux

-version muc.xmpp.org

moparisthebest: muc.xmpp.org doesn't reply to version requests

yeah that seems to be the problem

I don't see caps for muc.xmpp.org in my db

Yea so we don't actually know what it's running :)

<identity type='text' name='Prosody Chatrooms' category='conference'/>

Impossible to know

total mystery

It could be lying

Never trust a remote entity