-
pep.
To what <message/> should <origin-id/> be added? All?
-
pep.
Now that is was ruled out that LMC must refer to the original message, does it make sense to include an origin-id tag in it? As it will most likely not be referenced. (Same applies for other payloads I guess)
-
lovetox
pep. you need origin-id not for lmc
-
lovetox
you need it to deduplicate your own mam messages
-
lovetox
hm ah i remember now
-
pep.
hmm, because I don't get reflected messages in 1:1, I keep the origin-id and netx time I fetch MAM I can see where I stopped?✎ -
pep.
hmm, because I don't get reflected messages in 1:1, I keep the origin-id and next time I fetch MAM I can see where I stopped? ✏
-
lovetox
it was because in the MUC xep was a sentence that MUCs dont have to keep the ids that clients set
-
lovetox
so message-id was overwritten by muc service, so you had no chance to deduplicate with it
-
pep.
Well in MUC it wouldn't apply right? As there is stanza-ids? Well, when the MUC does MAM
-
pep.
Otherwise yeah I can use origin-id
-
lovetox
no you are joined in a MUC, you set a message id
-
lovetox
you send the message
-
lovetox
damn, you are right
-
lovetox
i forgot why we added this
-
lovetox
maybe i remember later, im sure there was a reason
-
pep.
Maybe the XEP could be clarified and add a clear motivation
-
lovetox
but yeah that reason is maybe gone, maybe ask Daniel i think he remembers
-
pep.
I have an implementation in slix, I'll wait for feedback a bit more I guess
-
Ge0rG
Too many ids, too much confusion.
-
Lance
origin-id was a way to stamp without leaking your jid
-
Lance
since stanza-id requires a by
-
adityaborikar
Is this site down, or is there anywhere else I could read this white paper https://www.isode.com/whitepapers/xmpp-constrained-bandwidth.html
-
adityaborikar
The link is on https://xmpp.org/about/myths.html
-
mathieui
adityaborikar, https://web.archive.org/web/20160310073118/https://www.isode.com/whitepapers/xmpp-constrained-bandwidth.html
-
adityaborikar
Thanks ๐
-
ralphm
I think they replaced it with https://www.isode.com/whitepapers/low-bandwidth-xmpp.html
-
Zash
Cool URLs don't change!
-
ralphm
But it would sure be nice if they kept URLs working.
-
ralphm
FWIW it seems to be a different article
-
ralphm
As in newer.
-
ralphm
I'm sure Tobias knows.
-
lovetox
pep. i think i remember again
-
lovetox
first everything previous to mam:2 did not inject stanza-id
-
pep.
yeah I assumed there was something something mam and stanza-id
-
lovetox
second many clients used message-ids which were not uuids
-
lovetox
like 1, 2, 3, 4
-
lovetox
so yeah if we live now in a world where most clients use uuids as message-ids, and almost every server uses mam:2
-
lovetox
i think you dont need origin-id
-
larma
message id and origin-id have different uniqueness guarantees. - XEP-0359 origin-id MUST BE globally unique (at least that's the common understanding of XEP-0359), so if everyone plays by the rules it is globally unique, otherwise it's unique per user if at least that user plays by the rules. - RFC 6120 stanza id attribute on messages can be unique within that stream or globally unique (per RFC 6120 ยง 8.1.3). There is no MUST in there, so one may argue that no uniquness guarantee is provided, but explicitly no global uniqueness. Interestingly in XEP-0045 v1.31+, there is a rule that MUCs SHOULD reflect the original message stanza id on reflected messages, which implies that there is not even a by-stream uniqueness guarantee on the message id as other MUC occupants can reuse prior message ids and thus cause a collision of message stanza id on a stream (and still be fully standards compliant).
-
larma
Technically there is no uniqueness guarantees on <origin-id> whatsoever, only <stanza-id> are unique per generating entity (as specified in their by attribute)
-
lovetox
yes larma, but all you need is unique within an archive
-
lovetox
and this is guaranted by mam
-
larma
for mam that's guaranteed through <stanza-id>, not message id or <origin-id>
-
larma
but there are many other use cases that use a message id (or origin/stanza id)
-
lovetox
yeah but they are not as important as message deduplication
-
lovetox
so i always thought if a client does not use unique ids, then this feature will just not work good
-
larma
LMC changing the wrong message?
-
lovetox
yes, of course you can implement LMC badly, then this results in very bad stuff
-
lovetox
but normally you have a timeframe where correction is allowed
-
lovetox
and even if you decide not, then you should always give the user the chance to see all corrections and original messages
-
lovetox
so worst is, the user sees weird stuff
-
lovetox
but most clients use uuids
-
lovetox
and if you implement some timeframe, like message is only allowed to be corrected within 5 minutes
-
lovetox
the chance of weird stuff happening is almost zero
-
lovetox
even with clients who dont use uuids
-
larma
I feel I don't agree with most of what you wrote...
-
larma
"user sees weird stuff" is literally the worst thing that can happen, after all MAM is only one utility to help not doing this
-
larma
"message is only allowed to be corrected within 5 minutes" I'd prefer that LMC was not only for the last message and also allow very old messages (up to days are acceptable for me)
-
larma
And even if you allow the user to display corrections, usually users won't do that anyway...
-
lovetox
hey i like it to, but this would depend on a uniqunes guarantee that can never exist
-
lovetox
you can always only trust the other client
-
lovetox
and if you trust message-id
-
lovetox
or some other made up element
-
lovetox
i mean if you feel better, trust origin-id :)
-
lovetox
i think this is the downside of a federated protocol
-
lovetox
make up for it by reducing the damage wrong IDs could make
-
lovetox
1. timeframe, 2. give user the chance to deactivate lmc for bad contacts
-
lovetox
3. give full transparancy, dont hide or replace messages
-
lovetox
thats how i deal with this
-
lovetox
if you got a better idea, im very interested
-
larma
origin-id should be unique at least by origin, easiest way to guarantee is to use a proper UUIDv4. If an attacker uses the same origin-id for multiple messages, the later ones can just be ignored completely, nothing to worry about as it is not possible to happen if it's not an attack and attackers are allowed to suffer from their wrongdoings.
-
larma
But, you can't do the same for message ids as they are not unique by origin, so duplicate message ids are allowed to happen
-
lovetox
im not following, so you plan that i can correct ANY message i ever sent you whenever i want
-
lovetox
what is there to attack?!
-
lovetox
i can already replace any message
-
lovetox
and just in case we are not talking about the same thing, IDs should always every matched in the database with a (jid, id) tuple
-
lovetox
there is zero abuse potential other then your contact abuses his own messages he sent you, in that case just stop talking with this dude
-
larma
Maybe the name "attacker" was misleading here, I meant an entity that does not correctly implement origin-id and generates the same origin-id twice. Those would suffer from degraded usability (easiest is to ignore the messages with duplicated origin-id). However a user sending the same message id twice should not suffer from degraded usability (i.e. the message should be displayed as usual) as this is perfectly standards compliant behavior
-
lovetox
are we still talking about LMC?
-
lovetox
if someone sends 2 messages with the same id, and afterwards a correction for that id
-
lovetox
just take the more recent one
-
lovetox
i dont need origin-id for that and i dont need to ignore a message
-
larma
There is no reason why message correction should always reference the latest instance of a message id, the user might have intended to do something different. Message correction might be a bad example as those always originate from the sender and usually the same client (though this is not necessarily the case). However if you pick other usages of ids, you'll even get more problems. Like message attaching might attach to the wrong message or something
-
larma
If you use the message id for message attaching (as suggested as a fallback when there is no origin-id in the XEP) you have the risk of this happening, as a client might not be able to know where to attach that message. So best would be to not allow attaching to messages that don't have an origin-id. But now apparently everyone for some reason tries to not implement origin-id
-
lovetox
the reason is easy, there are already 2 other ids we manage
-
lovetox
so if we need a third one, it should be a damn good reason
-
larma
There is your reason ๐
-
larma
And: if you don't want to support it on the receving end, at least support it as a sender for those clients that want to use it on the receiving end
-
larma
If your message id is already generated using UUIDv4, you'd just have to send the origin-id as well and be done with all the work on the sending side (I assume you already store your message id, so you will be able to handle any usages of the origin-id automatically if it is the same)
-
lovetox
the scenarios you think up seem unrealistic
-
lovetox
you talk to a contact, that has a client who sends the same message id over and over
-
lovetox
but for some reason its still so advanced that it supports message attaching
-
lovetox
so the client should now add origin-id instead of just making his message-id a uuid
-
lovetox
why cant the XEP not just say, clients who want to support message attaching should generate unique message-id
-
larma
The client sending the message and the client being able to display the message attaching must not necessarily be the same
-
pep.
lovetox, yeah I guess that would have the same effect? (mandating that clients generate unique @id)
-
larma
How do you mandate legacy clients?
-
larma
They are already out there, we can't change it afterwards.
-
pep.
If they want to implement XEP-foobar, this XEP can say "if you implement me you need to do X"
-
pep.
In practice it's a similar solution to origin-id
-
pep.
I guess XEP-foobar could say "you need to implement origin-id"
-
lovetox
larma, i dont have a problem with legacy clients having degraded experience
-
pep.
Same here
-
lovetox
but yeah message attaching is a bit unique, i have to look into it more
-
lovetox
LMC is really not a problem at all, with clients that use the same ids over and over
-
lovetox
i wonder how i would display message attaching on the sender side
-
lovetox
would i also only link in my own database to the id
-
lovetox
or would i copy the message im attaching and adding it in some other database column
-
lovetox
i think i would just copy the message into some "attached_data" column
-
lovetox
that way i dont even have to care about if the other client uses unique ids
-
lovetox
if he uses unique ids he sees the correct stuff, if not, he doesnt implement message attaching anyway and will not see it
-
larma
OK, completely realistic scenario: - On my desktop I am using some legacy client that does reuse message ids and does not support message attaching - On my phone I use a modern client that does support message attaching I send a message from my desktop, the recipient attaches a message to my message, I send another message from my desktop that happenns to have the same message id. I open my phone, the message attachment is attached to the wrong message. Even if the XEP of message attaching would require a unique message id, that would not apply to the legacy client, because it doesn't implement the XEP and the recipient does not know if the message id is unique, so can only guess. If the XEP would require origin-id, it would be missing from the legacy client and thus the recipient would be unable to attach to their message
-
lovetox
larma i agree origin-id would solve this case
-
larma
We could also replace origin-id by an empty <origin-id /> to signal that the message id was unique, but we need this information to be transported in the message.
-
lovetox
just upgrading your desktop client would also
-
lovetox
hm actually i like that idea, to add an empty element
-
larma
The desktop client is allowed to not send a unique id as long as they don't implement a XEP that does require them to. And I am pretty certain there are still a lot of clients that don't do unique ids
-
larma
Maybe we want to add this as on option to XEP-0359
-
larma
Make 'id' optional on <origin-id> if the id of the surrounding stanza already provides the uniqueness guarantees for <origin-id>
-
Lance
empty element would reintroduce the problem the origin-id is there to mitigate: MUCs rewriting the message ids. you'd lose the original id iagain n that case
-
lovetox
Lance, but this was fixed in the MUC XEP
-
larma
Lance, true for MUCs, but for non-MUCs the 'id' could still be optional
-
larma
lovetox, it's an optional feature of MUCs
-
larma
to keep the ID when reflecting
-
Lance
let me know when the jdev follows the latest MUC XEP✎ -
Lance
let me know when the jdev room follows the latest MUC XEP ✏
-
Lance
unfortunately :/
-
lovetox
Im not sure what your argument is, we have to stay compatible with old outdated software only because it is compliant with some XEP
-
larma
Introducing new requirements in a MUC is not really possible, there will always be legacy servers, only way would be to not connect to those anymore
-
lovetox
i couldnt care less about jabber.org
-
lovetox
if users tell me Gajim doesnt work with jabber.org, i tell them, go use another server
-
lovetox
not rewrite Gajim
-
larma
lovetox, > The service SHOULD reflect the message with the same 'id' that was generated by the client, to allow clients to track their outbound messages. If the client did not provide an 'id', the server MAY generate an 'id' and use it for all reflections of the same message (e.g. using a UUID as defined in RFC 4122 [18]).
-
lovetox
im fine with indicating with origin-id that a client uses a unique id
-
lovetox
i think thats a fine use for origin-id
-
lovetox
if you want to change your behavior on that, fine, i think would not have the motiviation
-
lovetox
and fyi Gajim sets origin-id since forever :)
-
larma
I wasn't talking about Gajim, I was talking about you suggesting pep. not to do it
-
lovetox
yes, message attaching is to new, i didnt look into it
-
lovetox
i stand by my opinion that for all other XEPs, like LMC you dont really need this
-
lovetox
and i think, all clients who support mam, support unique ids
-
lovetox
and you would never use a client without mam in a multi client setup
-
larma
You don't need it for LMC, but need it as soon as it becomes MC without L
-
lovetox
so yeah i agree there are edge cases like the one you told, but im not sure we should put to much effort into supporting these
-
lovetox
but adding <origin-id/> to indicate unique ids, is not that much effort :)
-
larma
I doubt pep. intention was to actually use a different ID for message @id and <origin-id>
-
lovetox
what i really would like to prevent is adding another db column "origin-id"
-
larma
I understand your point from a dev perspective ๐
-
lovetox
omg, jabber.ru does not send the muc subject on join
-
lovetox
if no subject is set
-
lovetox
do i interpret this correctly
-
lovetox
https://xmpp.org/extensions/xep-0045.html#order
-
lovetox
that a room subject MUST be sent, even if there is none?
-
Zash
correct
-
lovetox
i guess users will not join mucs without subject anymore with the next Gajim version :D
-
Zash
Prosody didn't send empty subjects in some earlier version but it's been fixed for some time. Not sure about other implementations.
-
eevvoor
Why does jabber.ru not send it? Is is due to the server they use? I thought it is a client issue, so how does it affect jabber.ru lovetox?
-
lovetox
i think jabber.ru uses some very old ejabberd version
-
lovetox
so they dont profit from bugfixes
-
lovetox
eevvoor, the MUC xep mandates that a subject has to be sent, and gajim waits for it
-
lovetox
and if it never comes, the joining process is never completed
-
ralphm
That's silly. I've seen Gajim have more issues like that.
-
eevvoor
ah thus not the server sw is the prob but the missing updates :D. Reminds me of the so beloved ccc server. Badly administrated ...
-
lovetox
ralphm, whats silly about it?
-
lovetox
should we expect servers not following xeps now or what
-
eevvoor
yes lovetox you should support backcompatibility forever!!!elven111! XD
-
ralphm
lovetox: I agree servers must send it, but waiting on it, even if other stuff for the room comes in, seems silly
-
lovetox
so i guess self presence is has 110 status code is also silly?
-
lovetox
i mean if there is other stuff coming in, no need for that
-
lovetox
subject is the line between history messages, and live messages
-
lovetox
its the order of events, it indicates that history messages is complete and the join is full completed
-
lovetox
that means i can tell the user, now you can send messages
-
ralphm
I understand what it is for
-
lovetox
and not in between a history message fetch
-
eevvoor
so it carries kin of semantics, lovetox, in your opinion.✎ -
eevvoor
so it carries kind of semantics, lovetox, in your opinion. ✏
-
ralphm
What I don't understand is that it stays in limbo forever
-
eevvoor
But of course it is nice if the client is robust and works well if the xep is not met, ralphm ?
-
eevvoor
So to avoid limbo?
-
lovetox
ralphm, of course i could add a timeout
-
lovetox
but all clients have to add this workaround
-
eevvoor
also not nice, in case it is just delayed ...
-
lovetox
or one server can just send a subject
-
lovetox
...
-
eevvoor
many combinations to be considered ...
-
ralphm
lovetox: this is what I mean. A timeout would take care of not keeping the user in limbo in the face of broken servers
-
eevvoor
Hm but timeouts can result in strange bugs.
-
ralphm
Another example: when Slack still had an XMPP gateway, it didn't respond to the iq for private storage. This would also hang Gajim indefinitely.
-
ralphm
I manually patched Gajim to work around this.
-
eevvoor
dining philosophers ...
-
lovetox
working around servers that dont answer IQs, ok
-
ralphm
Why? That's a MUST too
-
lovetox
but you still accept it
-
lovetox
and worked around it
-
lovetox
i mean answering IQs is one of the most basic rules in all of XMPP
-
lovetox
if we cant depend on that anymore ..
-
ralphm
I am more of the 'expect failure' variety.
-
lovetox
its a fine line
-
ralphm
And this example was extra tricky, because it is part of the Gajim connect sequence, not a random iq that is sent amongst other things.
-
ralphm
Of course I also reported it as a serious bug with Slack.
-
ralphm
And not getting a response for private storage shouldn't block the UI to look like the connection is not ready.
-
ralphm
So a timeout would have been a better approach, or async handling.
-
lovetox
yes i agree, its not anymore btw, requesting bookmarks is now independent of connection process
-
lovetox
only server disco info, roster, and roster delimiter iqs are now in the connection process
-
ralphm
Good