Basically it is a link list, but there were one or two articles that slipped my eyes
rtq3has left
rtq3has joined
jubalhhas joined
Lancehas joined
Dave Cridlandhas left
Dave Cridlandhas left
xnyhpshas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
Dave Cridlandhas left
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
Dave Cridlandhas left
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
dwdhas left
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
xnyhpshas left
tim@boese-ban.dehas joined
Dave Cridlandhas left
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
Dave Cridlandhas left
xnyhpshas joined
andyhas left
Dave Cridlandhas left
rionhas left
SaltyBoneshas left
dwdhas left
Kevhas joined
dwdhas left
Guushas left
ralphmhas joined
Marandahas joined
@Alacerhas left
Dave Cridlandhas left
rionhas left
pep.has joined
j.rhas left
j.rhas joined
Dave Cridlandhas left
Steve Killehas left
Steve Killehas left
Dave Cridlandhas left
dwdhas left
remkohas joined
@Alacerhas joined
Dave Cridlandhas left
Dave Cridlandhas left
andyhas joined
matlaghas joined
Lancehas left
danielhas left
dwdhas left
stefandxmhas left
andyhas left
Steve Killehas joined
jubalhhas joined
Dave Cridlandhas left
dwdhas left
vanitasvitaehas left
vanitasvitaehas joined
j.rhas joined
j.rhas joined
ralphmhas left
danielhas left
Alexhas joined
@Alacerhas left
Lancehas joined
rtq3has left
Dave Cridlandhas left
rtq3has joined
dwdhas left
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
Alexhas left
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
Dave Cridlandhas left
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
dwdhas left
tim@boese-ban.dehas joined
@Alacerhas joined
SaltyBoneshas left
rionhas left
@Alacerhas left
jubalhhas joined
jubalhhas left
lskdjfhas joined
la|r|mahas joined
ralphmhas left
la|r|mahas joined
la|r|mahas joined
ludohas left
ludohas joined
Ge0rG
I love it how xmpp.is say they won't support the spam fighting manifesto, and then describe how they essentially implement each of the stated requirements <https://xmpp.is/2018/02/21/the-jabber-spam-fighting-manifesto/>
moparisthebesthas joined
rionhas left
jubalhhas joined
moparisthebesthas joined
Dave Cridlandhas left
Maranda
Ge0rG, infact xmpp.is is one of the servers I get spim hits from atm, *the hilarity ™️®️*
ralphmhas joined
Maranda
I wonder why
ludohas joined
Ge0rG
Maranda: Contact addresses for xmpp.is are https://xmpp.is/contact/ (support, admin, feedback, abuse)
Ge0rG
please feel free to bother them.
Maranda
🤣
Dave Cridlandhas left
Dave Cridlandhas left
Dave Cridlandhas left
Ge0rG
When looking for xmpp.is in my log I found this one instead: https://isopres.de/impressum/
Alexhas joined
dwdhas left
Dave Cridlandhas left
mathieui
Thanks for the newsletter, by the way
Ge0rG
Yeah, it's awesome!
Ge0rG
I'd love to have the big ones (like EVE and Epic) mentioned on our twitter as well
Dave Cridlandhas left
lumihas joined
Dave Cridlandhas left
Dave Cridlandhas left
Dave Cridlandhas left
Dave Cridlandhas left
Dave Cridlandhas left
jonasw
this ID mess is a mess
Martinhas joined
jonasw
Kev, there’s no reasonable way we can actually force clients to generate globally unique IDs though, is there?
jonasw
so I don’t think that there’s an actual solution to the "appears to be rewriting history" issue.
Kev
No, I think there's not (I keep saying that, I think) - I wasn't arguing we can solve it, I was arguing that "they've got lots of power anyway" isn't the reason to not care.
Dave Cridlandhas left
lskdjfhas left
jonasw
oh, I must’ve misunderstood that
jonasw
how would we be caring then?
Kev
I felt that "they've got lots of power anyway" was a "we shouldn't care". We should care, we just probably can't avoid it, so we carefully document it in security considerations etc.
Kev
It sounded like Simon was saying that malicious clients/servers are a problem not worth thinking about. I might have misinterpreted his words.
Dave Cridlandhas left
lskdjfhas left
Dave Cridlandhas left
@Alacerhas joined
Dave Cridlandhas left
Dave Cridlandhas left
andyhas joined
Dave Cridlandhas left
SaltyBones
Yeah, that's not what I was trying to say...
blablahas left
danielhas left
Dave Cridlandhas left
j.rhas joined
had-hochas joined
SaltyBones
Hm...how to sum this up...1. A client and server can claim that a message-ID was A when it was B; that is almost unsolvable but the other recipients just won't believe it anyway. 2. There is usually no authentication in XMPP so the point seems a bit moot.
jonasw
the main issue is that an ID can be re-used
jonasw
as far as I understand it
Martinhas left
jonasw
and since we use IDs as identifiers in various protocols (LMC, but also References and stuff), that’s a problem
SaltyBones
Yeah, but there is a difference between assuming that it happens by accident and can be fixed and assuming that it is adversarial.
jonasw
I’m confused
Dave Cridlandhas left
Ge0rG
The only way to properly solve this is to limit the validity domain of IDs to a single session.
Kev
SaltyBones: "no authentication"?
Martinhas joined
Kev
I think one can reasonably argue with that statement.
jonasw
Kev, I think they refer to "cryptographic authentication". A server can esaily forge a stanza for anyone to or from his domain.
SaltyBones
Yeah, sorry.
SaltyBones
Ge0rG, but then you could force the server to have some sort of UUID and a session counter and if you throw the three things together you get reasonable global IDs.
Kev
There's cryptographic authentication, even. It's just that it's applied to the domain, not beyond.
goffi
is there a licence for the Newsletter? Would be nice so it could be translated
Dave Cridlandhas left
jubalhhas left
SaltyBones
goffi, has been discussed in the comm team channel
Tobias
isn't all website content on xmpp.org under a single license
andyhas left
Kev
Should be, but I think that notice was lost at some point.
SaltyBones
Kev, yes, but there is no message authentication so if you only store messages you cannot prove to anybody later that serverA sent you messageB with ID-C...
valohas joined
Kev
That's somewhat different.
valohas joined
SaltyBones
Yes, just pointing it out because you of your "somebody claims you liked a post about KKK" example✎
SaltyBones
Yes, just pointing it out because of your "somebody claims you liked a post about KKK" example ✏
Dave Cridlandhas left
jonasw
SaltyBones, the issue is that that scenario can be caused by a client alone.
SaltyBones
jonasw, the reason I ended up with this proposal is that it makes the situation better and it is very easy to implement.
jonasw
a mailicous server is really powerful, indeed, and we generally assume that each user can trust their own server, and that they have to some extent trust a MUC service if they’re using one
Kev
SaltyBones: I think my issue is that the core of your proposal (the hashing thing) doesn't make anything better :)
Kev
Or, I don't see how it does.
SaltyBones
Kev, that's a reasonable way to look at it. You could just as well not do the hashing and just use per-connectionserver-salt + connection-counter
SaltyBones
That was just a fix for the "oh but the connection counter leaks stuff" problem...
Kev
Oh, no, you couldn't do counters, because of the leaks.
Kev
But as telling the server what ID to use for MAM isn't sensible, AFAICS, I don't think the server being able to predict the client ID buys much at all.
SaltyBones
If the server can assure that the client ID is unique it can simply use that for MAM.
SaltyBones
That's the main point... :)
jonasw
SaltyBones, but that only solves the "client knows the eventual ID of the message"
j.rhas joined
jonasw
it doesn’t solve any of the malleability things cross-domain.
jonasw
that is only solved by your rewriting proposal, and I’m not confident that will work properly
jonasw, not sure what you mean with malleability cross-domain ✏
Kev
SaltyBones: You're assuming that a server is happy to have arbitrary client-provided IDs as the primary query into the archive. I'm suggesting I don't think that's valid.
Kev
I think you have to let the server decide how it indexes its database.
SaltyBones
Kev, they are not arbitrary at all because the server can verify that the client generated them correctly. Essentially they are just forced to use the same generation algo....
Dave Cridlandhas left
SaltyBones
Of course if there are servers that don't like this algorithm for some reason than we should look at what that reason is and how to fix it. :)
jonasw
SaltyBones, the reason for prosody/Zash is clear: their ID contains the date because that allows quick access to a bucket of MAM data
goffi
SaltyBones: and what was the conclusion of the discussion? Can we re-use? Tobias: I don't see any licence mention on the wesite, what is it?
Tobias
I'm sure you can translate it when referencing back and mention the original author
Dave Cridlandhas left
danielhas left
mimi89999has joined
SaltyBones
goffi, translations would be appreciated and then linked to from the main newsletter. Not a real legal discussion.
danielhas left
Dave Cridlandhas left
danielhas left
Dave Cridlandhas left
moparisthebesthas joined
goffi
Tobias: SaltyBones: I plan to translate in French to a popular website, but it require to specify licence, so I want to be sure I can set CC By-SA there. And yes I'll mention original post of course.
Ge0rGhas left
Tobias
what's the SA?
goffi
Share Alike
SaltyBones
goffi, join commteam@ and ask there.
goffi
SaltyBones: Tobias: asking there now, thanks
SaltyBones
jonasw, they could just include a timestamp for timestamp queries and use an increasing salt to have an increasing index...
SaltyBones
We can probably suss out all of these problems but I am not sure anybody but me is actually interested in doing that. xD
Ge0rG
> but I am not sure anybody but me is actually interested in doing that.
I know that feeling. Too well.
dwdhas left
Kev
I'm glad to have this discussion. I'm not convinced that the proposal, and the complexity that goes with it, is solving any problems that a simpler solution doesn't.
jonasw
SaltyBones, but timestamps aren’t monotonic
Kev
That is, it seems to me that the problems solved by this solution are the same solved by just saying to clients 'be unique in origin-id', and updating other XEPs to say 'reply with the origin-id' (LMC, Receipts, etc.).
jonasw
yeah
Ge0rG
SaltyBones: sorry, I didn't even read through the message-IDs thread due to lack of time.
SaltyBones
Kev, and maybe adding a reply "this is your mam-ID" to client messages
SaltyBones
jonasw, they aren't? :)
Kev
Or ignoring origin-id, using the stanza ID the way we always have, and killing MUC with fire :)
jonasw
SaltyBones, clock corrections make it non-monotonic. and of course there’s the issue of clock sync between nodes
Ge0rG
Kev: or just finally mandating that MUC must keep message IDs.
Kev
Or that.
Ge0rG
And mandating that clients which want to employ LMC and other references must use sufficiently unique IDs
SaltyBones
jonasw, ah I was thinking about the lamport kind of timestamp
jonasw
that sounds complex
Ge0rG
Maybe somebody wants to resurrect the Jul 2014 thread on MUC message IDs.
SaltyBones
a little
SaltyBones
jonasw, anyway if you have server generated timestamps and a monotonic counter the mapping should be pretty easy although not a NOP :)
Kev
You need more than monotonic, don't you?
Ge0rG
SaltyBones: but clustering!
Ge0rG
no wait, that was Dave's text.
Ge0rG
but race conditions!1!
SaltyBones
If you want clustering and query by timestamp I suppose you need the opposite of monotonic.
SaltyBones
You want to merge archives by timestamp. Although that sounds dubious.
vanitasvitaehas left
SaltyBones
So, let's suppose that servers really want to generate their own ID for internal use. That seems fair but then why does the client need to know this ID for MAM? That's a bit fishy imho...
jerehas joined
Kev
Because it's that ID that is the index into the archive.
Ge0rG
I wonder if we can do XMPP over that link: http://www.vodafone.com/content/index/media/vodafone-group-releases/2018/vodafone-and-nokia-to-create-first-4g-network-on-moon.html
SaltyBones
but why does the client need to know that
Kev
Because it queries the archive.
SaltyBones
To do what?
SaltyBones
I mean, there are obvious solution here as well: 1. The client query the server by its own ID and the server can try to use that as an additional index or 2. The server could just reply to client messages with the new ID.
SaltyBones
But the whole situation is weird...what are the clients trying to achieve by querying the MAM?
Kev
There is some massive logical disconnect here.
andyhas joined
Kev
You're asking why a user would want to retrieve messages from their message archive.
Kev
What's the point of having the archive if you *can't* query it?
Dave Cridlandhas left
SaltyBones
But why would you need to know what s in the archive to query it
Kev
Have you read the XEP? :)
SaltyBones
Well, I ve tried... :)
Kev
You say things like "Give me everything since message X", where X is the MAM ID.
Dave Cridlandhas left
Guushas left
Guushas left
SaltyBones
Yes, but now we are asking the client to query by and ID which the server generated and didn't tell it about...
SaltyBones
Why not just query by the clients ID or timestamp?
dwdhas left
Ge0rG
timestamps are unreliable
jonasw
SaltyBones, because it is exact
Kev
Syncing on timestamp doesn't work, indeed.
jonasw
timestamps are not exact
Kev
And the client ID means you're storing the primary index provided by the client, which enforces implementation details on the server that I'm not convinced we want to.
Kev
And maybe it's something we can live with, but I don't currently see what it buys us.
Marandahas joined
jonasw
I know at least one implementation which can’t live with that :)
Holger
The message in question might be an incoming message, an outgoing message sent by that client, or an outgoing message sent by another client. You'd use the client ID in some or all these cases?
Kev
jonasw: If you mean Prosody's timestamp one, I think additional stuff's going to end up needed there anyway, for all the other things we were discussing at the summit.
Kev
But yes.
Kev
Holger: Well, you obviously can't in all, for id clashing reasons. At least, not with this suggested scheme.
Holger
That's why I'm asking.
Holger
So basically the client ID is not an option.
Holger
If you don't want to introduce another great mess.
Alexhas left
andyhas left
andyhas joined
dwdhas left
jonasw
Holger, that’s a very good point, I like it :)
Ge0rG
another great mess! \o/
Dave Cridlandhas left
rtq3has left
rtq3has joined
andrey.ghas joined
marchas joined
andyhas left
andrey.ghas joined
Neustradamus
It is beautiful to see the first newsletter after XMPP Roundup and Jabber journal :)
Neustradamus
I see a new redirection problem: http://wiki.jabber.org/index.php/....
andrey.ghas joined
andrey.ghas joined
Holger
I'm confused. Say the client re-logs in after loosing the connection and knows the MAM ID not just of the last incoming but also of the last outgoing message because we solved that somehow. He then queries MAM with after=$ID. How does he decide whether to specify the $ID of the last outgoing or the last incoming message? The ordering of incoming vs. outgoing messages on the client side might be different from the server side, no? (Maybe *this* can only be solved properly by having the server reflect IDs?)
jonasw
yes, that can be solved by having the server reflect IDs
Ge0rG
Holger: that's an awesome point
Dave Cridlandhas left
Ge0rGwrites it on the back of his "race conditions" card
jonasw
hah
jonasw
that’s why I think we just want self-carbons by now.
Kev
It's the same point I made on the list earlier this morning.
jonasw
yeah
Kev
But with more words ;)
Holger
Kev: Oh sorry, I didn't catch up yet.
Kev
The chat in here was triggered by me replying to the mailing list thread, I think.
jonasw
yeah
Ge0rG
Kev: except almost nobody read your mail, it seems
SaltyBones
The ordering or messages in MAM can be different from the client?
Holger
Can we maybe somehow merge reflection of IDs with 0198 ACKs?
Ge0rG
SaltyBones: yes
Ge0rG
Holger: this is something I proposed when MAM first appeared.
Kev
Holger: I'd rather not, that's somewhat breaking layering.
Ge0rG
Holger: 0198 and Carbons and MAM in a single unholy union.
dwdhas left
SaltyBones
Why can they be different and who is right? :)
Holger
Kev: Then the layers are wrong IMO.
Kev
Really?
Holger
Kev: I find it a bit embarrassing to define a protocol that has the server generate two responses to a single message.
andrey.ghas joined
Kev
198 is about the network layer and stuff getting through.
Kev
MAM is about the protocol layer.
jonasw
Holger, two responses?
Zash
198 isn't per message
Dave Cridlandhas left
jonasw
not even per stanza, indeed
Holger
jonasw: (1) ACK I got message with ID $count, (2) ACK I got the message with MAM $ID.
andrey.ghas joined
jonasw
Holger, but the (1) ACK is explicitly requested by the client with an <{sm}r/>
Holger
Yes I'd usually request it per-stanza.
jonasw
hm, I don’
jonasw
*I don’t
Kev
So 198 could be bunching a load of stuff, for different stanzas (which may or may not be messages), and is generally going to happen once the server receives the stanzas, whereas 313 happens later, once it routes goes in the archive.
Holger
jonasw: If the client doesn't deem this necessary, why does it deem it necessary for MAM?
Holger
Ge0rG: Yay I'm good at re-inventing wheels.
Ge0rG
Kev: some servers will only emit the 0198 ack after fully processing the stanza
jonasw
Holger, because MAM contains things from other entities I suppose
Ge0rG
yaxim will emit an <r/> after each message because mobile is unreliable
jonasw
on the XEP-0198 stream, I know the order of the stanzas. In MAM, I don’t unless I get an in-order reflection of my own message.
SaltyBones
If a client queries the MAM by last ID but the order in the MAM might be different I don't understand how it works. :)
Holger
Ge0rG: And if it was reliable you wouldn't need 0198. I never got the idea of requesting an ACK only every now and then.
lskdjfhas joined
Dave Cridlandhas left
Holger
Except for requesting it only once per bunch of stanzas you sent in one go, or so. In which case a single MAM ID response is fine as well.
jonasw
Holger, when sending a bunch of messages at once, it makes sense to request an ack only after the last message
jonasw
saves overhead
jonasw
yeah
dwdhas left
jonasw
(or rather, sending a bunch of stanzas in general)
Holger
Yes what I don't get is how the requirements differ from those for MAM ID reflections.
andrey.ghas joined
jonasw
Holger, hm, maybe that would work.
jonasw
still requires some kind of knowledge about the relative ordering of messages you sent vs. messages you received and other resources sent
jonasw
I don’t think we can get that without something on the stanza layer?
Holger
Why?
jonasw
to be able to query correctly
andrey.ghas joined
Holger
The ordering is now defined by the order or stanza IDs you got on your incoming stream.
Holger
*the order of stanza IDs
jonasw
how would I know when I send and receive a stanza at the same time and then my connection drops without stream management.
moparisthebesthas joined
jonasw
now I need to query the archive
jonasw
which of the two IDs do I use to get a complete, dupfree picture?
Holger
You ditch unacknowledged messages locally and query MAM with after=$ID, where ID is the last ID you got from the server, no?
jonasw
(I’m currently too hungry to think of a more sophisticated case where using the wrong ID would actually lead to *missed* messages, but there might be some)
jonasw
okay, I need some food first
jonasw
food for thought, if you will.
Holger
I'll try coffee.
Holger
And I'll read Kev's email :-)
marmistrzhas left
dwdhas left
andrey.ghas joined
Ge0rGhas left
andrey.ghas joined
andrey.ghas joined
andrey.ghas joined
Dave Cridlandhas left
Dave Cridlandhas left
Guushas left
flow
MAM IDs in SM acks seems to be worth exploring
andrey.ghas joined
andrey.ghas joined
dwdhas left
Dave Cridlandhas left
MattJ
Depends whether you want to communicate to the client "this was the last entry in MAM at this point in the stream", or whether you want the client to know the ID of every message in the archive
Ge0rG
Holger: except we need SM for IQs as well.
flow
MattJ, last entry in archive should be sufficient for most cases
Ge0rG
MattJ: what about giving back a list of message id / message ID pairs.
dwdhas left
Holger
Ge0rG: Those will obviously not carry a MAM ID?
MattJ
I'm guessing Ge0rG means a map of @id -> MAM-ID
Ge0rG
I just love our nomenclature
Alexhas joined
dwdhas left
Dave Cridlandhas left
MattJ
What nomenclature?
andrey.ghas joined
MattJ
Nobody can agree on what to call anything :)
Ge0rG
Can't we just map jabber IDs to nonza IDs and be done?
Holger
:-)
marmistrzhas left
Ge0rG
This protocol is not for zimpies™
andrey.ghas joined
Holger
While at it, maybe just fix 0198 to return an ID for every stanza and ditch both <r/> and the counting which nobody gets right anyway :-P
Ge0rG
Holger: but layers!
Dave Cridlandhas left
Holger
Ge0rG: The TCP layer is responsible for reliable message delivery.
Ge0rG
Holger: no, TCP is a byte stream, not a message stream.
Guushas left
MattJ
I think 198 is fine as-is, and I'm not keen on extending it
MattJ
But yes, we do need to solve the MAM-ID-for-outgoing-messages problem
Ge0rG
MattJ: from a smart-server dumb-client point of view, having four different mechanisms to track messages sucks.
Seve/SouLhas joined
Holger
So then we need a separate acknowledgment for MAM.
Ge0rG
I'm talking of 0184, 0198, 0280 and 0313
jubalhhas joined
MattJ
Typically consensus has been about reflecting outgoing messages (in part, or in full), because this also has other benefits and we do it in Carbons anyway (just not for the originating resource)
Ge0rG
MattJ: yes.
Ge0rG
I wonder how that will play out with self-messages.
MattJ
Heh
Holger
I understand where you guys are coming from, I just think this adds a bit embarrasement when you show the procotol to a newcomer for the first time.
MattJ
Holger, and your preferred solution is?
MattJ
Oh sorry, you already said
Holger
MattJ: Merging 0198 ACKs with 0313 message reflections.
Ge0rG
Holger: XMPP is already mocked by HTTP developers. It can't get any worse.
Dave Cridlandhas left
Dave Cridlandhas left
danielhas left
SaltyBoneshas left
Guushas left
Alexhas left
Holger
But I see how just adding a stanza-id attribute and otherwise keeping 0198 as-is has downsides. So yes just adding a 0313 mechanism is probably an easier way forward.
jonasw
FWIW, I think keeping 198 and MAM IDs separate is sane separation of concerns
MattJ
For the most case I don't think we should be introducing newcomers to the protocol
Holger
It's just like other things where the end result is more convoluted than it would be if we addressed all this sync foo in one go.
MattJ
It's a library problem, and the problem is most libraries just leave you with stanza building
Ge0rG
with 0313 reflections we probably don't need to <r/> each message any more
Holger
We need new libraries!
Ge0rG
We need more libraries!
andrey.ghas joined
Ge0rG
There are only three(?) for python!
Dave Cridlandhas left
jonasw
i need to port poezio to aioxmpp, then there’ll be only one *evil laughter*
Ge0rG
jonasw: python-nbxmpp!
jonasw
that does barely count as library
andrey.ghas joined
Ge0rG
and sleekxmpp used to be a parent of slixmpp? There is also xmpppy
Holger
Ge0rG: Well what we really need is new library authors I guess, and at least those will have to be introduced to the protocol. In practice you'll have to understand most of that stuff as a serious client author as well, of course, even if your library is sane.
Ge0rG
So we have five.
Ge0rG
Holger: as it happens, the most active libraries are maintained by client devs.
Holger
See.
Ge0rG
We have much NIH here.
jonasw
I wonder why ;-)
Holger
So I'm not fully convinced that "but libraries!" is a good excuse for adding insanity to the protocol.
jonasw
I can at least argue that I didn’t NIH, there simply wasn’t anything for python3-asyncio when I started. And I even considered porting sleekxmpp to asyncio, but I thought this to be not reasonably possible.
jonasw
Holger, as both library and client author, I agree.
jonasw
but I think keeping SM and MAM separate is sane.
Ge0rG
jonasw: you are biased.
jonasw
Ge0rG, why?
Ge0rG
I also think that keeping SM and MAM separate is good.
dwdhas left
Ge0rG
And I argue in favor of a new type of session, which is MAM-Sub
jonasw
yeah
jonasw
that’d be great
Ge0rG
> Still, I like the idea of MAM subscriptions as a replacement or augmentation for carbons
Saying that for three years now.
jonasw
Ge0rG, implement it pls
jonasw
although bind2 will probably do pretty much that?
Ge0rG
jonasw: bind2 is just a mechanism to carry things.
jonasw
bind2 would have the effect of MAM-Sub though?
Ge0rG
jonasw: nope
jonasw
by doing MAM sync and carbon enablement in a single atomic step?
Ge0rG
Let me make a strawman proposal of MAM-sub:
- you initiate a bind2 session, supplying the last-known MAM-ID
- the server doesn't deliver offline messages
- the server delivers your pending MAM messages
- the server auto-enables carbons and mam-reflections to you, starting to deliver everything after the MAM sync as "live"
jonasw
yeah
Ge0rG
so MAM-Sub is like Carbons but with mam-reflections
jonasw
alternatively, the server could just give you the current last MAM ID so that you can do the sync asynchronously
Dave Cridlandhas left
jonasw
while already receiving live messages
Ge0rG
I'm sure I proposed that and a bunch of other nifty optimizations (0198 auto-resume/start in bind2) on the ML some time last year
jonasw
yo
jonasw
we just need implementations.
Ge0rG
jonasw: processing MAM after live will be a pita, but okay.
jonasw
depends on your client, I guess
jonasw
I’d be fine with that.
Zashhas left
jonasw
has a considerable latency advantage, especially if you’ve been out for more than just a few hours
Dave Cridlandhas left
flow
Ge0rG, do mam-reflections solve the issue Holger described between incoming and outgoing messages?
andrey.ghas joined
Ge0rG
flow: yes.
moparisthebesthas joined
Ge0rG
flow: MAM reflections will be part of your MAM archive, right between incoming messages, properly ordered.
jonasw
we just need to make sure that MAM-Reflections don’t rewrite IDs. this time for real *scnr*
jonasw
Ge0rG, what, why?
jonasw
wouldn’t you just have your outgoing messages in the MAM archive?
flow
Ge0rG, so mam-reflections are done after the message has been added to your archive, both incoming and outgoing messages
Ge0rG
flow: yes
Ge0rG
jonasw: I'm not following
flow
sounds good
flow
you should write a XEP
Dave Cridlandhas left
flow
or otherwhise the idea will possibly be burried in the standards@ archive
jonasw
Ge0rG, why would you have the MAM reflection thing (presumably <message from="mam" to="your client"><forwarded><inner message from="your client"/></forwarded><stanza-id…/></message>) in the archive instead of just <inner message/>?
dwdhas left
Ge0rG
jonasw: wait, what?
jonasw
what is a MAM reflection?
flow
jonasw, I don't think that is what Ge0rg wanted to say with "will be part of your archive"
Ge0rG
jonasw: whatever we make it to be
jonasw
Ge0rG, I am super confused now
Ge0rG
jonasw: could be a sent carbon of your outgoing message, or the outgoing message wrapped in MAM
jonasw
yeah
jonasw
okay
jonasw
but WHY would you put that wrapped message into MAM again?
Ge0rG
jonasw: or maybe just a small-ish ack with the MAM ID and your original @id
Ge0rG
jonasw: I wouldn't
Dave Cridlandhas left
jonasw
I don’t understand:
12:14:59 Ge0rG> flow: MAM reflections will be part of your MAM archive, right between incoming messages, properly ordered.
this then
Ge0rG
jonasw: I'd put the original message, obviously
Ge0rG
jonasw: ignore it please
jonasw
okay
jonasw
then I didn’t say a thing since 12:15:01Z
Ge0rG
jonasw: of course your *sent message* will be part of your MAM archive, plus the MAM-ID
jonasw
yeah, that’s a good thing :)
Dave Cridlandhas left
Alexhas joined
@Alacerhas left
andrey.ghas joined
Martinhas left
andrey.ghas joined
flow
Ge0rG, once MAMSub is active, clients will only receive messages not stored into mam via the usual way, all other archived messages will be mam-reflected, correct?
Dave Cridlandhas left
Ge0rG
flow: wait, what?
Dave Cridlandhas left
@Alacerhas left
Dave Cridlandhas left
Yagizahas left
flow
Ge0rG, specific mam-reflections please
SaltyBones
I find our reasoning so far somewhat questionable. Because servers might want to use different IDs for messages these IDs should be reflected to the client so that it can make queries with that ID. Shouldn't a simply be able to respond to a clients query if the client uses its original ID? Maybe this is not really practically possible anymore now but it seems somehow more logical. :)✎
Ge0rG
flow: no, you will receive all messages as usual, with MAM-IDs injected
SaltyBones
I find our reasoning so far somewhat questionable. Because servers might want to use different IDs for messages these IDs should be reflected to the client so that it can make queries with that ID. Shouldn't a server simply be able to respond to a clients query if the client uses its original ID? Maybe this is not really practically possible anymore now but it seems somehow more logical. :) ✏
Dave Cridlandhas left
flow
Ge0rG, and carbons still using forwarded?
Ge0rG
flow: let me sort this out: you will receive all(*) incoming messages as regular messages, sent carbons from your other clients as sent carbons and MAM reflections of your outgoing messages as whatever works (e.g. forwarded)
Ge0rG
all(*) = remember what I proposed at the summit / XMPP2 / routing2 brainstorming
Yagizahas joined
Zashhas left
Ge0rG
But maybe routing2 is still too controversial
Zashhas joined
dwdhas left
flow
What gives you the impression that it is too controversial?
Ge0rG
flow: it breaks existing XMPP routing
@Alacerhas left
flow
But it's opt-in. Do we have an example of a legacy protocol which breaks when another involved entity activated routing2?
Ge0rG
flow: not that I am aware of. But the proble is that it changes semantics of bare/full JID routing, which is guaranteed to leak outside the XMPP2 domain
@Alacerhas joined
Dave Cridlandhas left
flow
I guess that is just a fancy expression for "something could rely on the semantics and would break if someone else is using routing2"
MattJ
The root problem is that an XMPP1 entity will happily send to the full JID of an XMPP2 entity and expect it to be treated in the XMPP1 way
Yagizahas left
@Alacerhas left
Kev
Full-JID 'I'm xmpp2' annotation seems like it works though.
@Alacerhas joined
Ge0rG
Kev: maybe, yeah.
Dave Cridlandhas left
dwdhas left
Kev
Or heuristically 'fixing' xmpp1 full JID based on DPI, but that seems unappealing and probably fragile.
flow
Kev, DPI?
Kev
Deep Packet Inspection.
flow
deep packet inspection?
Kev
Which I'm abusing as a term to mean 'look inside the payloads'.
Lancehas left
Lancehas joined
Yagizahas joined
blablahas left
MattJ
The problem with the annotation is that it feels like it's undermining the point of routing2 in the first place
MattJ
If we're going to annotate, let's just annotate and we don't need to make any other changes
MattJ
and that's basically hints, but with a stronger definition of how to process them
Alexhas left
Kev
MattJ: Maybe, perhaps.
Kev
What else would you annotate, though?
MattJ
Exactly - nothing
MattJ
So XMPP2 is XMPP1 with annotations on stuff you want to treat as ephemeral
Kev
And changed routing rules for anything that's not annotated?
Kev
And clients need to know to ignore anything in an annotated message, because it shouldn't be getting them.
Kev
I don't know, I'm kinda concerned that people are going to go down the "Oh, let's annotated such-and-such special casing" like we have with no-copy and no-store at the moment.
Dave Cridlandhas left
MattJ
Indeed
Kev
In principle, it's technically equivalent.
Kev
I still feel we might want to start sending messages from the bare JID instead of the full JID.
MattJ
from or to?
Kev
From.
MattJ
I hadn't considered changing from
Kev
You certainly want to be sending to the bare JID.
jubalhhas left
Kev
Although if we're saying "treat all messages to a full JID as to a bare JID unless they have an ephemeral annotation", that may be reduced, I guess.
bearhas left
Maranda
Hmm conversations lost the message I sent here this morning from the backlog hm hm
Kev
I need to find time to write some specific words here, so we can bash them, I think.
Ge0rG
https://marc.info/?l=openbsd-misc&m=151974573718360&w=2 - Alright, I'm not complaining about XMPP protocol design any more
Dave Cridlandhas left
dwdhas left
jonasw
Ge0rG, lol
jubalhhas joined
dwdhas left
Guushas left
Ge0rG
Yup. OpenSSL. Still written by monkeys.
tim@boese-ban.dehas left
Dave Cridlandhas left
Alexhas joined
danielhas left
@Alacerhas left
Guushas left
dwdhas left
Dave Cridlandhas left
moparisthebesthas joined
moparisthebest
Random question without much thought, why can't the one true message id be implicit as the hash of the whole stanza?
SaltyBones
It's hard to define what should go into the hash and then some "things" change those things anyway so the ID would change...
moparisthebest
But isn't just hash the entire thing good enough?
Guushas left
Dave Cridlandhas left
SaltyBones
moparisthebest, yesterday I would have said yes but now it is clear that people don't just want unique IDs they want very specific IDs and pick them themselves.
Guushas left
moparisthebest
They can still do that
SaltyBones
moparisthebest, just read the backlog from today :)
moparisthebest
The public one is the hash, they can use whatever as the private one
moparisthebest
Encoding implementation decisions into the protocol seems wrong
tim@boese-ban.dehas left
moparisthebest
Especially when it has loads of downsides
MattJ
moparisthebest, hashing XML is problematic, and in any case the same stanza may change en-route
@Alacerhas joined
MattJ
Unless you're saying it's just hashed after the first hop
Martinhas joined
goffi
I was thinking about hash too, but the issue with hash is that you have to find one without collision. If you change, you'll break all existing ecosystem.
SaltyBones
moparisthebest, the problem is that we leak the private one because it is required for MAM queries.
SaltyBones
see my 13:25 comment :)
Guushas left
Dave Cridlandhas left
Kev
moparisthebest: Stanzas might change at every hop. So you can't just hash the whole thing.
Ge0rG
moparisthebest [14:11]:
> Encoding implementation decisions into the protocol seems wrong
Hashing parts of a message into its id is just that
Kev
goffi: Hashes changing is a solved problem, at least, you just specify the hash used.
Martinhas left
Ge0rG
We could just replace messages with their cryptographic ids and become the next peer to peer distributed content storage network
goffi
Kev: yes, but what for already emitted IDs ?
Kev
goffi: They don't have an embedded scheme.
Dave Cridlandhas left
dwdhas left
Guushas left
Guushas left
Dave Cridlandhas left
moparisthebesthas left
moparisthebest
Ge0rG, I'm not saying hashing parts, that gets complicated, I'm saying hash the entire thing
moparisthebest
as to changing at server hops, doesn't the id only matter between client and their server?
la|r|mahas joined
la|r|mahas joined
moparisthebest
so my client sends a message which I know has id AAAA because that's the hash
Dave Cridlandhas left
moparisthebest
my server can change it, if it does, it sends me back a message telling me AAAA is now BBBB
if any server can change the message, the id can only matter between a client and their server?
Kev
No, the id matters end to end.
Kev
But the content of a stanza might be changed at any hop.
Kev
So hashing to generate an id doesn't work.
rionhas left
moparisthebest
so now you might have an id that is the same and different contents?
moparisthebest
what's the point exactly?
Kev
See <delay/> for an obvious application.
moparisthebest
what's the point in a@a.com having the same id as b@b.com ?
moparisthebest
surely it only matters between a@a.com and a.com, and b@b.com and b.com ?
Dave Cridlandhas left
jerehas left
jerehas joined
@Alacerhas left
Kev
Because otherwise you can't reply to previous messages, which you obviously need to be able to do.
moparisthebest
wait where is this concept of replying to a specific message?
moparisthebest
as far as I'm aware, there is just a guaranteed order and that's it
Kev
In assorted XEPs.
Kev
LMC, Receipts, ...
Dave Cridlandhas left
Ge0rG
You'd have to track the message id associations for all eternity.
moparisthebest
it sounds flawed at a basic level, in a federated system, where a message can change at any server hop, how can you expect an id to refer to remotely the same thing on opposite servers?
moparisthebest
but really hashing would still work right? the server knows the incoming hash and the outgoing hash
moparisthebest
when it gets a read receipt etc etc, it just reverse maps it on the way out?
SaltyBones
Ge0rG, you need to track that anyway because read receipts, right?
SaltyBones
I mean the client has to do it not the server but still..
Ge0rG
SaltyBones: the client won't know the effective id between server a and server b
moparisthebest
and doesn't need to Ge0rG ?
Ge0rG
moparisthebest: yes, you need to reverse map, for the lifetime of the message. Which might be months.
moparisthebest
a@a.com can't talk directly to b.com
moparisthebest
sure
Holger
moparisthebest: Message contents aren't unique so you can't use a hash as an ID.
danielhas left
jonasw
inb4 nonce
moparisthebest
Holger, I don't understand what you mean, I mean hash an entire stanza for an id
jonasw
moparisthebest, you can’t reasonably expect a re-writing server to map between IDs for all eternity, *plus* to know all protocols where message IDs may be referenced
moparisthebest, a server would have to re-write a clients reference to another ID, e.g. for XEP-0184 (reciepts) or Last MEssage Correction.
moparisthebest
Holger, I don't understand, if 2 stanzas hash to the same id they are the same stanza
Holger
moparisthebest: But they are still two stanzas.
jonasw
moparisthebest, but maybe sent at a different time.
moparisthebest
no they are just 1 stanza
jonasw
like "that’s a good idea", I send that maybe ten times a week in here
jonasw
a stanza is always its context
Holger
moparisthebest: Hah what?!
jonasw
(aaand here we are at matrix’ DAG thing)
Maranda
Do I have to mention what kind of hard fail hashes are? Does DKIM ring a bell?
moparisthebest
storage-wise you store them once etc?
jonasw
Maranda, yeah, good point
jonasw
oh my god
Holger
moparisthebest: "etc" :-)
Holger
moparisthebest: What jonasw said.
moparisthebest
ah you are saying if you send the exact same stanza every day or something, got it
Dave Cridlandhas left
rtq3has left
moparisthebest
but maybe the rest would work? if id's are only valid per-hop
moparisthebest
and anything rewriting the id keeps a map for as long as needed?
jonasw
moparisthebest, how long is "as needed"?
jonasw
eternity?
moparisthebest
well for a server it'd be as long as it had the message
jonasw
especially with non-IM use cases I can imagine "as long as needed" can be quite a while
Holger
moparisthebest: So you're hashing contents and keeping a map because the contents of a given stanza may change and ignoring the fact that different stanzas might have identical stanzas. But apart from that hashing contents sounds like a perfect solution yes.
moparisthebest
in mam/smacks/offline storage
jonasw
moparisthebest, what about references to that message?
Holger
*identical contents
moparisthebest
Holger, no I've scrapped hashing :P
Holger
Ah.
moparisthebest
jonasw, surely those are only good while there is something to reference?
intosihas joined
Zashhas left
jonasw
moparisthebest, another server might have a MAM for longer than you do
Maranda
moparisthebest, good boy that's for the best 🤗
Maranda
(scrapping hashes)
moparisthebest
Maranda, to be fair that was my first question (if you want to uniquely identify a message why wouldn't hashing work?)
moparisthebest
to which the answer was, we need to uniquely identify a message as well as it's position in the stream
Kev
There were many answers, and I think that's one of the few things that wasn't an answer.
moparisthebest
jonasw, in which case it has a map?
Maranda
I guess that was plentily answered already by multiple sources about the "why"
moparisthebest
the answers I saw about hashing were 'well you have to decide what to hash' which is different
Maranda
Beside that attempting to reinvent "a DKIM version" for stanzas/xmpp is a horrifying thought 😘
jonasw
Maranda, but SPIM!!kk
moparisthebest
dkim is an entirely different solution for an entirely different problem
moparisthebest
that xmpp already has solved from day 1
moparisthebest
(that problem is, is server X allowed to send messages for domain Y)
Ge0rG
Can't we just store the message content on the blockchain and only exchange message IDs?
moparisthebest
so what's the problem with a simple solution like, client sets id, if any server changes it it keeps a map, the end?
rionhas joined
moparisthebest
servers don't *have* to change it, but if they do, they keep a map
Dave Cridlandhas left
Maranda
moparisthebest, sorry to contradict but that's not what DKIM ultimately *does*, not that it's important though.
moparisthebest
that's the goal isn't it Maranda ?
dwdhas left
Maranda
Nope DKIM is more about message authentication, contraffaction and tampering prevention.
moparisthebest
I think it has that side-effect because of the hashing, but the goal was server-that-sent-this-was-authorized-by-domain
Maranda
You're confusing with SPF me thinks.. And both are failing that's why they had to invent a third, DMARC that somehow fails too.
moparisthebest
and there are 2 methods, SPF doesn't hash but is only useable at first hop, and DKIM hashes and is therefore useable through multiple hops (as long as no servers change the hashed part :))
Holger
> DKIM provides a method for validating a domain name identity that is associated with a message through cryptographic authentication.
http://dkim.org/
moparisthebest
DMARC is not a 3rd, it's just enforcing/reporting on SPF and DKIM ?
Alexhas left
moparisthebest
still unsure why we are discussing this, XMPP already guarantees the sending server is authorized by the domain
Holger
Someone mentioned DKIM out of the blue, someone else responded :-)
Holger
I'd like to get back to FTP again!
Maranda
DMARC, nope not just reporting, and I'd avoid reading just introductions. Saves headache laters.
intosihas left
moparisthebest
I said it was enforcing and reporting
moparisthebest
and that's all it is?
Maranda
You did? Oh you did I'm blind apologies blame the cold exposure 😆
moparisthebest
that's fully understandable, unfortunately :)
Maranda
That was a grotesque example to explain how inadeguate I think hashes are in stanzas context, no big deal anyways. Brb.
Kev
Holger: Servers change IDs per-hop based on a hash of the contents, store the mapping between IDs, entities fetch the mapping over FTP and do the lookups there. Address of the FTP server is stored in a blockchain.
Kev
You're welcome.
moparisthebest
I feel like you're missing an opportunity to use GOPHER in there someplace
Holger
Kev: Perfect!
Dave Cridland
Can we guarantee forward secrecy of ids?
Kev
Yes, but not perfectly.
Dave Cridlandhas left
Dave Cridland
Also, no need to use a hash. We could base64 the entire stanza into the id attribute.
Dave Cridland
No collisions, and no need for hash agility then.
jonasw
Dave Cridland, ELOOP
Holger
We base64 the stanza including the original id attribute and then replace it with the result?
Dave Cridland
Without the id attribute, obviously. It'll solve the c14n problem.
Kev
Holger: No, you base64 it including the *new* id.
Holger
Ah. Now it makes sense to me.
Kev
Else the stanza's changing and you'd need a new id.
Dave Cridland
Kev, I don't know why you're being silly. My suggestion was practical, and just as sensible as all the others.
jonasw
Dave Cridland, it’s possible to do
Kev
One of those two statements is true.
jonasw
since there’s no length field, you can put the resul… nevermind
jonasw
oh my god
jonasw
I really should get to sleep
MattJ
I just switched to this tab, I can't tell what's a joke and what's not any more
jonasw
MattJ, assume everything is
Kev
MattJ: That's the joke.
Tobias
and don't forget, today is opposite day :)
Zashhas joined
Dave Cridland
Tobias, OR IS IT!!?!??!!
Dave Cridlandhas left
jubalhhas left
andyhas joined
blablahas left
Dave Cridlandhas left
Yagizahas left
Yagizahas joined
Dave Cridlandhas left
Dave Cridlandhas left
Dave Cridlandhas left
Dave Cridlandhas left
vanitasvitaehas left
j.rhas joined
j.rhas joined
andyhas left
Alexhas joined
vanitasvitaehas left
Dave Cridlandhas left
jjrhhas left
SaltyBones
So, to bring it back a bit. It seems this view is currently popular: The server needs to pick his own IDs, the client needs to use these IDs for MAM queries and we will add reflection so that using reflection and carbons a client gets copies of all its messages to have those IDs.
la|r|mahas left
moparisthebest
why is it more important for the server to pick it's own IDs than the client?
Kev
Concrete proposal: Set origin-ID uniquely on the client, add another id any time something archives it and wants it available, reflect that id back to the first client on first hop. Set the stanza id to the same as the origin-id, but generally ignore it and use the origin-id where available in things like LMC.
SaltyBones
I don't think it is an issue of what is more important it's just that the server writers want to do that. I am not one so I cannot tell you why. :)
Dave Cridlandhas left
moparisthebest
so I propose they can do whatever they want, they just need to keep a map?
Ge0rG
moparisthebest: keeeping a map is a hard task.
Ge0rG
moparisthebest: also actually replacing all references according to the map, because you don't know what's a reference and what's just random data.
moparisthebest
I'm pretty sure it's one of the simplest tasks in computer science
SaltyBones
Ge0rG, true but now we need the client to keep the map or to change the ID of old messages or something...it also seems hard :)
moparisthebest
it's not naming, cache invalidation, or off-by-one errors :P
Kev
No, it's two of those things in one :)
Ge0rG
moparisthebest: actually it is a part of cache invalidation.
Zash
Off by one naming of invalid caches
SaltyBones
But the point stands, doesn't the work still have to be done but now it must be done by the client?
Kev
No.
SaltyBones
It needs the origin-ID for references and the stanza-ID for MAM queries....
Kev
It doesn't need any mapping.
moparisthebest
I thought the point was to hose all those and go to 1 id ?
Ge0rG
and the message ID in case the origin-ID gets stripped
Kev
Ge0rG: I suggest (and it's a novel suggestion, because I know no-one's suggested it before) setting the id to the same as the origin id.
Ge0rG
Kev: you'll never get the author convinced to do that
moparisthebesthas joined
moparisthebesthas left
SaltyBones
Kev, I'm not sure I agree that keeping two IDs for messages is very different from a mapping... :)
Ge0rG
SaltyBones: the client needs to keep an index on the origin-ID, but probably not on the MAM ID
Ge0rG
I need to look up messages by origin-ID for LMC and ACKs, but not when fetching an archive
SaltyBones
Don't you have to merge the local and MAM archive somehow?
moparisthebest
in fact I think keeping 2 id's and an index on one is the very definition of a map
jonasw
Dave Cridland, I sent you an email with https://github.com/xsf/xeps/pull/593 for the council agenda :/
Ge0rG
moparisthebest: the difference is that on the client, you store that as part of the message DB, and you can clean it up together with the messages
Dave Cridlandhas left
Dave Cridlandhas left
dwdhas left
Kev
SaltyBones: These are logically two different things. You need (temporarily) a way of correlating messages and their replies ,which is origin-id. For MAM sync you only need a single ID, which is the latest thing to go in the archive that you've seen (and, depending on your model, even that is optional).
Kev
I don't see a situation in which you ever need to map between the two.
jubalhhas joined
moparisthebest
so what if the client sets an id, and if the server wants to change it, it just sends that info back to the client and doesn't keep a map?
moparisthebest
what's the downside of a single id ?
Ge0rG
moparisthebest: it won't work.
moparisthebest
why not?
Ge0rG
I'm sure this has been outlined in here multiple times today
moparisthebest
client sends id=A, server sends back A is now B, client changes id=A to id=B, done?
Ge0rG
moparisthebest: what if the client disconnects after sending A?
moparisthebest
smacks/mam handles all that
Ge0rG
moparisthebest: except when you don't know that A is B now and the smacks session expires
moparisthebest
B still gets it with mam though
moparisthebest
and C who knows nothing about id=A just ignores it
SaltyBones
Kev, it seemed to be annoying for client devs to handle the different kinds of IDs and merge them and use the appropriate one everywhere but maybe I just misunderstood...
moparisthebest
it seems to be annoying for both client and server devs
Kev
SaltyBones: I don't think it's a significant problem, just that sometimes people aren't sure which id to use.
Kev
(Because everything that talks about the id was written back when there was only the stanza id, and so didn't need to be explicit which one)
Ge0rG
e) none of the above.
moparisthebest
which turns out to be the significant problem
SaltyBones
Yeah, maybe. From this far away I really can't tell how I would handle syncing local history with mam :)
MattJ
There is only one id that MAM is concerned about, ignore everything else
SaltyBones
Sure, so clients will store everything with origin-ID first, then add the stanza-ID when they receive the reflection and then just merge in other messages from MAM based on that, right?
dwdhas left
andyhas joined
moparisthebest
I mean when everything gets this wrong you can pretend it's not a problem, just like everyone did for years with XHTML-IM, that doesn't make it not-a-problem though
MattJ
SaltyBones, personally I don't really understand why origin-id exists
moparisthebest
and to be clear I don't mean in the future things might get this wrong, I mean essentially any newly written thing gets this wrong now
Ge0rG
MattJ: because of MUC reflections.
SaltyBones
MattJ, I think most people agree that it is superfluous and should be the same as message-ID.
MattJ
Ge0rG, and I think servers breaking MUC reflections are broken
Ge0rG
and because back in 2014, somebody refused to mandate that MUCs shall retain the ID on reflection.
MattJ
so lets do that now, because 99% of servers get this right
Ge0rG
MattJ: OK. I'll revive the thread.
MattJ
and stop making this id discussion so much more confusing
rionhas left
MattJ
As I recall the most vocal opponent to mandating id preservation since changed their mind
Ge0rG
Okay, the other reason was that on origin-id you can assume client-enforced uniqueness, which you can't on @id
SaltyBones
what?
MattJ
So then we're back to stanza-id for tracking between you and your server, and @id for end-to-end tracking
SaltyBones
why would a client be able to generate unique origin-ID but not unique message-ID?
Ge0rG
SaltyBones: a client isn't *guaranteed* to generate a unique @id
MattJ
Who cares?
Ge0rG
I don't know.
Ge0rG
maybe people doing references.
MattJ
The only entity concerned with the uniqueness of @id are clients that generate them
Zash
Reference them*
SaltyBones
But if it *can* generate a unique origin-ID it can generate a unique message-ID, yes? :)✎
Ge0rG
MattJ: clients that process incoming @id's too, for references and ACKs
MattJ
so if you're doing receipts/LMC/etc. then just be sure you generate sensible ids
SaltyBones
But if it *can* generate a unique origin-ID it can generate a unique message-ID, can't it? :) ✏
jjrhhas left
Ge0rG
SaltyBones: yes. But with @id you don't know from the outside, with origin-id you do
jjrhhas left
SaltyBones
Ge0rG, you mean it's not mandated in the spec?
Ge0rG
SaltyBones: no. @id is optional
MattJ
SaltyBones, that's the whole root of this discussion
Ge0rG
SaltyBones: you could send all your messages with id="badgerbadger"
SaltyBones
MattJ, maybe, I'm haven't been around long enough to have seen the root of this discussion. ;)
@id is optional and controlled entirely by the sending client, nobody except that client can use it for anything. If you generate an error reply, you reflect the id attribute, that's all
SaltyBones
Ge0rG, ...I can't even imagine that "it seemed like a good idea at the time"
Ge0rG
SaltyBones: MUC is full of such ideas.
MattJ
Second problem is that some MUC server(s?) did not preserve the id attribute in MUC rooms, which broke some clients when they received their own messages back with a different id attribute
flow
> Ge0rG> SaltyBones: you could send all your messages with id="badgerbadger"
flow
I don't think this is true
flow
at least for messages part of the same session
Ge0rG
flow:
> It is up to the originating entity whether the value of the 'id' attribute is unique only within its current stream or unique globally.
Ge0rG
flow: now I could join this MUC with a multi-session nick, reset my session after each MUC message and you'd end up full of badgers.
Ge0rG
or do the same to you via type=chat.
flow
Right, but not within the same session/stream. I just wanted to clarify that
Ge0rG
@id is an end-to-end property, so binding it to the session lifetime doesn't make any sense.
Ge0rG
flow: technically you are correct.
flow
just a matter of the definiton of end"
Ge0rG
Winter's coming!
intosihas joined
Zash
Nah, noh it ends
SaltyBones
So, to sum up: Add reflection of messages to make it easier to figure out how to query MAM otherwise everything has to be the way it is...?
SaltyBones
Maybe document better what every kind of ID is used for to make it easier for devs?
tim@boese-ban.dehas left
danielhas left
MattJ
As far as I'm concerned there is one id set by the client and one id set by the server
flow
<origin-id/> was invented mostly a workaround for the MUC reflection issue
SaltyBones
MattJ, ...and the client set one is origin-ID the server set one is stanza-ID.
MattJ
I strongly feel that origin-id should go
MattJ
and that any broken MUC services should be fixed
Ge0rG
flow: except it doesn't actually solve it, as seen with biboumi.
SaltyBones
Okay, so move origin-ID to message-ID and forbid rewriting.
flow
Ge0rG, m0ar pls
Ge0rG
I'll bring it up on council.
MattJ
flow, summary is that transports can't always preserve it
flow
SaltyBones, why move? just use rfc6120 IDs like before
danielhas left
SaltyBones
I mean conceptually, move the responsibility...
MattJ
which puts them in the "broken MUC server" category as far as I'm concerned
Kev
FWIW, transports don't have to be written in such a way as they lose them.
Kev, ah, you're saying that transports can always be written in such a way that they do not lose IDs?
moparisthebest
not if it's impossible to keep a map of IDs >:)
Alexhas left
andyhas left
Alexhas joined
Lancehas left
Kev
SaltyBones: At the cost of complexity.
SaltyBones
Sure, sure
SaltyBones
Yeah forbidding ID mangling seems like it would be a very same thing to do
SaltyBones
Should standza-IDs be transmitted btw? Or is that between client and server only?
SaltyBones
s/same/sane
rtq3has left
Zashhas left
nycohas left
marmistrzhas left
Lancehas joined
Marandahas joined
SaltyBoneshas left
danielhas left
Guushas left
danielhas left
rtq3has joined
intosihas left
jubalhhas joined
andyhas joined
ralphmhas joined
MattJ
SaltyBones, the latter. It's stamped on incoming stanzas (to the user) by the server, to let the client know what id the server has assigned it (e.g. in the MAM archive)
MattJ
and in some version of the future, it will also be stamped on reflected outgoing messages
Zashhas left
andyhas left
andyhas joined
remkohas left
Zashhas joined
Lancehas left
Lancehas joined
andyhas left
andyhas joined
jubalhhas left
moparisthebesthas joined
Lancehas left
waqashas joined
Nekithas left
lskdjfhas left
SaltyBones
And are stanza-IDs used anywhere else? They seem to be presented as this kind of general concept that can be used to add stable IDs for some sort of domain...
Kev
They started off as just a part of MAM. They were extracted out (probably wrongly).
Kev
Well, no, not quite wrongly.
Kev
Because they're used beyond MAM, they're also going to be used for Unread sync, and injected into Carbons and stuff.
MattJ
SaltyBones, originally MAM used to stamp <archived by="archive-jid" id="unique-id-for-mam-queries" />
ludohas joined
MattJ
But the general feeling was that this unique id shouldn't be MAM-specific, but reusable
Zashhas left
Zashhas joined
SaltyBones
Yeah, I'm just wondering if it has actually ever been used for anything and if tha anything is also between client and server or not.
Dave Cridlandhas left
Kev
Yes, only between client and server (or client and client)