vanitasvitaeBasically it is a link list, but there were one or two articles that slipped my eyes
rtq3has left
rtq3has joined
jubalhhas joined
Lancehas joined
Dave Cridlandhas left
Dave Cridlandhas left
xnyhpshas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
Dave Cridlandhas left
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
Dave Cridlandhas left
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
dwdhas left
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
xnyhpshas left
tim@boese-ban.dehas joined
Dave Cridlandhas left
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
Dave Cridlandhas left
xnyhpshas joined
andyhas left
Dave Cridlandhas left
rionhas left
SaltyBoneshas left
dwdhas left
Kevhas joined
dwdhas left
Guushas left
ralphmhas joined
Marandahas joined
@Alacerhas left
Dave Cridlandhas left
rionhas left
pep.has joined
j.rhas left
j.rhas joined
Dave Cridlandhas left
Steve Killehas left
Steve Killehas left
Dave Cridlandhas left
dwdhas left
remkohas joined
@Alacerhas joined
Dave Cridlandhas left
Dave Cridlandhas left
andyhas joined
matlaghas joined
Lancehas left
danielhas left
dwdhas left
stefandxmhas left
andyhas left
Steve Killehas joined
jubalhhas joined
Dave Cridlandhas left
dwdhas left
vanitasvitaehas left
vanitasvitaehas joined
j.rhas joined
j.rhas joined
ralphmhas left
danielhas left
Alexhas joined
@Alacerhas left
Lancehas joined
rtq3has left
Dave Cridlandhas left
rtq3has joined
dwdhas left
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
Alexhas left
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
Dave Cridlandhas left
tim@boese-ban.dehas joined
tim@boese-ban.dehas joined
dwdhas left
tim@boese-ban.dehas joined
@Alacerhas joined
SaltyBoneshas left
rionhas left
@Alacerhas left
jubalhhas joined
jubalhhas left
lskdjfhas joined
la|r|mahas joined
ralphmhas left
la|r|mahas joined
la|r|mahas joined
ludohas left
ludohas joined
Ge0rGI love it how xmpp.is say they won't support the spam fighting manifesto, and then describe how they essentially implement each of the stated requirements <https://xmpp.is/2018/02/21/the-jabber-spam-fighting-manifesto/>
moparisthebesthas joined
rionhas left
jubalhhas joined
moparisthebesthas joined
Dave Cridlandhas left
MarandaGe0rG, infact xmpp.is is one of the servers I get spim hits from atm, *the hilarity ™️®️*
ralphmhas joined
MarandaI wonder why
ludohas joined
Ge0rGMaranda: Contact addresses for xmpp.is are https://xmpp.is/contact/ (support, admin, feedback, abuse)
Ge0rGplease feel free to bother them.
Maranda🤣
Dave Cridlandhas left
Dave Cridlandhas left
Dave Cridlandhas left
Ge0rGWhen looking for xmpp.is in my log I found this one instead: https://isopres.de/impressum/
Alexhas joined
dwdhas left
Dave Cridlandhas left
mathieuiThanks for the newsletter, by the way
Ge0rGYeah, it's awesome!
Ge0rGI'd love to have the big ones (like EVE and Epic) mentioned on our twitter as well
Dave Cridlandhas left
lumihas joined
Dave Cridlandhas left
Dave Cridlandhas left
Dave Cridlandhas left
Dave Cridlandhas left
Dave Cridlandhas left
jonaswthis ID mess is a mess
Martinhas joined
jonaswKev, there’s no reasonable way we can actually force clients to generate globally unique IDs though, is there?
jonaswso I don’t think that there’s an actual solution to the "appears to be rewriting history" issue.
KevNo, I think there's not (I keep saying that, I think) - I wasn't arguing we can solve it, I was arguing that "they've got lots of power anyway" isn't the reason to not care.
Dave Cridlandhas left
lskdjfhas left
jonaswoh, I must’ve misunderstood that
jonaswhow would we be caring then?
KevI felt that "they've got lots of power anyway" was a "we shouldn't care". We should care, we just probably can't avoid it, so we carefully document it in security considerations etc.
KevIt sounded like Simon was saying that malicious clients/servers are a problem not worth thinking about. I might have misinterpreted his words.
Dave Cridlandhas left
lskdjfhas left
Dave Cridlandhas left
@Alacerhas joined
Dave Cridlandhas left
Dave Cridlandhas left
andyhas joined
Dave Cridlandhas left
SaltyBonesYeah, that's not what I was trying to say...
blablahas left
danielhas left
Dave Cridlandhas left
j.rhas joined
had-hochas joined
SaltyBonesHm...how to sum this up...1. A client and server can claim that a message-ID was A when it was B; that is almost unsolvable but the other recipients just won't believe it anyway. 2. There is usually no authentication in XMPP so the point seems a bit moot.
jonaswthe main issue is that an ID can be re-used
jonaswas far as I understand it
Martinhas left
jonaswand since we use IDs as identifiers in various protocols (LMC, but also References and stuff), that’s a problem
SaltyBonesYeah, but there is a difference between assuming that it happens by accident and can be fixed and assuming that it is adversarial.
jonaswI’m confused
Dave Cridlandhas left
Ge0rGThe only way to properly solve this is to limit the validity domain of IDs to a single session.
KevSaltyBones: "no authentication"?
Martinhas joined
KevI think one can reasonably argue with that statement.
jonaswKev, I think they refer to "cryptographic authentication". A server can esaily forge a stanza for anyone to or from his domain.
SaltyBonesYeah, sorry.
SaltyBonesGe0rG, but then you could force the server to have some sort of UUID and a session counter and if you throw the three things together you get reasonable global IDs.
KevThere's cryptographic authentication, even. It's just that it's applied to the domain, not beyond.
goffiis there a licence for the Newsletter? Would be nice so it could be translated
Dave Cridlandhas left
jubalhhas left
SaltyBonesgoffi, has been discussed in the comm team channel
Tobiasisn't all website content on xmpp.org under a single license
andyhas left
KevShould be, but I think that notice was lost at some point.
SaltyBonesKev, yes, but there is no message authentication so if you only store messages you cannot prove to anybody later that serverA sent you messageB with ID-C...
valohas joined
KevThat's somewhat different.
valohas joined
SaltyBonesYes, just pointing it out because you of your "somebody claims you liked a post about KKK" example
SaltyBonesYes, just pointing it out because of your "somebody claims you liked a post about KKK" example
Dave Cridlandhas left
jonaswSaltyBones, the issue is that that scenario can be caused by a client alone.
SaltyBonesjonasw, the reason I ended up with this proposal is that it makes the situation better and it is very easy to implement.
jonaswa mailicous server is really powerful, indeed, and we generally assume that each user can trust their own server, and that they have to some extent trust a MUC service if they’re using one
KevSaltyBones: I think my issue is that the core of your proposal (the hashing thing) doesn't make anything better :)
KevOr, I don't see how it does.
SaltyBonesKev, that's a reasonable way to look at it. You could just as well not do the hashing and just use per-connectionserver-salt + connection-counter
SaltyBonesThat was just a fix for the "oh but the connection counter leaks stuff" problem...
KevOh, no, you couldn't do counters, because of the leaks.
KevBut as telling the server what ID to use for MAM isn't sensible, AFAICS, I don't think the server being able to predict the client ID buys much at all.
SaltyBonesIf the server can assure that the client ID is unique it can simply use that for MAM.
SaltyBonesThat's the main point... :)
jonaswSaltyBones, but that only solves the "client knows the eventual ID of the message"
j.rhas joined
jonaswit doesn’t solve any of the malleability things cross-domain.
jonaswthat is only solved by your rewriting proposal, and I’m not confident that will work properly
jonaswexcept with a lot of complexity on the server
Dave Cridlandhas left
SaltyBonesjonasw, not sure what you mean
SaltyBonesjonasw, not sure what you mean with malleability cross-domain
KevSaltyBones: You're assuming that a server is happy to have arbitrary client-provided IDs as the primary query into the archive. I'm suggesting I don't think that's valid.
KevI think you have to let the server decide how it indexes its database.
SaltyBonesKev, they are not arbitrary at all because the server can verify that the client generated them correctly. Essentially they are just forced to use the same generation algo....
Dave Cridlandhas left
SaltyBonesOf course if there are servers that don't like this algorithm for some reason than we should look at what that reason is and how to fix it. :)
jonaswSaltyBones, the reason for prosody/Zash is clear: their ID contains the date because that allows quick access to a bucket of MAM data
goffiSaltyBones: and what was the conclusion of the discussion? Can we re-use? Tobias: I don't see any licence mention on the wesite, what is it?
TobiasI'm sure you can translate it when referencing back and mention the original author
Dave Cridlandhas left
danielhas left
mimi89999has joined
SaltyBonesgoffi, translations would be appreciated and then linked to from the main newsletter. Not a real legal discussion.
danielhas left
Dave Cridlandhas left
danielhas left
Dave Cridlandhas left
moparisthebesthas joined
goffiTobias: SaltyBones: I plan to translate in French to a popular website, but it require to specify licence, so I want to be sure I can set CC By-SA there. And yes I'll mention original post of course.
Ge0rGhas left
Tobiaswhat's the SA?
goffiShare Alike
SaltyBonesgoffi, join commteam@ and ask there.
goffiSaltyBones: Tobias: asking there now, thanks
SaltyBonesjonasw, they could just include a timestamp for timestamp queries and use an increasing salt to have an increasing index...
SaltyBonesWe can probably suss out all of these problems but I am not sure anybody but me is actually interested in doing that. xD
Ge0rG> but I am not sure anybody but me is actually interested in doing that.
I know that feeling. Too well.
dwdhas left
KevI'm glad to have this discussion. I'm not convinced that the proposal, and the complexity that goes with it, is solving any problems that a simpler solution doesn't.
jonaswSaltyBones, but timestamps aren’t monotonic
KevThat is, it seems to me that the problems solved by this solution are the same solved by just saying to clients 'be unique in origin-id', and updating other XEPs to say 'reply with the origin-id' (LMC, Receipts, etc.).
jonaswyeah
Ge0rGSaltyBones: sorry, I didn't even read through the message-IDs thread due to lack of time.
SaltyBonesKev, and maybe adding a reply "this is your mam-ID" to client messages
SaltyBonesjonasw, they aren't? :)
KevOr ignoring origin-id, using the stanza ID the way we always have, and killing MUC with fire :)
jonaswSaltyBones, clock corrections make it non-monotonic. and of course there’s the issue of clock sync between nodes
Ge0rGKev: or just finally mandating that MUC must keep message IDs.
KevOr that.
Ge0rGAnd mandating that clients which want to employ LMC and other references must use sufficiently unique IDs
SaltyBonesjonasw, ah I was thinking about the lamport kind of timestamp
jonaswthat sounds complex
Ge0rGMaybe somebody wants to resurrect the Jul 2014 thread on MUC message IDs.
SaltyBonesa little
SaltyBonesjonasw, anyway if you have server generated timestamps and a monotonic counter the mapping should be pretty easy although not a NOP :)
KevYou need more than monotonic, don't you?
Ge0rGSaltyBones: but clustering!
Ge0rGno wait, that was Dave's text.
Ge0rGbut race conditions!1!
SaltyBonesIf you want clustering and query by timestamp I suppose you need the opposite of monotonic.
SaltyBonesYou want to merge archives by timestamp. Although that sounds dubious.
vanitasvitaehas left
SaltyBonesSo, let's suppose that servers really want to generate their own ID for internal use. That seems fair but then why does the client need to know this ID for MAM? That's a bit fishy imho...
jerehas joined
KevBecause it's that ID that is the index into the archive.
Ge0rGI wonder if we can do XMPP over that link: http://www.vodafone.com/content/index/media/vodafone-group-releases/2018/vodafone-and-nokia-to-create-first-4g-network-on-moon.html
SaltyBonesbut why does the client need to know that
KevBecause it queries the archive.
SaltyBonesTo do what?
SaltyBonesI mean, there are obvious solution here as well: 1. The client query the server by its own ID and the server can try to use that as an additional index or 2. The server could just reply to client messages with the new ID.
SaltyBonesBut the whole situation is weird...what are the clients trying to achieve by querying the MAM?
KevThere is some massive logical disconnect here.
andyhas joined
KevYou're asking why a user would want to retrieve messages from their message archive.
KevWhat's the point of having the archive if you *can't* query it?
Dave Cridlandhas left
SaltyBonesBut why would you need to know what s in the archive to query it
KevHave you read the XEP? :)
SaltyBonesWell, I ve tried... :)
KevYou say things like "Give me everything since message X", where X is the MAM ID.
Dave Cridlandhas left
Guushas left
Guushas left
SaltyBonesYes, but now we are asking the client to query by and ID which the server generated and didn't tell it about...
SaltyBonesWhy not just query by the clients ID or timestamp?
dwdhas left
Ge0rGtimestamps are unreliable
jonaswSaltyBones, because it is exact
KevSyncing on timestamp doesn't work, indeed.
jonaswtimestamps are not exact
KevAnd the client ID means you're storing the primary index provided by the client, which enforces implementation details on the server that I'm not convinced we want to.
KevAnd maybe it's something we can live with, but I don't currently see what it buys us.
Marandahas joined
jonaswI know at least one implementation which can’t live with that :)
HolgerThe message in question might be an incoming message, an outgoing message sent by that client, or an outgoing message sent by another client. You'd use the client ID in some or all these cases?
Kevjonasw: If you mean Prosody's timestamp one, I think additional stuff's going to end up needed there anyway, for all the other things we were discussing at the summit.
KevBut yes.
KevHolger: Well, you obviously can't in all, for id clashing reasons. At least, not with this suggested scheme.
HolgerThat's why I'm asking.
HolgerSo basically the client ID is not an option.
HolgerIf you don't want to introduce another great mess.
Alexhas left
andyhas left
andyhas joined
dwdhas left
jonaswHolger, that’s a very good point, I like it :)
Ge0rGanother great mess! \o/
Dave Cridlandhas left
rtq3has left
rtq3has joined
andrey.ghas joined
marchas joined
andyhas left
andrey.ghas joined
NeustradamusIt is beautiful to see the first newsletter after XMPP Roundup and Jabber journal :)
NeustradamusI see a new redirection problem: http://wiki.jabber.org/index.php/....
andrey.ghas joined
andrey.ghas joined
HolgerI'm confused. Say the client re-logs in after loosing the connection and knows the MAM ID not just of the last incoming but also of the last outgoing message because we solved that somehow. He then queries MAM with after=$ID. How does he decide whether to specify the $ID of the last outgoing or the last incoming message? The ordering of incoming vs. outgoing messages on the client side might be different from the server side, no? (Maybe *this* can only be solved properly by having the server reflect IDs?)
jonaswyes, that can be solved by having the server reflect IDs
Ge0rGHolger: that's an awesome point
Dave Cridlandhas left
Ge0rGwrites it on the back of his "race conditions" card
jonaswhah
jonaswthat’s why I think we just want self-carbons by now.
KevIt's the same point I made on the list earlier this morning.
jonaswyeah
KevBut with more words ;)
HolgerKev: Oh sorry, I didn't catch up yet.
KevThe chat in here was triggered by me replying to the mailing list thread, I think.
jonaswyeah
Ge0rGKev: except almost nobody read your mail, it seems
SaltyBonesThe ordering or messages in MAM can be different from the client?
HolgerCan we maybe somehow merge reflection of IDs with 0198 ACKs?
Ge0rGSaltyBones: yes
Ge0rGHolger: this is something I proposed when MAM first appeared.
KevHolger: I'd rather not, that's somewhat breaking layering.
Ge0rGHolger: 0198 and Carbons and MAM in a single unholy union.
dwdhas left
SaltyBonesWhy can they be different and who is right? :)
HolgerKev: Then the layers are wrong IMO.
KevReally?
HolgerKev: I find it a bit embarrassing to define a protocol that has the server generate two responses to a single message.
andrey.ghas joined
Kev198 is about the network layer and stuff getting through.
KevMAM is about the protocol layer.
jonaswHolger, two responses?
Zash198 isn't per message
Dave Cridlandhas left
jonaswnot even per stanza, indeed
Holgerjonasw: (1) ACK I got message with ID $count, (2) ACK I got the message with MAM $ID.
andrey.ghas joined
jonaswHolger, but the (1) ACK is explicitly requested by the client with an <{sm}r/>
HolgerYes I'd usually request it per-stanza.
jonaswhm, I don’
jonasw*I don’t
KevSo 198 could be bunching a load of stuff, for different stanzas (which may or may not be messages), and is generally going to happen once the server receives the stanzas, whereas 313 happens later, once it routes goes in the archive.
Holgerjonasw: If the client doesn't deem this necessary, why does it deem it necessary for MAM?
HolgerGe0rG: Yay I'm good at re-inventing wheels.
Ge0rGKev: some servers will only emit the 0198 ack after fully processing the stanza
jonaswHolger, because MAM contains things from other entities I suppose
Ge0rGyaxim will emit an <r/> after each message because mobile is unreliable
jonaswon the XEP-0198 stream, I know the order of the stanzas. In MAM, I don’t unless I get an in-order reflection of my own message.
SaltyBonesIf a client queries the MAM by last ID but the order in the MAM might be different I don't understand how it works. :)
HolgerGe0rG: And if it was reliable you wouldn't need 0198. I never got the idea of requesting an ACK only every now and then.
lskdjfhas joined
Dave Cridlandhas left
HolgerExcept for requesting it only once per bunch of stanzas you sent in one go, or so. In which case a single MAM ID response is fine as well.
jonaswHolger, when sending a bunch of messages at once, it makes sense to request an ack only after the last message
jonaswsaves overhead
jonaswyeah
dwdhas left
jonasw(or rather, sending a bunch of stanzas in general)
HolgerYes what I don't get is how the requirements differ from those for MAM ID reflections.
andrey.ghas joined
jonaswHolger, hm, maybe that would work.
jonaswstill requires some kind of knowledge about the relative ordering of messages you sent vs. messages you received and other resources sent
jonaswI don’t think we can get that without something on the stanza layer?
HolgerWhy?
jonaswto be able to query correctly
andrey.ghas joined
HolgerThe ordering is now defined by the order or stanza IDs you got on your incoming stream.
Holger*the order of stanza IDs
jonaswhow would I know when I send and receive a stanza at the same time and then my connection drops without stream management.
moparisthebesthas joined
jonaswnow I need to query the archive
jonaswwhich of the two IDs do I use to get a complete, dupfree picture?
HolgerYou ditch unacknowledged messages locally and query MAM with after=$ID, where ID is the last ID you got from the server, no?
jonasw(I’m currently too hungry to think of a more sophisticated case where using the wrong ID would actually lead to *missed* messages, but there might be some)
jonaswokay, I need some food first
jonaswfood for thought, if you will.
HolgerI'll try coffee.
HolgerAnd I'll read Kev's email :-)
marmistrzhas left
dwdhas left
andrey.ghas joined
Ge0rGhas left
andrey.ghas joined
andrey.ghas joined
andrey.ghas joined
Dave Cridlandhas left
Dave Cridlandhas left
Guushas left
flowMAM IDs in SM acks seems to be worth exploring
andrey.ghas joined
andrey.ghas joined
dwdhas left
Dave Cridlandhas left
MattJDepends whether you want to communicate to the client "this was the last entry in MAM at this point in the stream", or whether you want the client to know the ID of every message in the archive
Ge0rGHolger: except we need SM for IQs as well.
flowMattJ, last entry in archive should be sufficient for most cases
Ge0rGMattJ: what about giving back a list of message id / message ID pairs.
dwdhas left
HolgerGe0rG: Those will obviously not carry a MAM ID?
MattJI'm guessing Ge0rG means a map of @id -> MAM-ID
Ge0rGI just love our nomenclature
Alexhas joined
dwdhas left
Dave Cridlandhas left
MattJWhat nomenclature?
andrey.ghas joined
MattJNobody can agree on what to call anything :)
Ge0rGCan't we just map jabber IDs to nonza IDs and be done?
Holger:-)
marmistrzhas left
Ge0rGThis protocol is not for zimpies™
andrey.ghas joined
HolgerWhile at it, maybe just fix 0198 to return an ID for every stanza and ditch both <r/> and the counting which nobody gets right anyway :-P
Ge0rGHolger: but layers!
Dave Cridlandhas left
HolgerGe0rG: The TCP layer is responsible for reliable message delivery.
Ge0rGHolger: no, TCP is a byte stream, not a message stream.
Guushas left
MattJI think 198 is fine as-is, and I'm not keen on extending it
MattJBut yes, we do need to solve the MAM-ID-for-outgoing-messages problem
Ge0rGMattJ: from a smart-server dumb-client point of view, having four different mechanisms to track messages sucks.
Seve/SouLhas joined
HolgerSo then we need a separate acknowledgment for MAM.
Ge0rGI'm talking of 0184, 0198, 0280 and 0313
jubalhhas joined
MattJTypically consensus has been about reflecting outgoing messages (in part, or in full), because this also has other benefits and we do it in Carbons anyway (just not for the originating resource)
Ge0rGMattJ: yes.
Ge0rGI wonder how that will play out with self-messages.
MattJHeh
HolgerI understand where you guys are coming from, I just think this adds a bit embarrasement when you show the procotol to a newcomer for the first time.
MattJHolger, and your preferred solution is?
MattJOh sorry, you already said
HolgerMattJ: Merging 0198 ACKs with 0313 message reflections.
Ge0rGHolger: XMPP is already mocked by HTTP developers. It can't get any worse.
Dave Cridlandhas left
Dave Cridlandhas left
danielhas left
SaltyBoneshas left
Guushas left
Alexhas left
HolgerBut I see how just adding a stanza-id attribute and otherwise keeping 0198 as-is has downsides. So yes just adding a 0313 mechanism is probably an easier way forward.
jonaswFWIW, I think keeping 198 and MAM IDs separate is sane separation of concerns
MattJFor the most case I don't think we should be introducing newcomers to the protocol
HolgerIt's just like other things where the end result is more convoluted than it would be if we addressed all this sync foo in one go.
MattJIt's a library problem, and the problem is most libraries just leave you with stanza building
Ge0rGwith 0313 reflections we probably don't need to <r/> each message any more
HolgerWe need new libraries!
Ge0rGWe need more libraries!
andrey.ghas joined
Ge0rGThere are only three(?) for python!
Dave Cridlandhas left
jonaswi need to port poezio to aioxmpp, then there’ll be only one *evil laughter*
Ge0rGjonasw: python-nbxmpp!
jonaswthat does barely count as library
andrey.ghas joined
Ge0rGand sleekxmpp used to be a parent of slixmpp? There is also xmpppy
HolgerGe0rG: Well what we really need is new library authors I guess, and at least those will have to be introduced to the protocol. In practice you'll have to understand most of that stuff as a serious client author as well, of course, even if your library is sane.
Ge0rGSo we have five.
Ge0rGHolger: as it happens, the most active libraries are maintained by client devs.
HolgerSee.
Ge0rGWe have much NIH here.
jonaswI wonder why ;-)
HolgerSo I'm not fully convinced that "but libraries!" is a good excuse for adding insanity to the protocol.
jonaswI can at least argue that I didn’t NIH, there simply wasn’t anything for python3-asyncio when I started. And I even considered porting sleekxmpp to asyncio, but I thought this to be not reasonably possible.
jonaswHolger, as both library and client author, I agree.
jonaswbut I think keeping SM and MAM separate is sane.
Ge0rGjonasw: you are biased.
jonaswGe0rG, why?
Ge0rGI also think that keeping SM and MAM separate is good.
dwdhas left
Ge0rGAnd I argue in favor of a new type of session, which is MAM-Sub
jonaswyeah
jonaswthat’d be great
Ge0rG> Still, I like the idea of MAM subscriptions as a replacement or augmentation for carbons
Saying that for three years now.
jonaswGe0rG, implement it pls
jonaswalthough bind2 will probably do pretty much that?
Ge0rGjonasw: bind2 is just a mechanism to carry things.
jonaswbind2 would have the effect of MAM-Sub though?
Ge0rGjonasw: nope
jonaswby doing MAM sync and carbon enablement in a single atomic step?
Ge0rGLet me make a strawman proposal of MAM-sub:
- you initiate a bind2 session, supplying the last-known MAM-ID
- the server doesn't deliver offline messages
- the server delivers your pending MAM messages
- the server auto-enables carbons and mam-reflections to you, starting to deliver everything after the MAM sync as "live"
jonaswyeah
Ge0rGso MAM-Sub is like Carbons but with mam-reflections
jonaswalternatively, the server could just give you the current last MAM ID so that you can do the sync asynchronously
Dave Cridlandhas left
jonaswwhile already receiving live messages
Ge0rGI'm sure I proposed that and a bunch of other nifty optimizations (0198 auto-resume/start in bind2) on the ML some time last year
jonaswyo
jonaswwe just need implementations.
Ge0rGjonasw: processing MAM after live will be a pita, but okay.
jonaswdepends on your client, I guess
jonaswI’d be fine with that.
Zashhas left
jonaswhas a considerable latency advantage, especially if you’ve been out for more than just a few hours
Dave Cridlandhas left
flowGe0rG, do mam-reflections solve the issue Holger described between incoming and outgoing messages?
andrey.ghas joined
Ge0rGflow: yes.
moparisthebesthas joined
Ge0rGflow: MAM reflections will be part of your MAM archive, right between incoming messages, properly ordered.
jonaswwe just need to make sure that MAM-Reflections don’t rewrite IDs. this time for real *scnr*
jonaswGe0rG, what, why?
jonaswwouldn’t you just have your outgoing messages in the MAM archive?
flowGe0rG, so mam-reflections are done after the message has been added to your archive, both incoming and outgoing messages
Ge0rGflow: yes
Ge0rGjonasw: I'm not following
flowsounds good
flowyou should write a XEP
Dave Cridlandhas left
flowor otherwhise the idea will possibly be burried in the standards@ archive
jonaswGe0rG, why would you have the MAM reflection thing (presumably <message from="mam" to="your client"><forwarded><inner message from="your client"/></forwarded><stanza-id…/></message>) in the archive instead of just <inner message/>?
dwdhas left
Ge0rGjonasw: wait, what?
jonaswwhat is a MAM reflection?
flowjonasw, I don't think that is what Ge0rg wanted to say with "will be part of your archive"
Ge0rGjonasw: whatever we make it to be
jonaswGe0rG, I am super confused now
Ge0rGjonasw: could be a sent carbon of your outgoing message, or the outgoing message wrapped in MAM
jonaswyeah
jonaswokay
jonaswbut WHY would you put that wrapped message into MAM again?
Ge0rGjonasw: or maybe just a small-ish ack with the MAM ID and your original @id
Ge0rGjonasw: I wouldn't
Dave Cridlandhas left
jonaswI don’t understand:
12:14:59 Ge0rG> flow: MAM reflections will be part of your MAM archive, right between incoming messages, properly ordered.
this then
Ge0rGjonasw: I'd put the original message, obviously
Ge0rGjonasw: ignore it please
jonaswokay
jonaswthen I didn’t say a thing since 12:15:01Z
Ge0rGjonasw: of course your *sent message* will be part of your MAM archive, plus the MAM-ID
jonaswyeah, that’s a good thing :)
Dave Cridlandhas left
Alexhas joined
@Alacerhas left
andrey.ghas joined
Martinhas left
andrey.ghas joined
flowGe0rG, once MAMSub is active, clients will only receive messages not stored into mam via the usual way, all other archived messages will be mam-reflected, correct?
Dave Cridlandhas left
Ge0rGflow: wait, what?
Dave Cridlandhas left
@Alacerhas left
Dave Cridlandhas left
Yagizahas left
flowGe0rG, specific mam-reflections please
SaltyBonesI find our reasoning so far somewhat questionable. Because servers might want to use different IDs for messages these IDs should be reflected to the client so that it can make queries with that ID. Shouldn't a simply be able to respond to a clients query if the client uses its original ID? Maybe this is not really practically possible anymore now but it seems somehow more logical. :)
Ge0rGflow: no, you will receive all messages as usual, with MAM-IDs injected
SaltyBonesI find our reasoning so far somewhat questionable. Because servers might want to use different IDs for messages these IDs should be reflected to the client so that it can make queries with that ID. Shouldn't a server simply be able to respond to a clients query if the client uses its original ID? Maybe this is not really practically possible anymore now but it seems somehow more logical. :)
Dave Cridlandhas left
flowGe0rG, and carbons still using forwarded?
Ge0rGflow: let me sort this out: you will receive all(*) incoming messages as regular messages, sent carbons from your other clients as sent carbons and MAM reflections of your outgoing messages as whatever works (e.g. forwarded)
Ge0rGall(*) = remember what I proposed at the summit / XMPP2 / routing2 brainstorming
Yagizahas joined
Zashhas left
Ge0rGBut maybe routing2 is still too controversial
Zashhas joined
dwdhas left
flowWhat gives you the impression that it is too controversial?
Ge0rGflow: it breaks existing XMPP routing
@Alacerhas left
flowBut it's opt-in. Do we have an example of a legacy protocol which breaks when another involved entity activated routing2?
Ge0rGflow: not that I am aware of. But the proble is that it changes semantics of bare/full JID routing, which is guaranteed to leak outside the XMPP2 domain
@Alacerhas joined
Dave Cridlandhas left
flowI guess that is just a fancy expression for "something could rely on the semantics and would break if someone else is using routing2"
MattJThe root problem is that an XMPP1 entity will happily send to the full JID of an XMPP2 entity and expect it to be treated in the XMPP1 way
Yagizahas left
@Alacerhas left
KevFull-JID 'I'm xmpp2' annotation seems like it works though.
@Alacerhas joined
Ge0rGKev: maybe, yeah.
Dave Cridlandhas left
dwdhas left
KevOr heuristically 'fixing' xmpp1 full JID based on DPI, but that seems unappealing and probably fragile.
flowKev, DPI?
KevDeep Packet Inspection.
flowdeep packet inspection?
KevWhich I'm abusing as a term to mean 'look inside the payloads'.
Lancehas left
Lancehas joined
Yagizahas joined
blablahas left
MattJThe problem with the annotation is that it feels like it's undermining the point of routing2 in the first place
MattJIf we're going to annotate, let's just annotate and we don't need to make any other changes
MattJand that's basically hints, but with a stronger definition of how to process them
Alexhas left
KevMattJ: Maybe, perhaps.
KevWhat else would you annotate, though?
MattJExactly - nothing
MattJSo XMPP2 is XMPP1 with annotations on stuff you want to treat as ephemeral
KevAnd changed routing rules for anything that's not annotated?
KevAnd clients need to know to ignore anything in an annotated message, because it shouldn't be getting them.
KevI don't know, I'm kinda concerned that people are going to go down the "Oh, let's annotated such-and-such special casing" like we have with no-copy and no-store at the moment.
Dave Cridlandhas left
MattJIndeed
KevIn principle, it's technically equivalent.
KevI still feel we might want to start sending messages from the bare JID instead of the full JID.
MattJfrom or to?
KevFrom.
MattJI hadn't considered changing from
KevYou certainly want to be sending to the bare JID.
jubalhhas left
KevAlthough if we're saying "treat all messages to a full JID as to a bare JID unless they have an ephemeral annotation", that may be reduced, I guess.
bearhas left
MarandaHmm conversations lost the message I sent here this morning from the backlog hm hm
KevI need to find time to write some specific words here, so we can bash them, I think.
Ge0rGhttps://marc.info/?l=openbsd-misc&m=151974573718360&w=2 - Alright, I'm not complaining about XMPP protocol design any more
Dave Cridlandhas left
dwdhas left
jonaswGe0rG, lol
jubalhhas joined
dwdhas left
Guushas left
Ge0rGYup. OpenSSL. Still written by monkeys.
tim@boese-ban.dehas left
Dave Cridlandhas left
Alexhas joined
danielhas left
@Alacerhas left
Guushas left
dwdhas left
Dave Cridlandhas left
moparisthebesthas joined
moparisthebestRandom question without much thought, why can't the one true message id be implicit as the hash of the whole stanza?
SaltyBonesIt's hard to define what should go into the hash and then some "things" change those things anyway so the ID would change...
moparisthebestBut isn't just hash the entire thing good enough?
Guushas left
Dave Cridlandhas left
SaltyBonesmoparisthebest, yesterday I would have said yes but now it is clear that people don't just want unique IDs they want very specific IDs and pick them themselves.
Guushas left
moparisthebestThey can still do that
SaltyBonesmoparisthebest, just read the backlog from today :)
moparisthebestThe public one is the hash, they can use whatever as the private one
moparisthebestEncoding implementation decisions into the protocol seems wrong
tim@boese-ban.dehas left
moparisthebestEspecially when it has loads of downsides
MattJmoparisthebest, hashing XML is problematic, and in any case the same stanza may change en-route
@Alacerhas joined
MattJUnless you're saying it's just hashed after the first hop
Martinhas joined
goffiI was thinking about hash too, but the issue with hash is that you have to find one without collision. If you change, you'll break all existing ecosystem.
SaltyBonesmoparisthebest, the problem is that we leak the private one because it is required for MAM queries.
SaltyBonessee my 13:25 comment :)
Guushas left
Dave Cridlandhas left
Kevmoparisthebest: Stanzas might change at every hop. So you can't just hash the whole thing.
Ge0rGmoparisthebest [14:11]:
> Encoding implementation decisions into the protocol seems wrong
Hashing parts of a message into its id is just that
Kevgoffi: Hashes changing is a solved problem, at least, you just specify the hash used.
Martinhas left
Ge0rGWe could just replace messages with their cryptographic ids and become the next peer to peer distributed content storage network
goffiKev: yes, but what for already emitted IDs ?
Kevgoffi: They don't have an embedded scheme.
Dave Cridlandhas left
dwdhas left
Guushas left
Guushas left
Dave Cridlandhas left
moparisthebesthas left
moparisthebestGe0rG, I'm not saying hashing parts, that gets complicated, I'm saying hash the entire thing
moparisthebestas to changing at server hops, doesn't the id only matter between client and their server?
la|r|mahas joined
la|r|mahas joined
moparisthebestso my client sends a message which I know has id AAAA because that's the hash
Dave Cridlandhas left
moparisthebestmy server can change it, if it does, it sends me back a message telling me AAAA is now BBBB
moparisthebestdoes that not solve everything?
KevNo the idea matters between endpoint entities.
KevNo the id matters between endpoint entities.
moparisthebestif any server can change the message, the id can only matter between a client and their server?
KevNo, the id matters end to end.
KevBut the content of a stanza might be changed at any hop.
KevSo hashing to generate an id doesn't work.
rionhas left
moparisthebestso now you might have an id that is the same and different contents?
moparisthebestwhat's the point exactly?
KevSee <delay/> for an obvious application.
moparisthebestwhat's the point in a@a.com having the same id as b@b.com ?
moparisthebestsurely it only matters between a@a.com and a.com, and b@b.com and b.com ?
Dave Cridlandhas left
jerehas left
jerehas joined
@Alacerhas left
KevBecause otherwise you can't reply to previous messages, which you obviously need to be able to do.
moparisthebestwait where is this concept of replying to a specific message?
moparisthebestas far as I'm aware, there is just a guaranteed order and that's it
KevIn assorted XEPs.
KevLMC, Receipts, ...
Dave Cridlandhas left
Ge0rGYou'd have to track the message id associations for all eternity.
moparisthebestit sounds flawed at a basic level, in a federated system, where a message can change at any server hop, how can you expect an id to refer to remotely the same thing on opposite servers?
moparisthebestbut really hashing would still work right? the server knows the incoming hash and the outgoing hash
moparisthebestwhen it gets a read receipt etc etc, it just reverse maps it on the way out?
SaltyBonesGe0rG, you need to track that anyway because read receipts, right?
SaltyBonesI mean the client has to do it not the server but still..
Ge0rGSaltyBones: the client won't know the effective id between server a and server b
moparisthebestand doesn't need to Ge0rG ?
Ge0rGmoparisthebest: yes, you need to reverse map, for the lifetime of the message. Which might be months.
moparisthebesta@a.com can't talk directly to b.com
moparisthebestsure
Holgermoparisthebest: Message contents aren't unique so you can't use a hash as an ID.
danielhas left
jonaswinb4 nonce
moparisthebestHolger, I don't understand what you mean, I mean hash an entire stanza for an id
jonaswmoparisthebest, you can’t reasonably expect a re-writing server to map between IDs for all eternity, *plus* to know all protocols where message IDs may be referenced
moparisthebestif you rewrite, you map, easy
moparisthebestshouldn't need to know any specific protocols?
jonaswmoparisthebest, a server would have to re-write a clients reference to another ID, e.g. for XEP-0184 (reciepts) or Last MEssage Correction.
moparisthebestHolger, I don't understand, if 2 stanzas hash to the same id they are the same stanza
Holgermoparisthebest: But they are still two stanzas.
jonaswmoparisthebest, but maybe sent at a different time.
moparisthebestno they are just 1 stanza
jonaswlike "that’s a good idea", I send that maybe ten times a week in here
jonaswa stanza is always its context
Holgermoparisthebest: Hah what?!
jonasw(aaand here we are at matrix’ DAG thing)
MarandaDo I have to mention what kind of hard fail hashes are? Does DKIM ring a bell?
moparisthebeststorage-wise you store them once etc?
jonaswMaranda, yeah, good point
jonaswoh my god
Holgermoparisthebest: "etc" :-)
Holgermoparisthebest: What jonasw said.
moparisthebestah you are saying if you send the exact same stanza every day or something, got it
Dave Cridlandhas left
rtq3has left
moparisthebestbut maybe the rest would work? if id's are only valid per-hop
moparisthebestand anything rewriting the id keeps a map for as long as needed?
jonaswmoparisthebest, how long is "as needed"?
jonasweternity?
moparisthebestwell for a server it'd be as long as it had the message
jonaswespecially with non-IM use cases I can imagine "as long as needed" can be quite a while
Holgermoparisthebest: So you're hashing contents and keeping a map because the contents of a given stanza may change and ignoring the fact that different stanzas might have identical stanzas. But apart from that hashing contents sounds like a perfect solution yes.
moparisthebestin mam/smacks/offline storage
jonaswmoparisthebest, what about references to that message?
Holger*identical contents
moparisthebestHolger, no I've scrapped hashing :P
HolgerAh.
moparisthebestjonasw, surely those are only good while there is something to reference?
intosihas joined
Zashhas left
jonaswmoparisthebest, another server might have a MAM for longer than you do
Marandamoparisthebest, good boy that's for the best 🤗
Maranda(scrapping hashes)
moparisthebestMaranda, to be fair that was my first question (if you want to uniquely identify a message why wouldn't hashing work?)
moparisthebestto which the answer was, we need to uniquely identify a message as well as it's position in the stream
KevThere were many answers, and I think that's one of the few things that wasn't an answer.
moparisthebestjonasw, in which case it has a map?
MarandaI guess that was plentily answered already by multiple sources about the "why"
moparisthebestthe answers I saw about hashing were 'well you have to decide what to hash' which is different
MarandaBeside that attempting to reinvent "a DKIM version" for stanzas/xmpp is a horrifying thought 😘
jonaswMaranda, but SPIM!!kk
moparisthebestdkim is an entirely different solution for an entirely different problem
moparisthebestthat xmpp already has solved from day 1
moparisthebest(that problem is, is server X allowed to send messages for domain Y)
Ge0rGCan't we just store the message content on the blockchain and only exchange message IDs?
moparisthebestso what's the problem with a simple solution like, client sets id, if any server changes it it keeps a map, the end?
rionhas joined
moparisthebestservers don't *have* to change it, but if they do, they keep a map
Dave Cridlandhas left
Marandamoparisthebest, sorry to contradict but that's not what DKIM ultimately *does*, not that it's important though.
moparisthebestthat's the goal isn't it Maranda ?
dwdhas left
MarandaNope DKIM is more about message authentication, contraffaction and tampering prevention.
moparisthebestI think it has that side-effect because of the hashing, but the goal was server-that-sent-this-was-authorized-by-domain
MarandaYou're confusing with SPF me thinks.. And both are failing that's why they had to invent a third, DMARC that somehow fails too.
moparisthebestand there are 2 methods, SPF doesn't hash but is only useable at first hop, and DKIM hashes and is therefore useable through multiple hops (as long as no servers change the hashed part :))
Holger> DKIM provides a method for validating a domain name identity that is associated with a message through cryptographic authentication.
http://dkim.org/
moparisthebestDMARC is not a 3rd, it's just enforcing/reporting on SPF and DKIM ?
Alexhas left
moparisthebeststill unsure why we are discussing this, XMPP already guarantees the sending server is authorized by the domain
HolgerSomeone mentioned DKIM out of the blue, someone else responded :-)
HolgerI'd like to get back to FTP again!
MarandaDMARC, nope not just reporting, and I'd avoid reading just introductions. Saves headache laters.
intosihas left
moparisthebestI said it was enforcing and reporting
moparisthebestand that's all it is?
MarandaYou did? Oh you did I'm blind apologies blame the cold exposure 😆
MarandaThat was a grotesque example to explain how inadeguate I think hashes are in stanzas context, no big deal anyways. Brb.
KevHolger: Servers change IDs per-hop based on a hash of the contents, store the mapping between IDs, entities fetch the mapping over FTP and do the lookups there. Address of the FTP server is stored in a blockchain.
KevYou're welcome.
moparisthebestI feel like you're missing an opportunity to use GOPHER in there someplace
HolgerKev: Perfect!
Dave CridlandCan we guarantee forward secrecy of ids?
KevYes, but not perfectly.
Dave Cridlandhas left
Dave CridlandAlso, no need to use a hash. We could base64 the entire stanza into the id attribute.
Dave CridlandNo collisions, and no need for hash agility then.
jonaswDave Cridland, ELOOP
HolgerWe base64 the stanza including the original id attribute and then replace it with the result?
Dave CridlandWithout the id attribute, obviously. It'll solve the c14n problem.
KevHolger: No, you base64 it including the *new* id.
HolgerAh. Now it makes sense to me.
KevElse the stanza's changing and you'd need a new id.
Dave CridlandKev, I don't know why you're being silly. My suggestion was practical, and just as sensible as all the others.
jonaswDave Cridland, it’s possible to do
KevOne of those two statements is true.
jonaswsince there’s no length field, you can put the resul… nevermind
jonaswoh my god
jonaswI really should get to sleep
MattJI just switched to this tab, I can't tell what's a joke and what's not any more
jonaswMattJ, assume everything is
KevMattJ: That's the joke.
Tobiasand don't forget, today is opposite day :)
Zashhas joined
Dave CridlandTobias, OR IS IT!!?!??!!
Dave Cridlandhas left
jubalhhas left
andyhas joined
blablahas left
Dave Cridlandhas left
Yagizahas left
Yagizahas joined
Dave Cridlandhas left
Dave Cridlandhas left
Dave Cridlandhas left
Dave Cridlandhas left
vanitasvitaehas left
j.rhas joined
j.rhas joined
andyhas left
Alexhas joined
vanitasvitaehas left
Dave Cridlandhas left
jjrhhas left
SaltyBonesSo, to bring it back a bit. It seems this view is currently popular: The server needs to pick his own IDs, the client needs to use these IDs for MAM queries and we will add reflection so that using reflection and carbons a client gets copies of all its messages to have those IDs.
la|r|mahas left
moparisthebestwhy is it more important for the server to pick it's own IDs than the client?
KevConcrete proposal: Set origin-ID uniquely on the client, add another id any time something archives it and wants it available, reflect that id back to the first client on first hop. Set the stanza id to the same as the origin-id, but generally ignore it and use the origin-id where available in things like LMC.
SaltyBonesI don't think it is an issue of what is more important it's just that the server writers want to do that. I am not one so I cannot tell you why. :)
Dave Cridlandhas left
moparisthebestso I propose they can do whatever they want, they just need to keep a map?
Ge0rGmoparisthebest: keeeping a map is a hard task.
Ge0rGmoparisthebest: also actually replacing all references according to the map, because you don't know what's a reference and what's just random data.
moparisthebestI'm pretty sure it's one of the simplest tasks in computer science
SaltyBonesGe0rG, true but now we need the client to keep the map or to change the ID of old messages or something...it also seems hard :)
moparisthebestit's not naming, cache invalidation, or off-by-one errors :P
KevNo, it's two of those things in one :)
Ge0rGmoparisthebest: actually it is a part of cache invalidation.
ZashOff by one naming of invalid caches
SaltyBonesBut the point stands, doesn't the work still have to be done but now it must be done by the client?
KevNo.
SaltyBonesIt needs the origin-ID for references and the stanza-ID for MAM queries....
KevIt doesn't need any mapping.
moparisthebestI thought the point was to hose all those and go to 1 id ?
Ge0rGand the message ID in case the origin-ID gets stripped
KevGe0rG: I suggest (and it's a novel suggestion, because I know no-one's suggested it before) setting the id to the same as the origin id.
Ge0rGKev: you'll never get the author convinced to do that
moparisthebesthas joined
moparisthebesthas left
SaltyBonesKev, I'm not sure I agree that keeping two IDs for messages is very different from a mapping... :)
Ge0rGSaltyBones: the client needs to keep an index on the origin-ID, but probably not on the MAM ID
Ge0rGI need to look up messages by origin-ID for LMC and ACKs, but not when fetching an archive
SaltyBonesDon't you have to merge the local and MAM archive somehow?
moparisthebestin fact I think keeping 2 id's and an index on one is the very definition of a map
jonaswDave Cridland, I sent you an email with https://github.com/xsf/xeps/pull/593 for the council agenda :/
Ge0rGmoparisthebest: the difference is that on the client, you store that as part of the message DB, and you can clean it up together with the messages
Dave Cridlandhas left
Dave Cridlandhas left
dwdhas left
KevSaltyBones: These are logically two different things. You need (temporarily) a way of correlating messages and their replies ,which is origin-id. For MAM sync you only need a single ID, which is the latest thing to go in the archive that you've seen (and, depending on your model, even that is optional).
KevI don't see a situation in which you ever need to map between the two.
jubalhhas joined
moparisthebestso what if the client sets an id, and if the server wants to change it, it just sends that info back to the client and doesn't keep a map?
moparisthebestwhat's the downside of a single id ?
Ge0rGmoparisthebest: it won't work.
moparisthebestwhy not?
Ge0rGI'm sure this has been outlined in here multiple times today
moparisthebestclient sends id=A, server sends back A is now B, client changes id=A to id=B, done?
Ge0rGmoparisthebest: what if the client disconnects after sending A?
moparisthebestsmacks/mam handles all that
Ge0rGmoparisthebest: except when you don't know that A is B now and the smacks session expires
moparisthebestB still gets it with mam though
moparisthebestand C who knows nothing about id=A just ignores it
SaltyBonesKev, it seemed to be annoying for client devs to handle the different kinds of IDs and merge them and use the appropriate one everywhere but maybe I just misunderstood...
moparisthebestit seems to be annoying for both client and server devs
KevSaltyBones: I don't think it's a significant problem, just that sometimes people aren't sure which id to use.
Kev(Because everything that talks about the id was written back when there was only the stanza id, and so didn't need to be explicit which one)
Ge0rGe) none of the above.
moparisthebestwhich turns out to be the significant problem
SaltyBonesYeah, maybe. From this far away I really can't tell how I would handle syncing local history with mam :)
MattJThere is only one id that MAM is concerned about, ignore everything else
SaltyBonesSure, so clients will store everything with origin-ID first, then add the stanza-ID when they receive the reflection and then just merge in other messages from MAM based on that, right?
dwdhas left
andyhas joined
moparisthebestI mean when everything gets this wrong you can pretend it's not a problem, just like everyone did for years with XHTML-IM, that doesn't make it not-a-problem though
MattJSaltyBones, personally I don't really understand why origin-id exists
moparisthebestand to be clear I don't mean in the future things might get this wrong, I mean essentially any newly written thing gets this wrong now
Ge0rGMattJ: because of MUC reflections.
SaltyBonesMattJ, I think most people agree that it is superfluous and should be the same as message-ID.
MattJGe0rG, and I think servers breaking MUC reflections are broken
Ge0rGand because back in 2014, somebody refused to mandate that MUCs shall retain the ID on reflection.
MattJso lets do that now, because 99% of servers get this right
Ge0rGMattJ: OK. I'll revive the thread.
MattJand stop making this id discussion so much more confusing
rionhas left
MattJAs I recall the most vocal opponent to mandating id preservation since changed their mind
Ge0rGOkay, the other reason was that on origin-id you can assume client-enforced uniqueness, which you can't on @id
SaltyBoneswhat?
MattJSo then we're back to stanza-id for tracking between you and your server, and @id for end-to-end tracking
SaltyBoneswhy would a client be able to generate unique origin-ID but not unique message-ID?
Ge0rGSaltyBones: a client isn't *guaranteed* to generate a unique @id
MattJWho cares?
Ge0rGI don't know.
Ge0rGmaybe people doing references.
MattJThe only entity concerned with the uniqueness of @id are clients that generate them
ZashReference them*
SaltyBonesBut if it *can* generate a unique origin-ID it can generate a unique message-ID, yes? :)
Ge0rGMattJ: clients that process incoming @id's too, for references and ACKs
MattJso if you're doing receipts/LMC/etc. then just be sure you generate sensible ids
SaltyBonesBut if it *can* generate a unique origin-ID it can generate a unique message-ID, can't it? :)
jjrhhas left
Ge0rGSaltyBones: yes. But with @id you don't know from the outside, with origin-id you do
jjrhhas left
SaltyBonesGe0rG, you mean it's not mandated in the spec?
Ge0rGSaltyBones: no. @id is optional
MattJSaltyBones, that's the whole root of this discussion
Ge0rGSaltyBones: you could send all your messages with id="badgerbadger"
SaltyBonesMattJ, maybe, I'm haven't been around long enough to have seen the root of this discussion. ;)
MattJ@id is optional and controlled entirely by the sending client, nobody except that client can use it for anything. If you generate an error reply, you reflect the id attribute, that's all
SaltyBonesGe0rG, ...I can't even imagine that "it seemed like a good idea at the time"
Ge0rGSaltyBones: MUC is full of such ideas.
MattJSecond problem is that some MUC server(s?) did not preserve the id attribute in MUC rooms, which broke some clients when they received their own messages back with a different id attribute
flow> Ge0rG> SaltyBones: you could send all your messages with id="badgerbadger"
flowI don't think this is true
flowat least for messages part of the same session
Ge0rGflow:
> It is up to the originating entity whether the value of the 'id' attribute is unique only within its current stream or unique globally.
Ge0rGflow: now I could join this MUC with a multi-session nick, reset my session after each MUC message and you'd end up full of badgers.
Ge0rGor do the same to you via type=chat.
flowRight, but not within the same session/stream. I just wanted to clarify that
Ge0rG@id is an end-to-end property, so binding it to the session lifetime doesn't make any sense.
Ge0rGflow: technically you are correct.
flowjust a matter of the definiton of end"
Ge0rGWinter's coming!
intosihas joined
ZashNah, noh it ends
SaltyBonesSo, to sum up: Add reflection of messages to make it easier to figure out how to query MAM otherwise everything has to be the way it is...?
SaltyBonesMaybe document better what every kind of ID is used for to make it easier for devs?
tim@boese-ban.dehas left
danielhas left
MattJAs far as I'm concerned there is one id set by the client and one id set by the server
flow<origin-id/> was invented mostly a workaround for the MUC reflection issue
SaltyBonesMattJ, ...and the client set one is origin-ID the server set one is stanza-ID.
MattJI strongly feel that origin-id should go
MattJand that any broken MUC services should be fixed
Ge0rGflow: except it doesn't actually solve it, as seen with biboumi.
SaltyBonesOkay, so move origin-ID to message-ID and forbid rewriting.
flowGe0rG, m0ar pls
Ge0rGI'll bring it up on council.
MattJflow, summary is that transports can't always preserve it
flowSaltyBones, why move? just use rfc6120 IDs like before
danielhas left
SaltyBonesI mean conceptually, move the responsibility...
MattJwhich puts them in the "broken MUC server" category as far as I'm concerned
KevFWIW, transports don't have to be written in such a way as they lose them.
SaltyBonesKev, what?
SaltyBonesKev, what? parse_error
xnyhpshas joined
danielhas left
Steve Killehas left
Guushas left
intosihas left
KevThe ids
Steve Killehas left
rtq3has joined
andyhas left
andyhas joined
danielhas left
Lancehas left
lskdjfhas joined
jjrhhas left
jjrhhas left
lskdjfhas joined
j.rhas joined
jjrhhas left
jjrhhas left
danielhas left
jjrhhas left
danielhas left
jjrhhas left
Lancehas joined
MarandaBiboumi... 🤔
mimi89999has left
Lancehas left
Lancehas joined
lovetoxhas joined
tuxhas left
jubalhhas left
Tobiashas joined
mimi89999has left
mimi89999has left
mimi89999has joined
danielhas left
peterhas joined
andyhas left
andyhas joined
marmistrzhas left
SaltyBoneshas left
danielhas left
Guushas left
ralphmhas left
danielhas left
Tobiashas joined
danielhas left
danielhas left
Guushas left
rtq3has left
rtq3has joined
ralphmhas joined
Guushas left
danielhas left
marmistrzhas left
Yagizahas left
Yagizahas joined
danielhas left
la|r|mahas left
Lancehas left
lskdjfhas joined
Lancehas joined
jubalhhas joined
nycohas left
Lancehas left
rtq3has left
Lancehas joined
suzyohas joined
Marandahas joined
danielhas left
rtq3has joined
waqashas joined
ralphmhas joined
suzyohas joined
tuxhas left
moparisthebesthas joined
moparisthebesthas joined
Steve Killehas left
moparisthebesthas left
jubalhhas left
Steve Killehas left
waqashas left
waqashas joined
waqashas left
waqashas joined
Dave Cridlandhas left
Dave Cridlandhas left
danielhas left
Ge0rGhas left
danielhas left
lskdjfhas joined
Guushas left
danielhas left
Ge0rGhas left
rtq3has left
rtq3has joined
xnyhpshas left
xnyhpshas joined
marmistrzhas left
marmistrzhas left
waqashas left
Guushas left
Guushas left
Steve Killehas joined
marmistrzhas left
lskdjfhas left
ralphmhas left
Yagizahas left
Guushas left
Kevhas left
Guushas left
danielhas left
Kevhas joined
rtq3has left
rtq3has joined
SamWhitedhas left
Guushas left
Guushas left
Steve Killehas left
lskdjfhas joined
rtq3has left
rtq3has joined
Alexhas left
Alexhas joined
lskdjfhas left
lskdjfhas left
tuxhas joined
tuxhas joined
intosihas joined
efrithas joined
danielhas left
ralphmhas joined
rtq3has left
rtq3has joined
marmistrzhas left
lskdjfhas left
intosihas left
Guushas left
intosihas joined
goffihas left
ralphmhas joined
ralphmhas left
waqashas joined
waqashas left
ralphmhas joined
SaltyBonesKev, ah, you're saying that transports can always be written in such a way that they do not lose IDs?
moparisthebestnot if it's impossible to keep a map of IDs >:)
Alexhas left
andyhas left
Alexhas joined
Lancehas left
KevSaltyBones: At the cost of complexity.
SaltyBonesSure, sure
SaltyBonesYeah forbidding ID mangling seems like it would be a very same thing to do
SaltyBonesShould standza-IDs be transmitted btw? Or is that between client and server only?
SaltyBoness/same/sane
rtq3has left
Zashhas left
nycohas left
marmistrzhas left
Lancehas joined
Marandahas joined
SaltyBoneshas left
danielhas left
Guushas left
danielhas left
rtq3has joined
intosihas left
jubalhhas joined
andyhas joined
ralphmhas joined
MattJSaltyBones, the latter. It's stamped on incoming stanzas (to the user) by the server, to let the client know what id the server has assigned it (e.g. in the MAM archive)
MattJand in some version of the future, it will also be stamped on reflected outgoing messages
Zashhas left
andyhas left
andyhas joined
remkohas left
Zashhas joined
Lancehas left
Lancehas joined
andyhas left
andyhas joined
jubalhhas left
moparisthebesthas joined
Lancehas left
waqashas joined
Nekithas left
lskdjfhas left
SaltyBonesAnd are stanza-IDs used anywhere else? They seem to be presented as this kind of general concept that can be used to add stable IDs for some sort of domain...
KevThey started off as just a part of MAM. They were extracted out (probably wrongly).
KevWell, no, not quite wrongly.
KevBecause they're used beyond MAM, they're also going to be used for Unread sync, and injected into Carbons and stuff.
MattJSaltyBones, originally MAM used to stamp <archived by="archive-jid" id="unique-id-for-mam-queries" />
ludohas joined
MattJBut the general feeling was that this unique id shouldn't be MAM-specific, but reusable
Zashhas left
Zashhas joined
SaltyBonesYeah, I'm just wondering if it has actually ever been used for anything and if tha anything is also between client and server or not.
Dave Cridlandhas left
KevYes, only between client and server (or client and client)
KevUnread sync is the obvious one that it's needed.