XSF Discussion - 2018-02-12

  1. Seve

    daniel, Matrix already does that. http://mirror.onet.pl/pub/mirrors/video.fosdem.org/2018/H.1309%20(Van%20Rijn)/matrix_webvr.webm

  2. rion

    looks cool

  3. zinid

    daniel: Psi gets crazy when it receives muc self-presence, it renders "empty" participant in muc roster

  4. rion


  5. zinid


  6. rion

    What server should I install to have such a self-presence? ejabberd?

  7. zinid

    rion, no, it's on local machine

  8. rion

    in any case last two day I rewrote muc roster. so it won't happen anymore. not commited yet

  9. zinid


  10. zinid

    the problem is that now I'm not sure if other clients don't break though

  11. zinid

    rion, Psi requests MUC vCard on every join?

  12. rion


  13. rion

    I think with self-presence and caps I'll change this too

  14. rion

    and I have to improve avatars caching for mucs too as well as their requesting algo. otherwise it's a lot of traffic on join.

  15. zinid


  16. zinid

    poezio has the same problem wrt avatars 😉

  17. zinid

    gajim seems doing fine, doesn't render anything strange

  18. Seve

    By the way, regarding the email I sent to JUser, is there a better mailing list for that, please?

  19. jonasw

    I still haven’t subscribed to juser :/

  20. Seve

    That's why I would like to send it somewhere else too, seems not many people are subscribed there

  21. jonasw

    Seve, I think operators would be more fitting, maybe

  22. jonasw

    possibly with a cross-post to members@

  23. jonasw

    I’m also not sure if a separate MUC for this is a good idea

  24. Zash

    Posted something to some KDE list?

  25. jonasw

    frankly, I’m not sure XMPP can fulfil the requirements at this instant in time

  26. jonasw

    > Stickers are available

  27. jonasw

    we doomed

  28. Zash

    Find out which are requirements and which are wishlist

  29. jonasw

    Zash, they don’t even know that yet

  30. jonasw

    > Requirements are not prioritized (incl. must-have vs. nice-to-have), that comes at a later step.

  31. Zash

    jonasw: All are wishlist. There's online one requirement, that everyone you care about is already using it. :)

  32. Seve

    jonasw, the point of the MUC is to discuss this topic with people interested in helping out. Almost everybody here would find this discussion annoying here.

  33. jonasw

    Seve, I’m running out of screen space for more MUCs.

  34. marc

    Ge0rG, jonasw can you review https://git.zapb.de/xeps.git/commit/?h=fix_xep_0401&id=52725e993987f205dc253cc5a4e6937fe3955d81 and merge please?

  35. jonasw

    marc, EBUSY right now

  36. jonasw

    marc, if you refuse to make a github PR (which would really be the most useful thing for me), please send an email with the first line of the commit message as subject to editor@xmpp.org

  37. marc

    jonasw, I can do a GitHub PR if you like

  38. jonasw

    marc, that would be much appreciated, thanks

  39. marc

    jonasw, you're welcome

  40. jonasw

    gotta run now, see you

  41. marc


  42. marc


  43. jonasw

    zinid, regarding the server capabilities invalidation, how about a message?

  44. jonasw

    server changes caps -> server sends <message/> to local clients. for server-to-server links, it could simply close the link so that the peer re-opens it, thus getting new stream features

  45. zinid

    jonasw, closing and opening thousands of s2s will create avalanche effect

  46. jonasw


  47. zinid

    we see this on jabber.ru for examlpe

  48. zinid

    so I just think about nonza

  49. zinid

    stream-level element

  50. jonasw

    zinid, thanks for your input, I’ll post a few suggestions to the mailing list and I hope you’ll comment on them :)

  51. zinid

    sure 😉

  52. SaltyBones

    Is there a good resource on the problems with message ids?

  53. jonasw

    zinid, replied, happy to hear from you d)

  54. jonasw

    zinid, replied, happy to hear from you :)

  55. jonasw

    SaltyBones, I’m not sure

  56. SaltyBones

    I'm looking at XEP-0359 but the problems mentioned at the summit are not in there.

  57. jonasw

    maybe the summit notes as first starting point

  58. SaltyBones

    1.2 Stable IDs Discussion on entropy ensues. dwd: <stanza-ids> don't work for unique addressing because we don't trust clients to do them properly. Kev: lots of possible attacks with spoofable stanza IDs.

  59. SaltyBones

    Not verbose enough.

  60. jonasw

    SaltyBones, I think the gist of that is: we need to generate stanza IDs on the server because weak/constructed stanza IDs are a problem. but the client needs to know the stanza ID for archive operations and possibly other things

  61. zinid

    jonasw, well, dunno, for me message looks too complex

  62. zinid

    can't we just send <c/> as a nonza?

  63. jonasw

    zinid, I think RFC 6120 would slap you in the face for that ;-)

  64. jonasw

    really, it needs negotiation

  65. jonasw

    it’s a shame that we have no way for a client to send stream features :(

  66. zinid

    yeah, because some implementation may close a stream if they receive unrecognized nonzas

  67. jonasw


  68. jonasw

    I know at least one.

  69. zinid

    however, I tried to implement such behaviour (closing a stream) and got lots of problems and left the idea

  70. jonasw

    never had issues so far; did you encounter things which sent unsolicited nonzas?

  71. zinid

    alas, I've already forgotten what that was exactly 😕

  72. jonasw

    zinid, the best we could do is probably say "okay, if the client sent a presence update with caps, we can send a nonza with a caps update"

  73. jonasw

    and similarly for servers ("offered caps in stream feature -> send nonza")

  74. jonasw

    but I don’t know enough about s2s to be sure that this works

  75. jonasw

    this would definitely need a namespace bump though

  76. zinid

    well, we're talking about new caps xep?

  77. zinid

    we can do it there

  78. jonasw

    yeah, it requires a namespace bump there too ;-)

  79. zinid

    can't we just negotiate this feature?

  80. zinid

    not sure how

  81. jonasw

    negotiation will add a round-trip

  82. jonasw

    which people will react very adversely to

  83. zinid


  84. jonasw

    but please comment on-list, hoping to get more input on that

  85. zinid

    I don't know what to say

  86. jonasw

    what you said here, essentially?

  87. SaltyBones

    There are stanza-id and origin-id in XEP-0359. Somebody at the summit also mentioned "message-id" is that defined somewhere?

  88. MattJ

    They were probably referring to the id attribute, I guess?

  89. SaltyBones

    MattJ, this https://tools.ietf.org/html/rfc6120#section-8.1.3 ?

  90. MattJ


  91. SaltyBones


  92. MattJ

    The reason XEP-0359 exists is because the id attribute is controlled (and can only be trusted by) the original sender of the stanza

  93. MattJ

    It's really just designed for tracking errors, though a couple of XEPs have re-used it for other purposes

  94. SaltyBones

    The id-attribute you mean?

  95. jonasw


  96. SaltyBones

    So, I guess, stanza-id is used by MAM and origin-id is used to detect MUC reflections (that's what 0359 says anyway). And id-attribute is used for errors but the error will have the same id-attribute iiuc, right?

  97. jonasw

    yes, pretty much

  98. jonasw

    except that origin-id won’t work with all MUCs

  99. SaltyBones


  100. jonasw

    because not all MUCs may be able to reflect that ID for whatever implementation specific reason

  101. jonasw

    (just like not all mucs can reflect the message @id)

  102. MattJ

    There is only one implementation I'm aware of that doesn't (didn't?) reflect @id, and I'm not even sure of the current status of that

  103. SaltyBones

    Okay, so why does the standard not just mandate that it be reflected?

  104. MattJ

    Simple oversight I suspect

  105. Dave Cridland

    SaltyBones, Non-MUC things pretending to be MUC, in part.

  106. Dave Cridland

    SaltyBones, Also by the time people had noticed this was a problem, enough implementations didn't.

  107. SaltyBones

    So, could this be fixed by changing the standard and requiring that origin-ids be reflected?

  108. SaltyBones

    Dave Cridland, what do you mean by "non-muc things pretending to be muc"?

  109. jonasw

    SaltyBones, changing the standard doesn’t fix the implementations magically

  110. MattJ

    Dave Cridland, enough = 1? Also in the case of bridging to non-MUC, isn't it as simple as 1) if the non-MUC supports acks, wait for the ack and reflect the id or 2) reflect the id immediately?

  111. jonasw

    SaltyBones, IRC transports are "non-muc things pretending to be MUC" for example

  112. SaltyBones

    jonasw, not magically but ...

  113. Dave Cridland

    MattJ, More than one, I think. One classic MUC implementation, but quite a few transports.

  114. Dave Cridland

    FWIW, I've gone back and forth over whether MUC ought to reflect ids. It's a special case of maintaining the id on bradcast, which is itself both useful in some cases and bad in others.

  115. SaltyBones

    Why is this reflection necessary?

  116. Dave Cridland

    SaltyBones, Reflection isn't - we could add in a subelement which indicated the original id.

  117. Dave Cridland

    SaltyBones, Broadcast ids are harder to work around - there's a few protocols which use id as reference.

  118. Dave Cridland

    This all said I *really* dislike having a zillion ids as subelements of stanzas when we already have an id attribute.

  119. SaltyBones

    Well, what is the subelement necessary for? Is this just as an ACK to the client?

  120. SaltyBones

    What do you mean by broadcast-id?

  121. MattJ

    SaltyBones, if you're asking why MUC reflects messages to the sender in the first place (some systems, e.g. IRC don't) - it has a few benefits

  122. MattJ

    Such as ensuring everyone in the room sees the same messages in the same order

  123. jonasw

    it also is natural as soon as multiple clients are in the play, somewhat like message carbons

  124. Dave Cridland

    SaltyBones, So the id attribute in the message stanza for a groupchat message sent to the MUC can be reused in the reflected message, and/or the broadcast messages to other participants in the group.

  125. MattJ

    In IRC if two people send a message at the "same time", the messages will be shown in a different order on both their clients

  126. MattJ

    and yes, multiple clients from one user in the room

  127. jonasw

    it allows MUCs to do fancy things to messages and still have everyone see the same thing

  128. Dave Cridland

    (As a side-note, Matrix achieves a similar goal by a complex hash chain)

  129. MattJ

    SaltyBones, and also servers modifying messages (e.g. in the Prosody MUC we automatically convert long multi-line messages to a pastebin link). The reflection allows the sending client to see what everyone else sees.

  130. jonasw

    (in contrast to IRC, where you don’t see that your message was truncated)

  131. SaltyBones

    Okay, pretty convincing...

  132. MattJ

    Dave Cridland, and as many people at FOSDEM noted, much RAM

  133. Dave Cridland

    MattJ, Yes but BLOCKCHAIN.

  134. jonasw

    #bingo #triggered

  135. SaltyBones


  136. Dave Cridland

    I hope there's a ZWJ there.

  137. SaltyBones

    *Yes, there is.*

  138. SaltyBones

    Okay, so we want reflected messages. Sounds fair. Let's suppose for a second that we can just make them mandatory in the standard and people would add them.

  139. jonasw

    they *are* mandatory

  140. jonasw

    the issue is with message IDs

  141. jonasw

    clients need to be able to recognize the reflection

  142. SaltyBones

    You mean id-attribute?

  143. jonasw

    or any ID really

  144. jonasw

    id attribute would be easiest

  145. Dave Cridland

    We could indicate in disco if ids were stable.

  146. jonasw

    wasn’t that proposal rejected-ish?

  147. SaltyBones

    Okay, 1. is the origin-id used for anything else and 2. is the whole point of origin-id not to detect reflection?

  148. jonasw

    speaking of MUC, I think the vote on https://github.com/xsf/xeps/pull/559 expired, didn’t it?

  149. Ge0rG

    I've attemtped to fix MUC reflected IDs some years back, not even in the normative language but merely by fixing the examples

  150. Ge0rG

    And got significant flack for it.

  151. Ge0rG

    I also attempted to make origin-id == message-id, but it was refused by the XEP author.

  152. SaltyBones groans.

  153. Ge0rG

    Now I'll just sit and wait, and ocassionaly proclaim: *I told you so!*

  154. SaltyBones

    So, is the point of origin-id to be used for reflection or is there something else?

  155. Flow

    SaltyBones, that is one use case

  156. SaltyBones

    Ge0rG, great, if you could also occasionally chime in with reasons for things and explanations that would be awesome.

  157. SaltyBones

    Flow, like?

  158. Ge0rG

    SaltyBones: it's mainly for reflection, except that MUCs are not guaranteed to keep non-body elements in the reflection

  159. Flow

    other use cases include finding your own messages in a archive

  160. Ge0rG

    SaltyBones: https://wiki.xmpp.org/web/XEP-Remarks/XEP-0045:_Multi-User_Chat#Matching_Your_Reflected_Message

  161. SaltyBones

    Flow, MAM uses stanza-id, right?

  162. Flow

    SaltyBones, it does

  163. Ge0rG

    SaltyBones: yes, but you don't know the stanza-id for the messages you sent

  164. SaltyBones

    Ge0rG, would you say this is a requirement by nature or just a problem with implementations?

  165. daniel

    i agree that the xep should specify that the sender should set origin-id=message-id

  166. Ge0rG

    SaltyBones: what?

  167. Ge0rG

    daniel: MUST set.

  168. SaltyBones

    Ge0rG, the fact that MUCs don't reflect non-body attributes.

  169. daniel


  170. Ge0rG

    because anything different is just a path into insanity

  171. daniel

    (i didn't mean SHOULD)

  172. SaltyBones

    Ge0rG, also, why do I need the stanza-id for a message I sent?

  173. Ge0rG

    SaltyBones: have a look at biboumi, a modern IRC transport. It doesn't reflect your message ID, it doesn't reflect any non-body elements and it mangles multi-line messages

  174. Ge0rG

    Welcome to message tracking hell.

  175. daniel

    i think thinks will already break if some client doesn't do that and then expects a delivery reciept or something

  176. Ge0rG

    SaltyBones: because when you ask for a MAM sync, you'll get yout sent messages copied to you as well

  177. daniel

    when sending messages to Conversations i mean

  178. Ge0rG

    Yes, message correction, delivery receipts and anything else that references IDs is a mess already.

  179. SaltyBones

    Ge0rG, and why is that a problem? They should have their origin-id and be recognizable, right?

  180. Ge0rG

    SaltyBones: you can't rely on origin-id being there, and you can't rely on it matching the message-id.

  181. SaltyBones

    daniel, what breaks when a client does what?

  182. Ge0rG

    SaltyBones: but yes, you can match MAM archives based on origin-id

  183. daniel

    if an origin id is set Conversations will use that as a reference in the receipt

  184. daniel

    so if your client doesn't do id=origin-id and expects the receipt for the id it won't work

  185. daniel

    same with everything else that references ids

  186. SaltyBones

    So, there seems to be consensus at least in this MUC right now that attribute-id should be the same as origin-id?

  187. daniel

    i'd argue there is no good reason not do make this a MUST

  188. daniel

    i mean most client would naturally do this anyway

  189. SaltyBones

    Ge0rG, who was the xep-author who was against this and do you remember why?

  190. jonasw

    SaltyBones, Flow

  191. daniel

    because why would you generate a second id if you can reuse the existing one

  192. zinid

    Who is generating origin-id? A client?

  193. Ge0rG

    SaltyBones: https://mail.jabber.org/pipermail/standards/2017-September/033415.html

  194. jonasw

    zinid, the original sender

  195. daniel

    zinid, yes

  196. jonasw

    i.e. most of the times a client

  197. daniel

    note that muc rewrites a irrelevant to the scenerio described above

  198. Dave Cridland

    jonasw, #559 expired, yes.

  199. zinid

    This is to track its own messages?

  200. daniel

    zinid, yes

  201. daniel

    or references to that

  202. Dave Cridland

    jonasw, But with no veto and the rest are +1, so apply it.

  203. SaltyBones

    jonasw, what is this ID-rewriting MUC shit? :)

  204. daniel

    some mucs are known to change the 'attribute' id

  205. Ge0rG

    SaltyBones: have a look at examples 44 and 45 in https://xmpp.org/extensions/xep-0045.html#message

  206. daniel

    so tracking your own message are parsing references don't work

  207. zinid

    daniel: why, you can rely on origin-id in this case

  208. Ge0rG

    If we have a MUC that rewrites message IDs, can we mandate that it also MUST rewrite references in all XEP payloads that reference message IDs, please?

  209. Ge0rG

    zinid: https://wiki.xmpp.org/web/XEP-Remarks/XEP-0045:_Multi-User_Chat#Matching_Your_Reflected_Message

  210. daniel

    zinid, in the case of muc? because most mucs won't remove child elements from the stanza

  211. zinid

    I'm lost

  212. zinid

    daniel: but that's exactly what's needed

  213. daniel

    now i'm lost :-)

  214. zinid

    So you can fetch origin-id and check if this is your id

  215. zinid

    Why do you need to rely on message-id if you inject origin-id

  216. daniel

    i just said the sender should set id=origin-id

  217. SaltyBones

    zinid, I think there was a sub-discussion about forcing message-id = origin-id

  218. daniel

    and/or deal with delivery receipts that reference the origin-id instead of the id

  219. daniel

    which is made easier if you id=origin-id

  220. zinid

    Ok, I don't get it 🤔

  221. daniel

    doesn't matter. that client to client stuff anyway :-)

  222. zinid

    From what I understand we just need to ditch IRC transports

  223. zinid


  224. zinid

    That's really their problem

  225. daniel

    oh yeah. or make irc transports track the messages themselves and re-add origin-id and stuff on reflection

  226. daniel

    it's probably better if the irc transport does the right thing(tm) than letting each and every client figure it out

  227. zinid

    daniel: agreed, nobody said writing transport is simple

  228. daniel

    irc transport should also reassemble the message if they previously split it and stuff like that

  229. daniel

    because they know best if they did

  230. SaltyBones


  231. SaltyBones

    Okay, are there any actual problems with the current state of three IDs except that we don't like having so many?

  232. daniel


  233. daniel

    read what i said before

  234. SaltyBones

    Uh, assuming a well-behaved server?

  235. daniel


  236. daniel

    my argumentation has nothing to do with servers

  237. jonasw

    daniel, IRC doesn’t even reflect

  238. jonasw

    so the transport generates the reflections on its own

  239. SaltyBones

    Okay, can you please point out what you are referring to then?

  240. jonasw

    biboumi however decided to reflect the split version of the message, and there’s some argument to doing that

  241. daniel

    assume i sent: <message id="1"><body>hi</body><request/><origin-id="2"></message> and then i will receive from conversations: <message><receipt id="2"></message>

  242. zinid

    who cares about IRC, really, are we designing a protocol for old nerds?

  243. daniel

    but we are talking about two different things. the how should irc transports behave has noting to do with the problem i'm describing

  244. daniel

    this applies to 1:1 chats as well

  245. jonasw

    daniel, yes, I don’t argue that

  246. jonasw

    I’m just replying to your message from "13:49:15Z" because I was away from keyboard.

  247. daniel

    the irc transport with the name i can not pronounce or remember does a few things that i don't like :-)

  248. zinid

    daniel: if we don't care about IRC then we probably don't need origin-id and thus there is no the problem you just described

  249. daniel

    there are non transport mucs which also rewrite the id

  250. Ge0rG

    daniel: https://lab.louiz.org/louiz/biboumi/issues/3283

  251. daniel

    zinid, if it weren't only for transports i wouldn't care

  252. daniel

    Ge0rG, is that an issue to change the name to something i can remember?

  253. daniel

    or pronounce?

  254. zinid

    daniel: if they are abandoned, then I personally give zero fucks

  255. Ge0rG

    daniel: no, it's about maintaining IDs on reflection

  256. Ge0rG

    zinid: everything in XMPP is abandoned. Now stop giving fucks.

  257. daniel

    i should probably write my own irc transport

  258. edhelas

    biboumi is not good ?

  259. Ge0rG

    daniel: you should just patch biboumis two or three warts.

  260. daniel

    i haven't used irc though ever since counterstrike 1.6 came out

  261. zinid

    Ge0rG: not everything, so I still have a few fucks to give

  262. daniel

    there are a number of things i don't like that seem to be design decisions rather than bugs

  263. daniel

    so i'm not sure if they even want me to change them

  264. SaltyBones

    daniel, so regarding your example, what you are saying is that the read receipt might reference the message-id or the origin-id and it is not properly specified which?

  265. zinid

    Ok, whatever, so what do you guys think about muc subscriptions?

  266. SaltyBones

    Or are you getting at the fact that origin-id requires "by=" and is therefore sometimes not applicable?

  267. daniel

    > daniel, so regarding your example, what you are saying is that the read receipt might reference the message-id or the origin-id and it is not properly specified which? This

  268. daniel

    not just read recepits but everything that references something

  269. daniel

    but yes

  270. zinid

    daniel: isn't there a place in the xep which says you should copy the message-id?

  271. daniel

    zinid, no.

  272. zinid


  273. daniel

    that's what Ge0rG and I want to add to the XEP

  274. daniel

    that's pretty much what the entire discussion is about :-)

  275. zinid

    So add it 😀

  276. daniel

    i can't without permission of the author

  277. zinid

    No shit?

  278. SaltyBones

    So, there is some problem that I am still not aware of....

  279. SaltyBones

    Flow, why is the by attribute in the origin-id?

  280. zinid

    Who's the author?

  281. SaltyBones

    The one that causes the privacy problems...

  282. Kev

    Not entirely true, BTW, that it's impossible to do things without the author. But it's the path of least resistance.

  283. daniel

    i don't think there is a by attribute in the origin id

  284. SaltyBones

    Ah, sorry that is only for stanza IDs. origin-id does not have by.

  285. zinid

    Peter is the author

  286. SaltyBones

    That might almost always be correct buy 0359 is Florian Schmaus

  287. SaltyBones

    That might almost always be correct but 0359 is Florian Schmaus

  288. zinid

    > In addition, it SHOULD include an 'id' attribute that echoes the 'id' attribute of the content message.

  289. zinid

    And you want this SHOULD to be a MUST?

  290. SaltyBones

    zinid, what some people here want is that origin-id = message-id

  291. SaltyBones

    Ge0rG, you had an objection to that in https://mail.jabber.org/pipermail/standards/2017-September/033415.html but I don't quite understand it. Why do you want to know if somebody creates strong message-id?

  292. zinid

    SaltyBones: but the sender controls this itself, so what is a problem to set them equal?

  293. daniel

    zinid, we want wording that tells the client developer to do this

  294. daniel

    there is no 'problem'

  295. Ge0rG

    SaltyBones: I thought it would matter, but it doesn't, because there always can be malicious entities

  296. zinid

    daniel: if they don't then they only harm themselves, I guess

  297. Ge0rG

    zinid: no, they harm the other participants

  298. zinid

    Ge0rG: ah

  299. SaltyBones


  300. zinid

    Yeah, how?

  301. Ge0rG

    zinid: also, it's well possible that stanza ids are generated by the xmpp library, and origin ids by the client, causing a mismatch

  302. Ge0rG

    By making all message references break

  303. Ge0rG

    We had that above already.

  304. SaltyBones

    Before we move to stanza-ids....

  305. SaltyBones

    There is nothing wrong with requiring origin-id = message-id?

  306. zinid

    If we require this then why origin-id is needed, wtf?

  307. SaltyBones

    zinid, it's possible that it was a mistake

  308. SaltyBones

    daniel, does requiring origin-id = message-id actually solve your problem or are there cases when stanza-id might be used to refer to a message instead??

  309. daniel

    It solves my problem

  310. SaltyBones

    daniel, wouldn't it be better to just mandate that clients always use the origin-id to avoid the problem of dealing with id-rewriting MUCs?

  311. SaltyBones

    I mean, certainly removing an unnecessary id would also be desirable but this might be a good intermediate step.

  312. SaltyBones

    Would everything be better if clients would generate proper stanza-ids?

  313. SaltyBones

    For ...uh...some definition of proper that makes them UUIDish.

  314. zinid

    they should be strictly monotonically increasing, so we don't need XEP-0198

  315. SaltyBones

    And by "would everything be better" I specifically also mean, wouldn't that allow us to ditch the other IDs?

  316. zinid

    and not UUIDs

  317. jonasw

    zinid, "strictly monotonically increasing" is not gonna happen

  318. zinid


  319. SaltyBones

    zinid, would increasing IDs really solve all the same problems as stream management? Seems unlikely?

  320. jonasw

    zinid, also, increasing IDs have security implications

  321. jonasw

    or rather: predictable IDs

  322. zinid

    SaltyBones, it solves in data replication, but of course XMPP is too unique

  323. zinid

    version vectors are based on such ids

  324. zinid

    jonasw, what security implications?

  325. jonasw

    zinid, I think there were some IQ response injection attacks based on predictable IDs

  326. jonasw

    even though in that case you probably already made the mistake of not verifying the sender of the response

  327. jonasw

    error injections would most likely work though because errors can come from entities different than the original recipient

  328. zinid

    jonasw, this can be fixed by adding routing information

  329. SaltyBones

    jonasw, maybe this is too much of an assumption for all of XMPP but don't we have transport encryption?

  330. zinid

    we anyway need this functionality already

  331. jonasw

    SaltyBones, that doesn’t stop me (jonas@zombofant.net) from sending an iq type="error" id="whateverIguessed" to="you" to break whatever you were doing

  332. SaltyBones

    zinid, I agree, if this is a problem I don't see why it cannot be exploited right now by somebody who randomly sees the message first

  333. jonasw

    SaltyBones, if I can guess the ID, I can attack you from off-path

  334. jonasw

    I can’t do that when I can’t guess the ID

  335. jonasw

    if I’m on path, you’re right, everything is lost already in XMPP.

  336. zinid

    but if you cannot match incoming errors against requests you sent then you should really consider to change the job

  337. jonasw

    zinid, how’d you match incoming errors against requests other than the ID?

  338. jonasw

    you can’t really use from, because as I said, it might come from an entity you didn’t know about yet

  339. MattJ

    jonasw, no?

  340. zinid

    jonasw, you can use (from, ID) I think

  341. MattJ

    errors come from the original recipient that you addressed

  342. jonasw

    MattJ, even if an s2s error causes an error?

  343. MattJ


  344. jonasw

    oh okay

  345. MattJ

    Otherwise it would be a nightmare

  346. Dave Cridland

    jonasw, There's a "by" attribute, if I remember right, that tells you the generator/reporter of the error.

  347. SaltyBones

    So, I assume that if I run a h4xx0r server and try to send replies to messages that were not directed to me to some other server they will be discarded?

  348. SaltyBones

    And the same happens between client and server...?

  349. MattJ

    SaltyBones, by "replies", you mean error replies?

  350. SaltyBones

    Whatever magical interferring replies that jonasw was referring to :)

  351. SaltyBones

    yes, errors ;)

  352. MattJ

    For that to work you would have to know the original sent message id, and the original sender's full JID

  353. SaltyBones

    But if I did, I could?

  354. MattJ

    and then yes, you could send them whatever you wanted - it would be up to their client to match it up (or not) to one of its outgoing messages

  355. SaltyBones

    So, the statement that predictable IDs would allow me to spoof responses remains correct?

  356. MattJ

    so as said above, the client should use (from, id) to identify error responses, not just the id

  357. SaltyBones

    Because I cannot spoof "from"?

  358. MattJ


  359. SaltyBones

    Yes, okay...

  360. SaltyBones

    That's what I was looking for. :)

  361. MattJ

    id "abc123" from userA@domainA is different from "abc123" from userB@domainB

  362. SaltyBones

    And servers only accept s2s with from=...@domainA if they know that the other server is in charge of domainA?

  363. Dave Cridland

    SaltyBones, That's what they're supposed to do, yes.

  364. SaltyBones


  365. MattJ

    SaltyBones, yes, they do

  366. Dave Cridland

    MattJ, No, they don't, but I love your optimism.

  367. MattJ

    If they don't, it's a bad bug or not an XMPP server

  368. Dave Cridland

    SaltyBones, So Metre (my S2S proxy mad thing) does this very strictly, and as a result I can't join this chatroom from my personal server.

  369. Dave Cridland

    SaltyBones, Not sure if MattJ is saying that's a bad bug in this server, or if he's saying it's not an XMPP server. :-)

  370. MattJ

    Dave Cridland, for what reason?

  371. SaltyBones

    Dave Cridland, so which messages do you get that don't pass?

  372. SaltyBones

    Bunneh, version xmpp.org

  373. Bunneh

    SaltyBones: xmpp.org is running Prosody version 0.9.12 on Linux

  374. Dave Cridland

    MattJ, It's sending a response down the stream obtained by reversing the stream to/from, and not by reversing the stanza to/from. I forget the details, Zash knows.

  375. Dave Cridland

    MattJ, I don't know if Prosody would accept that or not. Metre doesn't, Openfire does (but because it knows it's multiplexed on the outbound, so weirdness.)

  376. Dave Cridland

    MattJ, For maximum fun, Openfire used to do similar "implicit" auth on outbound, but I stopped that as from 4.2.0.

  377. MattJ

    I don't really understand, but maybe Zash will enlighten me

  378. Dave Cridland

    MattJ, Metre sends traffic for d.c.n -> muc.xmpp.org over a stream for d.c.n -> xmpp.org (after adding that domain via dialback).

  379. Zash

    How far back should I read to have any idea what you are talking about?

  380. SaltyBones

    15:58 should suffice

  381. MattJ

    SaltyBones, and a time machine

  382. Dave Cridland

    MattJ, Prosody (0.9) responds to the traffic on the reverse stream, xmpp.org -> d.c.n - which it hasn't requested authorization for yet.

  383. SaltyBones

    Do you mean a timezone machine or a MAM machine?

  384. Zash

    No MAM here

  385. Dave Cridland

    SaltyBones, For your amusement and mild confusion, XMPP authorizes streams by a 3-tuple of (from-domain, to-domain, direction).

  386. Zash

    Page up button tho

  387. Dave Cridland

    SaltyBones, Well. One or more of those 3-tuples anyway.

  388. MattJ

    Dave Cridland, who hasn't requested authorization for what? You mean Prosody connects to d.c.n as 'xmpp.org' and sends stanzas from 'muc.xmpp.org'?

  389. Dave Cridland

    MattJ, Yup, that.

  390. Zash

    The thing

  391. Dave Cridland

    MattJ, Like this: DEBUG 2018-02-12T15:14:35 /home/dwd/src/Metre/src/xmlstream.cc:95 : NS296914 - G ot [399] : <?xml version='1.0'?><stream:stream xmlns:db='jabber:server:dialback' xmlns:stream='http://etherx.jabber.org/streams' version='1.0' from='xmpp.org' t o='dave.cridland.net' xml:lang='en' xmlns='jabber:server'><iq id='472-452048' ty pe='error' to='dave.cridland.net' from='xsf@muc.xmpp.org/tim@boese-ban.de'><erro r type='cancel'><not-acceptable xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'/></e rror></iq>

  392. dwd

    This time I'm joined. Timing, I guess.

  393. Zash

    Uh, what was the thing with that again?

  394. Dave Cridland

    Zash, The bug itself? Responding to inbound traffic on a multiplexed stream by reversing the wrong domain pair.

  395. MattJ

    dwd, I don't see how timing would play a part

  396. Zash

    Dave Cridland: Do you remember where we discussed this?

  397. Dave Cridland

    MattJ, I think whether I can join is dependent on what streams are open, which is dependent on the session lifetime and activity.

  398. Dave Cridland

    MattJ, Not timing as in a race condition, mind.

  399. Dave Cridland

    Zash, Erm. 1:1 messages, I think, until we figured out it definitely wasn't a security issue.

  400. Dave Cridland

    Zash, Possibly some discussion in jdev.

  401. dwd

    FWIW, I have to put in something similar to an implicit auth for X2X, anyway. So this may provide a workaround.

  402. Zash

    MattJ: https://hg.prosody.im/trunk/file/0de0018bdf91/plugins/mod_s2s/mod_s2s.lua#l203

  403. MattJ

    Zash, exactly where I ended up

  404. Zash

    stanza from/to may differ from the stream to/from in case of dialback multiplexing

  405. Dave Cridland

    MattJ, You switching to FreeBSD?

  406. Dave Cridland

    MattJ, https://svnweb.freebsd.org/base?view=revision&revision=329166

  407. zinid

    svn o_O

  408. edhelas

    gitis too mainstream

  409. zinid

    capitalist pigs use git!

  410. Zash


  411. Holger

    NetBSD has Lua in the kernel for years!

  412. Zash

    Dave Cridland: Heh, nice

  413. Holger


  414. Holger

    And it has CVS :-)

  415. zinid

    damn, I only get used to svn...

  416. Dave Cridland

    Holger, Can you download NetBSD yet, or is it still only available on tape?

  417. zinid

    Holger, cool stuff btw

  418. SamWhited

    More importantly, once I download it can I run it on any machines made after 1995? (this is always the problem I had trying to run NetBSD; a few of the ops people at work run it, but they all use *very* old machines)

  419. Dave Cridland gets prepared to explain "tape" to the younger readers.

  420. Dave Cridland

    SamWhited, Sure! It runs pretty well even modern machines like the Amiga 4000.

  421. Dave Cridland

    (Sorry, that was 1993).

  422. SamWhited


  423. Holger

    Well yes it's about as dead as XMPP :-P

  424. SamWhited

    I would like to use at *least* a P3.

  425. Holger

    But it always worked okay-ish for me on non-recent ThinkPads.

  426. Holger

    I no longer do that though.

  427. Holger

    cpu0 at mainbus0 apid 0: Intel(R) Core(TM) i3-2120T CPU

  428. Holger

    ... is the one box I still run NetBSD on.

  429. Zash

    Uh, was NetBSD the one being difficult, or OpenBSD?

  430. Holger


  431. SamWhited

    Actually, jokes aside, that might be a nice thing to do with my broken old thinkpad; doesn't work very well as a laptop anymore, but it could be a {Free,Net,Open}BSD or Illumos "desktop".

  432. Holger

    Zash: Isn't all computers difficult except for Apple?

  433. SamWhited

    This is true… ⤴

  434. Zash

    Et Apple, Holger

  435. Dave Cridland

    I find Apple incomprehensibly difficult to use, I must admit.

  436. Dave Cridland

    I've yet to figure out how to go up a directory consistently in file dialogs.

  437. Kev

    cd ..

  438. Kev

    Happy to help.

  439. Holger

    Managing FreeBSD feels more or less like Debian to me, the two others feel a bit more basic but not really harder to use. All dead simple compared to the dances you had to go through when partitioning hard disks or getting X11 to run 20 years ago with any Linux or BSD.

  440. Dave Cridland

    Holger, Ah, X11. Kids these days don't know they're born.

  441. Dave Cridland

    Kev, Don't get me started on the antiquated command line tools Apple foists upon you.

  442. Holger

    Yes, we're horribly old.

  443. Dave Cridland

    Holger, We're very experienced. I'm sure that's what you meant.

  444. Zash

    I'm sure wayland will bring back the good old times for those who miss configuring Xorg

  445. Holger


  446. SamWhited

    The only thing I want out of a machine is to sync an old ipod without Rhythmbox crashing…

  447. Dave Cridland

    SamWhited, Rhythmbox never crashes.

  448. Dave Cridland

    SamWhited, At that speed it's called "parking".

  449. SamWhited

    Rhythmbox is one of those people who parallel parks by backing up until they hit the car behind them, then driving forward until they hit the car in front.

  450. SaltyBones

    So, why is it that a client cannot create the stanza-id?

  451. SaltyBones

    Is this a thing about malicious clients or what's going on there?

  452. jonasw

    SaltyBones, no, unaware clients are sufficient; colliding IDs would lead to interesting issues with MAM

  453. SaltyBones

    jonasw, but if the client is only unaware the server could just return an error informing the client that this ID has been used.

  454. Dave Cridland

    SaltyBones, But we could use, say, stream-id + attribute-id to form a stable, non-colliding reference identifier.

  455. jonasw

    SaltyBones, that’d be annoying to handle in the client, and the server would need an O(1) (or similar) way to determine that the ID has been used already...

  456. SaltyBones

    Yeah, or we could hash things and god knows what else...it seems very solvable...

  457. Dave Cridland

    SaltyBones, I mean, unless the client isn';t generating unique ids within its stream, in which case it's presumably not going to be fixed to use some other identifier.

  458. jonasw

    Dave Cridland, stream id + sm-counter?

  459. jonasw

    that’s verifiable by the server

  460. jonasw

    and predictable for both

  461. Dave Cridland

    jonasw, True. But I think we had some ideas on predictability outside the stream?

  462. SaltyBones

    throw a hash on top to avoid privacy issues, done

  463. SaltyBones

    Dave Cridland, I thought we dicussed those into oblivion earlier? :)

  464. Dave Cridland

    SaltyBones, Really I've lost track.

  465. jonasw

    Dave Cridland, hmmm ... hmac(stream-id, sm-counter)?

  466. Dave Cridland

    SaltyBones, I have the feeling that nobody quite knows the entire picture here.

  467. SaltyBones

    I almost lost track and I basically didn't work at all today and just discuss here. :p

  468. Dave Cridland

    jonasw, HMAC() requires a secret to be of any use.

  469. jonasw

    Dave Cridland, stream-id.

  470. SaltyBones

    Seems reasonable on first sight...

  471. jonasw

    (post-TLS obviously...)

  472. Dave Cridland

    jonasw, Is that secret? And why HMAC over a hash?

  473. SaltyBones

    Oh, that would leave out the hash...

  474. jonasw

    Dave Cridland, I’m fine with hash(stream-id || sm-counter) too, but that’s nearly an hmac ;-)

  475. SaltyBones

    hmac basically IS a hash

  476. SaltyBones

    yeah :)

  477. Dave Cridland

    jonasw, I'm pretty sure it's not. :-)

  478. SaltyBones

    I am very sure it is. :)

  479. jonasw

    that’s why I said "nearly"

  480. jonasw

    I think the concat is slightly different

  481. SaltyBones

    (yes, nearly)

  482. SaltyBones


  483. jonasw

    doesn’t matter anyways

  484. Dave Cridland

    jonasw, An HMAC is two nested hashed with concats and masks, yes.

  485. SaltyBones

    Okay, so we only disagree on our definition of nearly. ;)

  486. Dave Cridland

    jonasw, But the security properties are distinct.

  487. jonasw

    Dave Cridland, I don’t argue that

  488. jonasw

    (and I’m aware)

  489. SaltyBones

    Anyway, the suggestion of HMAC(key=stream-id, msg=sm-counter) seems good, doesn't it?

  490. Dave Cridland

    In any case, given a stanza with a known id on a given stream, we clearly want to be able to predict the MAM id.

  491. SaltyBones

    I mean, I am also fine with SHA-*(stream-id || counter-sm)

  492. SaltyBones

    Dave Cridland, but mam-id = stanza-id which is what I am proposing to set to this...

  493. Dave Cridland

    Question is, do we think the MAM id is likely to become public, and if so, can someone relatively easily figure out the next (or previous) value, and then do Something Bad?

  494. SaltyBones

    Dave Cridland, that might be the question bit if it only costs us a hash per message to not answer it might also not be...

  495. SaltyBones

    Dave Cridland, that might be the question but if it only costs us a hash per message to not answer it might also not be...

  496. SaltyBones

    Dave Cridland, that might be the question but if it only costs us a hash per message to not answer it, it might also not be...

  497. Dave Cridland

    What I'm wondering, you see, is whether we give clients an algorithm to generate attribute-id and then let them signal to the server that they're going to use globally-unique attribute-ids and the server is allowed to call them out if they don't.

  498. jonasw

    Dave Cridland, okay, right, this is about the MAM ID. this *probably* doesn’t matter, but if the goal is to consolidate all IDs at some point, choosing a way where the ID can become public is safer

  499. Dave Cridland

    jonasw, That's roughly my thought, yes.

  500. jonasw

    and at that point, HMAC(…) seems like a sane choice.

  501. Dave Cridland

    jonasw, Well, that and "But we *HAVE* stanza ids - right there in the stanza!"

  502. Kev

    How would the server know that they didn't use a globally unique id?

  503. SaltyBones

    Kev, if the server can simply re-run the generation algorithm and compare results..?

  504. jonasw

    Kev, if the ID doesn’t follow the algorithm for a globally (predictable, for the server and client only) unique ID

  505. Dave Cridland

    Kev, Well, it'd know if they used no id at all, and if it saw a collision within a stream.

  506. Kev

    If we only cared about collisions within a stream, we could trivially solve all issues already.

  507. SaltyBones

    And that is how you can tell that Kev is a VIP

  508. SaltyBones

    when he says something three people reply!

  509. Dave Cridland

    Kev, Well, we care about collisions for a user's account for MAM, and no further.

  510. Kev

    But having a useful id to refer to a message would be jolly useful, and I don't see how we can get there yet.

  511. SaltyBones

    Okay, so you're saying when two servers federate and you assume one of them is malicious how can the other make sure he doesn't get duplicate IDs?

  512. Kev

    Or non-malicious server with a malicious client.

  513. Dave Cridland

    SaltyBones, Do they care? ANd if not, should they?

  514. jonasw

    SaltyBones, you don’t use other servers stanza-ids in your own MAM

  515. SaltyBones

    Well, the malicious client would be detected by the server.

  516. Kev

    jonasw: Bloody useful if you could, though, no?

  517. Kev

    SaltyBones: How would the server know it's malicious, if it doesn't know all globally generated ids?

  518. SaltyBones

    Kev, it can simply check if the client generates their IDs correctly...

  519. Dave Cridland

    Kev, Again, why would a server care?

  520. Kev

    Without exploding the size of IDs, at least.

  521. jonasw

    (we should mandate the use of shakespeare-inspired adjectives in each bloody sentence in this room, by the way)

  522. jonasw

    Kev, how would it be useful?

  523. Kev

    I'd like to send a reference to a stanza, where the reference makes sense in my archive, the MUC/MIX archive, and the user's archive.

  524. Dave Cridland

    jonasw, I don't think "bloody" is Shakespeare. "Submarine" is, though, as far as I remember.

  525. SaltyBones

    Kev, you simply make it a hash or hmac of something that both the client and the server know and the server can check the calculation.

  526. Kev

    "simply" :)

  527. SaltyBones

    Kev, but doesn't that only involve you, the muc/mix server and $other_client? i.e. only one server...?

  528. Dave Cridland

    Kev, Is the (from, id) attributes of the stanza sufficient? If not, why?

  529. Zash

    If this is for MAM purposes, then I will cry if I can't stick some time based bits into the ID

  530. Kev

    Dave Cridland: No, because the from gets rewritten.

  531. Dave Cridland

    Kev, When?

  532. SaltyBones

    Zash, what?

  533. Kev

    In a MUC or a MIX.

  534. Zash

    My MAM ids are basically yyyy-mm-dd-random

  535. jonasw

    Kev, those are different use-cases I think.

  536. Kev

    jonasw: If we cherry-pick the trivial use cases, it's going to be trivial to solve them :)

  537. jonasw

    Kev, this is where origin-id makes more sense. It is generated by the client. If the client doesn’t ensure that there’s enough entropy in there, references to their message won’t work.

  538. SaltyBones

    Zash, and this is important because?

  539. Dave Cridland

    Kev, Ah, so given a message from a MUC fanout, do you want to be able to identify the originating id? I'm not actually sure you always want to allow that.

  540. SaltyBones

    I still don't get it. Why does from get rewritten?

  541. SaltyBones

    And what does that have to do with anything? :D

  542. jonasw

    SaltyBones, when you send a message from saltybones@yourdomain.example/client-resource to this MUC, we see the message from xsf@muc.xmpp.org/SaltyBones

  543. jonasw

    so the from is being rewritten.

  544. jonasw

    and if references are based on (from, id) pairs, you can’t use the same reference we all can use for your message

  545. jonasw

    (simplified, of course we could still refer to the reflection of the message, but that would break down with MIX, too, I think)

  546. Kev

    jonasw: Well, no, because if you can fake someone else's id, suddenly you can maliciously change the target of a reference.

  547. jonasw

    Kev, good point. so we indeed need something like (from, id) :(

  548. SaltyBones

    Wait what?

  549. zinid


  550. SaltyBones

    How can you now fake stuff again?

  551. jonasw

    but given that all our fanouts and from-rewriting things do actually do reflections, I’m not sure if that’s so bad, Kev?

  552. jonasw

    your local archive woudl still be able to resolve everything

  553. Kev

    jonasw: Well, it enforces a round-trip before you can refer to or correct anything, which isn't ideal.

  554. jonasw

    Kev, you normally know the "from" you’ll be having.

  555. jonasw

    and since you set the origin-id yourself, you already know everything you need

  556. jonasw

    (meh, races with server-enforced nickname changes in MUC)

  557. Kev


  558. jonasw

    (but I guess those will be rare enough for us not to care)

  559. SaltyBones

    halp! :(

  560. Kev

    SaltyBones: Because there are potentially malicious entities.

  561. SaltyBones

    Are these only clients or servers as well?

  562. Kev

    jonasw: Still doesn't help where a user generates the same id twice, I think, and you're left with ambiguous references/corrections/whatever.

  563. Kev

    SaltyBones: Both.

  564. SaltyBones

    So why do you connect to a malicious server and expect things to work?

  565. SaltyBones

    Maybe that's not a good approach.

  566. jonasw

    Kev, that’s not an issue. put 256 bits of entropy in there and you are officially allowed to not care

  567. Kev

    Ah, if you have a mechanism for identifying malicious intent on the Internet, you could be a very famous person.

  568. jonasw

    Kev, and if we want to have defined behaviour in that case, always assume the most-recent message

  569. Kev

    jonasw: That only works for accidents, rather than manipulation, no?

  570. SaltyBones

    Kev, but you cannot manipulate as long as the server is checking.

  571. jonasw

    Kev, indeed

  572. jonasw

    but if you manipulate your own origin-ids, it’s your own fault?

  573. SaltyBones

    Of course if we have malicious servers now a bit more work is required. :)

  574. Kev

    And I'm concerned that 'most recent' falls apart if you can manage different people receiving different subsets.

  575. jonasw

    can one inflict harm by producing duplicate origin-ids?

  576. jonasw

    I’m not convinced that this is actually an issue.

  577. Kev

    Most likely.

  578. Kev

    If I can cause the 'wrong' message to get corrected.

  579. jonasw

    maybe with something like Reactions based on (from, origin-id) references.

  580. Kev

    Or a like to apply to the wrong message.

  581. Kev

    If I can manipulate you into liking a Godwin instead of something sensible, that's not ideal.

  582. jonasw


  583. SaltyBones

    But messages aren't authenticated anyway, if you are a malicious server you can claim to have received whatever likes from me...

  584. jonasw

    Kev, then the only way is a MUC/MIX stanza ID generated by the MUC/MIX and waiting for a round-trip before references can be done.

  585. Kev

    SaltyBones: Other way around.

  586. jonasw

    SaltyBones, all it needs for now is a malicious client though

  587. jonasw

    SaltyBones, or a malicious client+server pair if we do the hmac-stanza-id thing for origin-id

  588. jonasw

    I can still run my own server and make it fake origin-ids

  589. zinid

    your own malicious server?

  590. jonasw

    Kev, I’d however argue that actively, from a client/own server side, make participants in a MUC/MIX receive only parts of the conversation would be rather trikcy

  591. jonasw

    and targeted parts even

  592. jonasw

    and that without them suspecting that things are broken in funny ways and thus not trusting the system anymore

  593. Kev

    jonasw: Ah, that's ok then, no exploits have ever involved doing anything tricky :)

  594. jonasw

    Kev, I see your point, but I’m not sure this is actually something which is reasonably exploitable.

  595. jonasw

    but yes

  596. jonasw

    unless we let the ID be generated by the fanout service, there’s no way to be sure I’m afraid

  597. jonasw

    I mean, I don’t have an issue with that, I’m fine with that even.

  598. Kev

    Yes. I don't see a way of solving this, which was why I brushed it under the carpet at the Summit.

  599. jonasw

    complicates things though

  600. jonasw

    Kev, I’m having a really hard time constructing a successful attack which wouldn’t be seen by the victim in the MUC case

  601. jonasw

    even when I omit arbitrary messages

  602. jonasw

    to only some participants

  603. jonasw

    (which would be really really tricky to achieve targetedly I think)

  604. jonasw

    ah, now I have it

  605. jonasw

    this is the scenario: 1. user A, "bad statement", origin-id=1 2. user A, "good statement", origin-id=1 3. user B, like (from=user A, origin-id=1) if (2) isn’t seen by all users, they see user B liking "bad statement" instead of "good statement"

  606. jonasw

    with an arbitrary amount of messages between (2) and (3), it is also not too difficult to make people not see (2) in a MAM-less MUC.

  607. Kev


  608. jonasw

    I don’t think there’s a way unless the MUC does things there

  609. jonasw

    (i.e. if the MUC generates the ID?)

  610. SaltyBones

    But, all it requires to prevent that is for the MUC to check that the ID is unique just like the assumed-non-malicious server did in our previous discussionn...

  611. jonasw

    (i.e. if the MUC generates the ID)

  612. jonasw

    SaltyBones, but that is a rather expensive check to do

  613. jonasw

    you need to record IDs for all eternity, or have a defined way to generate the ID which can be executed by clients

  614. jonasw

    the latter would be tremendously tricky to get to work synchronously

  615. jonasw

    but possible...

  616. SaltyBones

    Which is a little more tricky because there is no obvious common "secret" like the stream-id but a simple Hash("counter") would do

  617. jonasw

    something like hash(message-counter || presence-id), where presence-id is an ID assigned to the client on join

  618. SaltyBones

    Kev, not that in this case I used the word simple again but I was totally lying...

  619. SaltyBones

    Kev, note that in this case I used the word simple again but I was totally lying...

  620. jonasw

    hash counter doesn’t really work with multi-session nicks I think, it would lead to collisions or races

  621. SaltyBones

    jonasw, yeah it was just to get the idea. Indeed you would at least need some salt.

  622. SaltyBones

    But if the server gives you the salt on join and you use that to generate the IDs it should be good again.

  623. jonasw

    that would be verifable by the MUC and the MUC would drop stanzas which do not adhere to this schema

  624. jonasw

    (or rather, reject)

  625. SaltyBones

    and since the counter can be per-muc there is no issue with having it

  626. jonasw

    per client even

  627. SaltyBones

    I think :p

  628. jonasw

    Kev, ^

  629. SaltyBones

    well, yes per client and per muc

  630. SaltyBones

    just saying it doesn't seem to be a privacy issue

  631. jonasw

    SaltyBones, yeah

  632. jonasw

    so origin-id = one-way-function(presence-id, message-counter), where presence-id is assigned on MUC join

  633. jonasw

    only need to define what happens when message-counter wraps around or becomes large or something

  634. jonasw

    (same thing for SM by the way, SM has wraparound semantics)

  635. SaltyBones


  636. SaltyBones


  637. jonasw

    SaltyBones, itym "y u have so long sessions"

  638. SaltyBones

    still, if this is defined somewhere we can probably renegotiate the salt at that point...

  639. jonasw

    while wrapping around a 64bit counter in a single session is sure challenging, we need to be prepared if this is to be solid :)

  640. jonasw


  641. zinid

    I hear counter?

  642. zinid

    sorry, I don't track the discussion

  643. jonasw

    zinid, I’m not going to repeat everything you can read in the backlog :)

  644. zinid

    as you wish

  645. zinid

    not that I care

  646. SaltyBones

    counter works in this case because you can restart counting when rejoining the muc

  647. zinid

    whatever you will end up will be shit anyways, so

  648. zinid

    you already created 4 fucking ids: stanza-id, origin-id, attribute-id and SM id

  649. zinid

    maybe it's time to stop and think?

  650. jonasw

    what is an sm ID?

  651. zinid


  652. jonasw


  653. jonasw

    zinid, all of this is about reducing the number of IDs, so I thnik this is kinda what we’re doing?

  654. zinid

    jonasw, from what I see you want to add yet another counter

  655. jonasw

    zinid, to generate origin-ids, yes

  656. jonasw

    for MUCs and MIXes

  657. jonasw

    to make them verifiable by the service

  658. jonasw

    zinid, or rather, we’re replacing origin-id by some one-way-function(counter)

  659. zinid

    how this will cover sm id?

  660. SaltyBones

    We haven't discussed sm-id, yet. Only the other three.

  661. SaltyBones

    I actually have no clue what sm-id is. :)

  662. zinid

    SaltyBones, then I need to wait while you recognize the problem 😉

  663. SaltyBones

    zinid, so reducing 4 -> 2 is not enough, eh? :p

  664. jonasw

    zinid, sm-id stays, but stanza-id (and attribute-id) could potentially become one-way-function(stream-id, sm-id)

  665. zinid

    2 is enough, however I think first should be counter and second should be routing information, and not an ID

  666. jonasw

    stanza-id being used for the archive only, origin-id being used throughout the network together with the sender address for refernecing a specific message

  667. zinid

    what ID will be used to sync?

  668. zinid

    with SM IDs you just create a pointless queue in c2s state

  669. SaltyBones

    Okay, I think we are trying to solve a problem that is very orthogonal to stream management. But we have only discussed this a bit and it might very well not work even for what we want it to do.

  670. zinid

    SaltyBones, I don't think SM should be separated from archive

  671. zinid

    it's pointless

  672. SaltyBones


  673. jonasw

    zinid, stanza-id is used for ysnc

  674. jonasw

    (for MAM sync that is)

  675. zinid

    SaltyBones, because why you need this separation?

  676. zinid

    SaltyBones, you keep messages in MAM and then put them in c2s SM queue

  677. zinid

    why can't you just inc(counter), store in MAM and send via c2s?

  678. SaltyBones

    Why are you asking me that? I am 100 % sure that you know more about SM than I do!

  679. zinid

    what I know about SM is that I need to maintain stupid queues inside c2s processes, which sucks as hell

  680. zinid

    even though a client can request those messages via MAM

  681. zinid

    if you receive an ID out of order, you just reconnect and ask everything started from the ID you received

  682. zinid

    and server will send you this from archive

  683. zinid

    and vice versa

  684. SaltyBones

    But SM-IDs are per-stream not per-message, right?

  685. SaltyBones

    At least that's what it looks like in https://xmpp.org/extensions/xep-0198.html

  686. zinid

    when you connect a server, you provide last seen ID and it will resend you everyting what's greater than this ID

  687. zinid

    SaltyBones, yes, they are totally separated instances

  688. SamWhited

    What if the thing you missed was an IQ or a presence that isn't stored in MAM?

  689. SamWhited

    If you temporarily disconnect and miss something, SM acks allow you to find out. Doesn't help as much if it only covers messages.

  690. zinid

    SamWhited, I think we can drop them

  691. zinid

    SamWhited, we already don't care about IQs with Push, so why would we start care?

  692. zinid

    try to make jingle call when I'm in "push" mode

  693. SamWhited

    Does that not work? That does seem like something we need to care about to me

  694. zinid

    SamWhited, but we don't and we cant with push stuff

  695. jonasw

    shouldn’t an IQ trigger a push? :-O

  696. zinid

    SamWhited, what if I want to receive your software version and you're in push mode?

  697. SamWhited

    That seems like a problem that needs fixing then, not an example of something being done right that we should copy.

  698. zinid

    anyway, you can keep IQs in MAM if you prefer

  699. zinid

    you need to keep subscription requests for sure there

  700. zinid

    we keep them already, but in a separate database due to historic reasons only

  701. jonasw

    IQs (which are inherent request-response, with exactly one response) in MAM sounds like a terrible idea.

  702. SamWhited

    yah, now you have to try and figure out which IQs are time sensitive, which are important to store, etc. it seems like a comlicated way to work around having a stanza counter.

  703. zinid

    SamWhited, but you have to decide now too: for example, after 5 minutes of inactivity (by default) a server just bounce all IQs from SM queue

  704. zinid


  705. SamWhited

    I'm not saying it's perfect, just that this doesn't seem like a good fix.

  706. zinid

    a fix? the behaviour is the same

  707. zinid

    but we let a client to decide which IQ to reply

  708. SamWhited

    Except now you have to store all stanzas in MAM, or not store some stanzas and risk those being the ones that are dropped and you don't know you missed something. The point of SM is to make sure you know if you lose something, which is often important.

  709. zinid

    do you really want to loose incoming call?

  710. zinid

    that's how it works now: if somebody calls you and you're not connected you loose any track

  711. SamWhited

    As I said, I'm not claiming that the current solution is good; just that this makes it worse.

  712. zinid

    no, it makes it better

  713. zinid

    you just need to define what to store and what not, well, it's too much to redesign, yes

  714. SamWhited

    If you don't store everything you now lose the ability to detect connection drops though

  715. zinid


  716. zinid

    not sure how will you address missing call though

  717. zinid

    with your approach

  718. SamWhited

    I do not have an approach; I agree that is a problem that needs solving though.

  719. SaltyBones

    It seems like we need storage for management messages and storage for actual messages. I don't see why these should be mixed up.

  720. zinid

    we need to decide what to store and what not

  721. SaltyBones

    hm..that distinction was wrong, yeah

  722. SaltyBones

    there should be a storage until delivered and an optional long term storage

  723. SaltyBones

    users might not even want mam :p

  724. zinid

    if you don't want mam then just lose messages

  725. SaltyBones

    Okay, let me rephrase, some users might not want long-term storage of messages...

  726. zinid

    not sure why this should be specific to any approach

  727. SaltyBones

    Because I think that's what MAM is intended for and that's why using it for all messages as you suggest is strange and might have unexpected results if people built it under the assumption that it is for long term storage...

  728. zinid

    servers can drop MAM archives on reconnect for example, what's the problem?

  729. SaltyBones


  730. zinid

    or later, when it's delivered

  731. SaltyBones

    this is just me confusing MAM and the other thing again...

  732. SaltyBones

    for the one-millionth time...

  733. Ge0rG

    What if I want my messages both on my pc and my mobile? I can't just drop MAM when my pc is online

  734. zinid

    you can if everything is delivered

  735. zinid

    anyway, what you are trying to say is a partial replication, and this problem is very hard to resolve

  736. SaltyBones

    what is the other thing again, server side archiving?

  737. SaltyBones

    zinid, does MAM know if everything was delivered?

  738. Ge0rG had a very concerning realization about guessable IDs and packet filters in smack earlier today.

  739. zinid

    SaltyBones, well, no, I think you cannot know that in general case

  740. SaltyBones

    zinid, but that is what SM does, right?

  741. zinid

    SaltyBones, well, if you introduce acks, then yes

  742. SaltyBones

    Hm.... :)

  743. Holger

    All this is orthogonal to the question whether having two separate stanza/message queues is sane. I agree with zinid that it isn't.

  744. Zash

    Two whatnow

  745. Ge0rG

    I'm saying for many years now that we need to replace 0184, 0198, 0280 and 0313 with one single proper message syncing thing.

  746. Holger

    SM is already mostly just an optimization, and I think we should fix the remaining issues to make stream management superfluous.

  747. SaltyBones

    I just want to point out that this is orthogonal to our earlier discussion about IDs :)

  748. zinid

    SaltyBones, I said "no" in general case because we have Bysantine General Problem, it's unresolvable, what we can gurarantee is sequential consistency

  749. Ge0rG

    SaltyBones: please show your ID before being allowed into this room :P

  750. SaltyBones

    Holger, does anybody already know what those issues are?

  751. Zash

    Ge0rG: Kidnap some server devs and lock yourself in a room until that one single proper message syncing thing to rule them all is properly implemented and XEP'd

  752. Holger

    SaltyBones: Syncing of outgoing messages (in a sane way) and maybe avoiding some round trips during session startup.

  753. zinid

    Zash, and watch them die 😉

  754. Ge0rG

    Zash: I'll lock you and zinid up in that room, and watch the live stream

  755. SaltyBones

    Holger, so, can we just turn off stream management and leave MAM and see what happens to find out?

  756. Holger

    Sure you can :-)

  757. SaltyBones

    Or do you think fixing up MAM is a bad idea and something new is required?

  758. Zash

    SaltyBones: That's what I did, actually. Haven't died from SM-lessness yet.

  759. Holger

    SaltyBones: I think you can already implement proper sync with MAM as-is. But something new is required to let clients implement this in a sane way, without having to de-duplicate and whatnot.

  760. zinid

    that's my point: what we need is to store messages and some other restricted stuff and call it a day

  761. SaltyBones

    Holger, so would unique message IDs solve this?

  762. SaltyBones

    I mean, at least de-duplication would be reasonably easy then, I guess.

  763. SaltyBones

    Of course some sort of counter makes more sense in this case...

  764. Zash

    SM has counters. MAM has server-issued, guarante to be unique message ids.

  765. SaltyBones

    Why are IDs not a counter?

  766. Holger

    Not sure what sort of uniqueness we need to solve what problem. Didn't read the backlog, sorry :-) The thing missing for proper sync of outgoing messages is an algorithm to compute the MAM IDs of outgoing messages.

  767. SaltyBones

    I mean the MAM-IDs....or is that the same as the stanza-ids?

  768. Holger

    (Sorry, typed this before reading the last few messages.)

  769. zinid

    Zash, now answer the question "get me last messages I didn't receive" with current SM and MAM approach 😉

  770. SaltyBones

    Holger, what do you mean by outgoing? Messages that we sent?

  771. Holger


  772. zinid

    if you say "time", then no

  773. Zash

    zinid: query after = id of last message I saw

  774. SaltyBones

    Okay, that should be solved by the hash idea....if it works :p

  775. Zash

    cry over outgoing messages sent after that

  776. Holger

    SamWhited: I need to know their IDs so I can tell the server "give me all messages since $id".

  777. SaltyBones

    But indeed if we want to query MAM by a "point in time" or "counter" unique IDs are not really the best thing :)

  778. SaltyBones

    Holger, but then the server still has to search the MAM archive for that ID and give you everything after it....

  779. zinid

    Zash, define after in distributed system 😉

  780. SaltyBones

    So a counter would be much better.

  781. Holger

    SaltyBones: Sure?

  782. Zash

    zinid: MAM archive ids on incoming messages

  783. SaltyBones


  784. Zash

    zinid: after is a MAM term

  785. zinid

    Zash, but you need some ordering

  786. Zash

    zinid: huh

  787. Zash

    zinid: XMPP streams are ordered

  788. zinid

    yes, but you need to maintain a timestamp index in the database

  789. Zash

    You've lost me

  790. zinid

    well, you're probably right that if we assume timestamp ordering then we don't need counters and SM at all

  791. Zash


  792. Zash

    In a MAM query, 'after' is a field that carries a MAM archive ID

  793. zinid

    so, what's the ordering? 🙂

  794. zinid


  795. zinid

    from what I understand you use timestamp+id, which means time ordering

  796. Zash


  797. Zash

    As I said, you lost me. I have no idea what any of us are talking about anymore.

  798. zinid


  799. Ge0rG

    https://summerofcode.withgoogle.com/organizations/?sp-page=5 no xsf?

  800. zinid

    BEAM community is there 😛

  801. moparisthebest

    SaltyBones: like 98% of what you were talking about is here https://github.com/moparisthebest/xmpp-ircd

  802. moparisthebest

    Basically it works if IRC users don't want nickserv or chanserv but they do and I never got back to it :)

  803. SaltyBones

    moparisthebest, you mean of a way to connect to a muc using the irc protocol?

  804. moparisthebest


  805. moparisthebest

    Running that makes a muc look like an IRC server to an IRC client

  806. SaltyBones

    I'm pretty sure nobody wants that. At least I don't! :D

  807. SaltyBones

    But I think it might provide an excuse for people who have to convince irc users ;)

  808. moparisthebest

    SaltyBones: yesterday you said

  809. moparisthebest

    Hm...maybe the transport should be the other way round. Offer an IRC server that connects to MUCs.

  810. moparisthebest

    That's what I was referring to

  811. vanitasvitae

    Ignite didnt make it into GSoC

  812. vanitasvitae


  813. SaltyBones

    moparisthebest, I know, I said that regarding the discussion of the KDE folks looking for an IM solution

  814. Flow

    `> * Ge0rG had a very concerning realization about guessable IDs and packet filters in smack earlier today Care to share?

  815. Flow

    Or is it just something in the ancient smack library yaxim uses?

  816. moparisthebest

    SaltyBones, right so they could use a MUC, but also have an IRC server that IRC users could use

  817. Dave Cridland

    moparisthebest, I like this, BTW.

  818. moparisthebest

    and everyone would end up in the same place, but it'd be a muc

  819. Dave Cridland

    moparisthebest, Is the GPLv3 your addition or from telepaatti?

  820. Dave Cridland

    Anyone know what servers support XEP-0288 these days?

  821. Dave Cridland

    Does this Prosody instance, for example?

  822. moparisthebest

    Dave Cridland, looks like the original telepaatti is gone, but GPLv3 goes back to at least the next fork looks like

  823. moparisthebest

    honestly I kind of abandoned it because I like rust so much more than python nowadays but can't be asked to rewrite everything yet haha :)

  824. Flow

    -xep 288

  825. Bunneh

    Flow: Bidirectional Server-to-Server Connections (Standards Track, Draft, 2016-10-17) See: https://xmpp.org/extensions/xep-0288.html

  826. moparisthebest

    I still run an IRC server I would love to rm -rf

  827. SamWhited

    > I need to know their IDs so I can tell the server "give me all messages since $id". I know, apparently I wasn't clear about something, sorry. I'm not suggesting we get rid of MAM or leave everything exactly as it is today, just that MAM doesn't cover some important parts of SM and probably can't be made to cover it without significant downsides.

  828. Ge0rG

    Flow: it's affecting the old smack for sure, I'll have a look into smack 4 and let you know

  829. SamWhited

    (sorry for the long delay, got pulled into a meeting and then was AFK for a bit)

  830. Dave Cridland greps his logs

  831. Dave Cridland

    So there's actually only one server I talk to that does bidi. That's scary.

  832. Flow

    Ge0rG, let me know if you didn't just re-discover CVE-2014-0364

  833. Ge0rG

    Flow: it's related but different

  834. Zash

    Dave Cridland: Is it mine? :)

  835. Dave Cridland

    Zash, No, Lance's. Although actually grep might have gone into Annoying Binary Mode.

  836. SaltyBones

    -I ?

  837. Dave Cridland

    -a, but yeah.

  838. Dave Cridland

    OK, so that's actually lots of servers doing bidi. Happy bunny, now. I've been putting the support into Metre.

  839. Zash

    Oh? Hm, feeling up for doing a survey?

  840. Dave Cridland

    Zash, What sort of survey?

  841. Zash

    "How many servers do 288?"

  842. Zash

    Hooooold on now

  843. Zash

    Is that the same number as ...

  844. Zash

    https://www.youtube.com/watch?v=azEvfD4C6ow !!!

  845. Dave Cridland

    Zash, Looks like it's PSYC and Prosody.

  846. Zash

    Prosody doesn't do it out of the box, you need to go a bit out of your way to install a community module.

  847. Dave Cridland

    Really? Quite a few people have, then.

  848. Zash

    Dave Cridland: Question is, how much self-selection bias is there among people that have you in their roster? :)

  849. Dave Cridland

    Zash, Lots.

  850. Dave Cridland


  851. Zash


  852. Kev

    M-Link does bidi, but we disabled it because of bug reports from Prosody about us not accepting stanzas down the right streams. Some of which I'm starting to question :)

  853. Dave Cridland


  854. dwd

    So, Metre now does Bidi.

  855. dwd

    OK, this is weird. Metre is successfully negotiating Bidi with various Prosody servers. OK, great. But absolutely nothing ever tries to negotiate bidi with it, despite it offering the feature.