XSF Discussion - 2018-07-27

  1. Ge0rG

    Andrew Nenakhov: if your company only employs jabber fans, this is an expected outcome. If you start with a bunch of people who never before did business chat, slack is much easier than any of the alternatives, especially xmpp

  2. jonasw

    Andrew Nenakhov, to be frank, I don’t like how you seem to do decent development, but don’t discuss with the community whenever you make such choices.

  3. jonasw

    (re 0221 vs. 0066, but also MUC etc.)

  4. jonasw

    (although muc is kinda understandable given the state MIX is in)

  5. Ge0rG

    > Zulip is 100% open source software, built by a vibrant community of hundreds of developers from all around the world. With 120,000 words of developer documentation, a high quality code base, and a welcoming community, it’s easy to extend or tweak Zulip. What did they do right that we did wrong?

  6. SamWhited

    They built a service, we build a protocol.

  7. SamWhited

    We need a service to champion the protocol.

  8. Ge0rG

    Yes, but where are we going to get the "vibrant community of hundreds of developers"?

  9. SamWhited

    Same thing, I think: By building a service. Their community is building on one client and one server. We have a fragmented ecosystem where lots of devs work on lots of clients and servers.

  10. SamWhited

    That's fine, but we need some bigger ones with commercial support too.

  11. waqas

    Ge0rG: It's fairly rare for the hundreds of developers to be doing much, it's probably a handful of core contributors

  12. SamWhited

    yah, that's probably true as well

  13. Ge0rG

    waqas: so where is the modern xmpp client with a handful of core contributors, then?

  14. waqas

    The vibrancy though, we need to learn to vibrate appropriately

  15. Ge0rG

    We could start out by making more of a buzz

  16. Seve/SouL

    My phone can vibrate, if that counts.

  17. goffi

    Hi. Is there a way with OMEMO to specify xml:lang?

  18. jonasw

    re vibration: https://www.youtube.com/watch?v=JWAAbmps_Zs

  19. Andrew Nenakhov

    jonasw, to me it seems that coding > discussing. No point to argue how to do something until we have a working prototype. If there are competing prototypes, community might stick to the one that is better, the other ones will die out by themselves. As is the case with MIX, creating a protocol without implementation might be a perpetual affair.

  20. MattJ

    I think there is a balance to be struck between the two

  21. MattJ

    "Rough conensus and running code" is a decent mantra in that regard

  22. jonasw

    Andrew Nenakhov, fair enough. this wasn’t necessarily meant as an accusation by the way, although I can see that it may read that way. I think this might also be a symptom of the issue that we have way too many ways to do the same thing.

  23. MattJ

    e.g. I looked at MUC Sub recently, and it definitely suffers from code-first problems, a number of things are not specified adequately, or at all

  24. Andrew Nenakhov

    > Same thing, I think: By building a service. Their community is building on one client and one server. We have a fragmented ecosystem where lots of devs work on lots of clients and servers. Currently what we're trying to do is to build server AND clients for all major platforms, so they simultaneously support new features and do it so that all clients have a similar look and feel

  25. jonasw

    Andrew Nenakhov, for example, any reason why you’re not using XEP-0385, which seems like a more natural choice for transmitting that type of stuff?

  26. Andrew Nenakhov

    Because it has a way too big number and wit myriad of xeps it's hard to read up to that point.

  27. Andrew Nenakhov

    And it looks kinda vague

  28. Andrew Nenakhov


  29. jonasw

    uh, I haven’t read all XEPs up to that. I follow more of a mix&match approach ;-)

  30. jonasw

    what is appealing about 0385 to me is that it has a well-defined way to share multiple sources for the data (like XEP-0221) has, it is reasonably simple to implement, and it allows to provide metadata like file size, mime type and dimensions

  31. Andrew Nenakhov

    We slightly extended 0221 for exactly that.

  32. jonasw


  33. jonasw

    why not use something which already exists instead of modifying something else which ~nobody else uses?

  34. Andrew Nenakhov

    https://xmpp.redsolution.com/upload/4bddf4f264f5c6577f16551f16a0abdf3f7ff84d/sHmWWFaYLaa0hl29XM0iYHhDMe07ouSq3B0jrZF6/IMG_20140212_165807.jpg https://xmpp.redsolution.com/upload/4bddf4f264f5c6577f16551f16a0abdf3f7ff84d/AccuS9gfadNqhr46nO3c6JokNJQ85sM3ZMmoSSfn/Screenshot_20180727-031120.png https://xmpp.redsolution.com/upload/4bddf4f264f5c6577f16551f16a0abdf3f7ff84d/Hmg5tqjf5dxZOYWngVKfMEmnhgkYOP55kkb96iqh/IMG_20140212_165719.jpg https://xmpp.redsolution.com/upload/4bddf4f264f5c6577f16551f16a0abdf3f7ff84d/V1e5V9S2fBwTzTEGPi08gkLMbcuLDsQC4jypTxc0/4b92c4f3-2cd6-4a75-ae43-cb1abc98ddf2.jpg

  35. Andrew Nenakhov

    Here are 4 files. All with sizes, dimensions, file types, etc

  36. Andrew Nenakhov

    jonasw, because no one generally uses xmpp at all. So being compatible with some obsolete clients that are a long time abandoned is futile.

  37. jonasw

    Andrew Nenakhov, there are actively developed clients outside of redsolution.

  38. Andrew Nenakhov

    We have a clear picture of the paying audience we plan to reach. They'll be happy to use our clients exclusively. If we succeed, our approach will be standard. If we fail, well, other clients approach will win.

  39. Ge0rG

    Andrew Nenakhov: your approach will not be "standard", it will be a silo.

  40. jonasw

    I expect it more to end up like whatsapp with no compatibility to other clients at all, yet another XMPP fork.

  41. Ge0rG

    or slack.

  42. Ge0rG

    or any other cloud-based chat offering with a half life of five years.

  43. Andrew Nenakhov

    Ge0rG, > Andrew Nenakhov: your approach will not be "standard", it will be a silo. X in XMPP stands for extensible. It seems to me that we understand Extensibility differently. If our approach succeeds with users, we'll suggest updating the standard. And we do everything with providing compatibility, not breaking anything anyway.

  44. Ge0rG

    Andrew Nenakhov: what MattJ said. "rough consensus and working code". Without consensus from the community, your code will end up in a dead end

  45. Ge0rG

    Andrew Nenakhov: I can understand your position from a business point of view, but I'm not yet sure it will actually benefit the XMPP community

  46. jonasw

    Andrew Nenakhov, while you figure out your business model and acquire customers, the community is doing its own development. then we end up with two strains of XMPP, one of which in your company which isn’t written down in standards, and one used by the community actually written down in the XEPs.

  47. jonasw

    I doubt that modifications to the XEPs which replicate things the community already has figured out in a more consensual way would be accepted

  48. MattJ

    I think it's also not taking advantage of community expertise that exists

  49. jonasw

    that, too

  50. Andrew Nenakhov

    What exactly things do we replicate?

  51. jonasw

    Andrew Nenakhov, the use of XEP-0221 over XEP-0385 is a good example

  52. jonasw

    but also the multi-user-chat discussion.

  53. Andrew Nenakhov

    Why did community develop unnecessary standard instead of slightly updating 0221?

  54. jonasw

    because data forms are---to me at least, but I think others may agree---not primarily a way to share (meta-)data but for interactive things. '221 was written with captchas in mind. '385 was specifically written with file sharing in mind, and I think it achieves that much better. There’s no way to annotate individual URLs in a <body/> using '221. '385 re-uses references for taht.

  55. Andrew Nenakhov

    And with mix, I'm probably one of the few who mostly read it along with my dev team. It's broken by design. And we already have a working group chat that works great, it's fast and reliable. Maybe 30% work left on server side.

  56. jonasw

    (but this is just waht I assume, '385 happenend before I was around here)

  57. Ge0rG

    Let's not talk about MIX

  58. Andrew Nenakhov

    Ok. )

  59. jonasw

    I understand your point about code-first, standards-second, but I don’t see how a quick "Hey, we’re going to do file sharing with metadata and plan to use '221 for that, any reasons we shouldn’t?" email would’ve hurt anyone.

  60. jonasw

    I think you would have gotten a clear response that building on '385 (I’m not saying that '385 is perfect, it can certainly use some love) would be preferrable.

  61. Andrew Nenakhov

    You see. I respect you guys who do stuff . And I like the idea of federated messaging. I firmly believe humanity needs an independent protocol to exchange messages. That's why I'm using my time, money and attract investors to fund our efforts in this space. However, xmpp is already older than one of my developers who joined the team yesterday. If in all these years doing things like it's done xmpp didn't reach even a fraction of WhatsApp success, did it occur to you that things should be done differently? At least, tried to be done differently?

  62. jonasw

    I see your point, however, going the same way WhatsApp (originally XMPP based, too) went by forking the protocol and ignoring the community won’t help here.

  63. Andrew Nenakhov

    jonasw, ok I'll write an email on 221 use. We actually considered 385 and rejected it, I'll explain why.

  64. jonasw

    that’d be great

  65. jonasw

    Looking forward to see why you came to the conclusion that modifying '221 was a better way forward than modifying '385.

  66. Ge0rG

    Andrew Nenakhov: we are building a protocol, not a product. You can't attract users with a protocol ;)

  67. Andrew Nenakhov

    jonasw, we are not going the exact WhatsApp way. If we do stuff we'll publish protocol extensions and release open source products that support them.

  68. jonasw

    (especially given that '385 is Experimental and thus has a much lower barrier to modification than '221 has)

  69. Andrew Nenakhov

    Btw I'm still not over how xhtml XEP was made deprecated.

  70. Seve/SouL smiles.

  71. jonasw

    I’m still not certain on that one either.

  72. jonasw

    I’m swaying between "it’s not *that* hard to get it right, even if done by simply disallowing all CSS" and "ugh. XHTML brings us a horde of security issues"

  73. Link Mauve

    Disallowing all CSS isn’t even a good solution, the subset defined in 0071 is known to not have any vulnerability on current browsers.

  74. jonasw

    Link Mauve, sure, but it’s not trivial to properly sanitise CSS

  75. jonasw

    which is why a simple implementation may choose to ignore it altogether.

  76. jonasw

    also, you’d want to modify any colors used to blend in with your UI.

  77. Link Mauve

    Maybe I should split out xmpp-parsers’s XHTML handling into another crate, and provide bindings and compilation targets for a bunch of other languages.

  78. Link Mauve

    jonasw, sure, and you can.

  79. jonasw

    yes, but it’s work

  80. jonasw

    work which can be gotten wrong

  81. jonasw

    and which can lead to interesting issues when gotten wrong

  82. jonasw

    although CSS will most likely only cause tracking since javascript URLs have been forbidden there. although I wouldn’t put my money on that one.

  83. Ge0rG

    > As the team chat market is overcrowded by copycats, Nayego is different. Said yet another xmpp client developer.

  84. Seve/SouL

    Someone from the XMPP community was involved in it, is it?

  85. Ge0rG

    I'd say nyco

  86. Dave Cridland

    Ge0rG, Team chat market isn't as crowded as it was yeterday, mind.

  87. Ge0rG

    Dave Cridland: it's crowded by the bodies of those who tried and zombies of those who don't want to realize that they failed.

  88. Dave Cridland

    Ge0rG, Yeah, I was making a nod toward Stride/Hipchat/Jitsi.

  89. Ge0rG

    Oh, I didn't know Jitsi was Atlassian as well. Is it also EOL now?

  90. Dave Cridland

    Jitsi was part of HipChat's business unit. The code was incorporated into Stride. Slack, OTOH, use a modified Janus SFU and things - I can't see they'd use much from it.

  91. Ge0rG

    Will they make use of the team?

  92. Dave Cridland

    I wouldn't know.

  93. Dave Cridland

    I've not examined the details of the transfer, but I believe it's a sale of customers basically.

  94. Ge0rG

    Except it's not providing customers with a clean migration path, except "install slack, kill your old chat".

  95. Dave Cridland


  96. Dave Cridland

    And honouring contracts, of course, though they were priced more or less identically.

  97. Ge0rG

    I suppose this merger will make few people happy and many people sad, for a variety of reasons.

  98. Alex


  99. Alex


  100. Dave Cridland

    Alex, Indeed. Nothing on Jitsi, mind, but that's part of the same business unit.

  101. jonasw

    goffi, could you make a PR against '384 regarding the singleton implementation note?

  102. Zash

    There's a thing

  103. jonasw

    Zash, that thing? https://xmpp.org/extensions/xep-0060.html#impl-singleton

  104. Zash

    https://xmpp.org/extensions/xep-0060.html#impl-singleton > When this pattern is desired, it is RECOMMENDED for the publisher to specify an ItemID of "current" to ensure that the publication of a new item will overwrite the existing item.

  105. jonasw

    that’s what I’m talking about

  106. Zash

    jonasw: Yes. That thing.

  107. goffi

    jonasw: OK I can check that, will do it later this week-end.

  108. jonasw

    goffi, thx!

  109. Guus

    With regards to CSI, does RFC6120#section-10.1 forbid the queuing of some stanzas, while letting other ones through?

  110. Holger

    I've been told it's fine to break RFC rules if the XEP allows it.

  111. Guus

    That's ... Surprising

  112. Guus

    And feels wrong

  113. Holger

    The rationale was that the rule breakage is negotiated and hence won't lead to trouble.

  114. Holger

    The problem I see is implementation-wise. If you encoded those rules in the lower layers, you must change those layers to implement an extension.

  115. Guus

    At best, I feel that that should be worded better in the XEP.

  116. Guus

    It now appears to describe that you should be compliant with section 10.1

  117. Guus

    MattJ, thoughs?

  118. Guus

    (penny for your)

  119. MattJ

    My thoughts are that the XEP does not give any mandate to break the RFC

  120. MattJ

    In-order processing is relied upon in multiple edge cases

  121. MattJ

    The CSI XEP originally, very intentionally, avoided saying anything about what a server might do with its knowledge of the client state

  122. Zash

    And then some got annoyed because they couldn't be sure what the server would do.

  123. MattJ

    Right, so community consensus seemed to be that it should at least provide a list of example behaviours

  124. Guus

    So you feel that you cannot withold say a status update from a client when you push it a message that was sent to it later?

  125. MattJ

    I don't, no

  126. Holger

    > The CSI XEP originally, very intentionally, avoided saying anything about what a server might do with its knowledge of the client state The problem I see with that is that the client can't rely on anything once sending <inactive/>.

  127. Guus

    And would it be ok to withold from Alice a status update sent from Bob, when Bob later sends a message to Jacky?

  128. MattJ

    Guus, I think it's a little safer between two distinct (bare?) JIDs

  129. MattJ

    Holger, I don't see a problem with that

  130. Guus

    Sure, but it either does or doesn't break spec.

  131. MattJ

    Holger, the CSI XEP was never intended to have any observable effect to the client, at a high level

  132. MattJ

    Guus, CSI does not break the RFC

  133. MattJ

    CSI is a way for the client to say whether it is "active" or not, it's very simple

  134. Guus

    The withold bit in my example, I mean

  135. MattJ

    As far as I'm concerned, what servers do with this info is up to them

  136. MattJ

    and not standardised

  137. MattJ

    and neither should be, without very careful consideration

  138. MattJ

    As far as I'm concerned, if the server breaks the client by messing around with the stream, the server is broken

  139. MattJ

    Maybe someone feels like making a standard for this, possibly with an opt-in for the client, they are free to

  140. Guus

    So, as far as queuing goes, a CSI based optimization should not queue anything when sending something else to the same bare jid?

  141. Holger

    MattJ: Server implementors would have to anticipate all possible use cases to judge whether a given optimization is 'safe', i.e. "has no observable effect".

  142. Holger

    > So, as far as queuing goes, a CSI based optimization should not queue anything when sending something else to the same bare jid? In ejabberd, if I send you a message, I send all queued traffic from the same sender first. (Not sure why I don't just flush the queue in that case, the radio is woken anyway.)

  143. Guus

    From the same sender?

  144. Guus

    I'd think you'd flush all data to the same recipient. Possibly all its full jids?

  145. Holger

    Yeah, that's the part I'm not sure makes sense 🙂 (It's been a while and I recently thought about just flushing the queue whenever I send traffic.)

  146. Zash

    The thing I personally use has a single queue, that gets flushed when it's "full" or when there's something "important". The original order is preserved.

  147. Holger

    Guus: Right now if I send you a message from=alice, I only flush all stanzas that were sent from=alice.

  148. Guus

    Zash: I'm leaning towards something like that now too.

  149. MattJ

    Yes, I'm planning to include the single-queue one in the next Prosody release

  150. Guus

    Holger: that's weird to me, as a message from Bob that was sent to the same recipient might arrive out of order.

  151. MattJ

    The others are just too risky, but people can use them if they want (from prosody-modules) (and they do)

  152. MattJ

    Despite no tests as to how effective any of these methods are on real traffic

  153. Holger

    Apart from that I try to dedup "state" things (presence, chat states, PEP), i.e. you only receive the current state. Mainly to avoid frequent queue flushes due to the queue size limit being exceeded.

  154. Zash

    Guus: The Mobile Optimizations XEP (or whatsitcalled, by Dave Cridland) basically says that too, "send a lot while the radio is hot"

  155. Guus

    Zash: makes sense

  156. Holger

    Guus: Yes queued traffic can go out of order that way. I.e. Alice sent presence before Bob, then Bob sent a message, so you receive Bob's presence before Alice's. This stuff can be hairy with MUC joins, for example. Just don't do this 🙂

  157. Zash

    Holger: As I've said before, the rules for things like that quickly grow complicated

  158. Guus

    I'm now wondering if I should stop CSI support with dropping chat state notifications and the like only...

  159. Holger

    Yes. But you might consider the complexity with it to avoid frequent queue flushes.

  160. Guus

    And not have a queue at all

  161. Holger

    *worth it

  162. MattJ

    Dropping chat states -> never drop "active", never drop if there is any other payload in the message (that isn't recognised as safe to drop)

  163. MattJ

    Is that the only required logic?

  164. Guus

    Why not 'active' if there's no other payload?

  165. MattJ

    Because then someone could be typing, and then stop typing while I was inactive

  166. MattJ

    and then they would be forever typing

  167. MattJ

    in my client

  168. Guus

    I now have "drop messages that have no other child elements that do define a namespace, other than chatstates"

  169. MattJ

    So you would have the perpetual typing issue?

  170. Guus

    Yeah, I didn't think of that

  171. Zash

    Isn't what Holger says to keep the last one?

  172. Zash

    That makes sense

  173. Zash

    Like that module that does that for presence already

  174. Guus

    But, really, only dropping chatstates, really doesn't make a dent in the traffic

  175. Guus

    Zash: yeah, but that'd involve queuing which I think either is flushed all the time (rendering CSI effectless) or break RFC mandated ordering

  176. Zash

    I do think trying to group stuff into batches is better than trying to drop things

  177. Zash

    If we bring back Compression-except-safe, then it should be possible to squeeze in all them chatstates without using much data

  178. Guus

    Meh, this all is a bit disappointing. I kinda hoped for a silver bullet.

  179. MattJ

    Don't we all? :)

  180. MattJ

    But when was the last time anyone measured XMPP client traffic or battery usage?

  181. Guus

    Sooooo.... Would things be more effective if we explicitly let the client tell the server for what kind of data it allows the server to break the delivery order requirement?

  182. MattJ

    I don't think that's a nice thing to do to client devs

  183. Guus

    "only send me the last non subscription presence update for each contact that changed state after I went inactive"

  184. MattJ

    Complex protocol for something where basically everyone will want the same rules

  185. Guus

    Well, without that, they don't get any benefit from CSI.

  186. MattJ

    Just the rules aren't easy to describe or discover

  187. MattJ

    Single queue has benefit

  188. MattJ

    In theory

  189. Kev

    I'm not reading up for all the context, but ...

  190. Kev

    Is there any way we can get rid of CSI and instead disconnect when in the background and then get our reconnects to a sensible speed?

  191. MattJ

    I'm not sure how different that is in the end

  192. MattJ

    E.g. if the server optimizes the 198 queue with the same rules

  193. Kev

    I don't know. I just can't shake the feeling that both CSI and 198 Resumption are trying to work around a fundamental problem rather than address it.

  194. Guus

    Well, without SM you'd not show as online

  195. MattJ

    What if they are addressing it?

  196. Guus

    Sorry, "available"

  197. Kev

    The issue being that getting current state is slow unless you were online when it changed, so we're trying to avoid going offline, with 198 letting us resume, and CSI letting us cut down the cost of being online.

  198. Kev

    But if we could solve the fundamental "It's expensive to catch up" issue, neither would be needed.

  199. Guus

    Kev: there's truth in that. Unsure if it's sufficiently feasible.

  200. Zash

    I think there's value in staying connected

  201. Kev

    If it's not feasible I think we end up with significant issues anyway, because at some point you're going to need to fire up a new client, or an old one will get disconnected beyond 198 periods, or whatever.

  202. Kev

    It feels to me like we should look at what we really want to do on login (rather than what current protocol dictates we do), and see how close to that we can get.