XSF Discussion - 2020-10-06


  1. pep.

    Can participants fetch MUC-registered nicknames of other participants?

  2. Daniel

    Will those appear in the member list?

  3. Daniel

    Admin/owner list

  4. Daniel

    That's probably way underspecified. But it would make sense

  5. Daniel

    Since prosody is the only implementation that even has registration ask MattJ?

  6. Ge0rG

    MattJ proposed to send all registered participants that are not an occupant with unavailable presence.

  7. pep.

    (one more place to fetch nicks yay)

  8. Ge0rG

    IMHO, sending a bunch of unavailable presence with a join shouldn't break anything

  9. Daniel

    Shouldn't...

  10. Daniel

    But yes that probably makes sense

  11. MattJ

    pep., "one more place" -> the idea was to consolidate them all into this one place (presence), so the client only has one mechanism to track occupants

  12. MattJ

    In my head I have a draft thing for MUC join flags to opt into things like this

  13. MattJ

    and also to filter messages, e.g. bots that only want to receive certain things or from certain occupants (and enforcing this server-side if necessary, so it becomes safe to add bots to a room without them logging your conversations)

  14. Daniel

    Uh. Like muc/sub

  15. Ge0rG

    MattJ: also a flag to not send occupant presence and a roser-versioning-alike thing for the participants?

  16. pep.

    MattJ: there are other places where a client can fetch nicks / display name that a MUC doesn't have access to

  17. pep.

    right?

  18. MattJ

    pep., oh, sure

  19. MattJ

    Daniel, quite possibly, but I'd actually put this through as a XEP and hope to keep the changes simple enough that it gets adoped (if other server devs even care about improving MUC anymore, it's possible that they don't)

  20. MattJ

    Daniel, quite possibly, but I'd actually put this through as a XEP and hope to keep the changes simple enough that it gets adopted (if other server devs even care about improving MUC anymore, it's possible that they don't)

  21. Ge0rG

    I still think that improving MUC is worthwhile. MIX has become a significant engineering effort.

  22. Daniel

    Implementing it on the client side is probably easy

  23. Daniel

    but for me the biggest issue with MUC is reconnection/reliability

  24. Daniel

    before we can solve that it feels a little pointless to waste time and energy on other improvements

  25. Daniel

    but I understand that different people have different priorities

  26. Daniel

    and/or that work on one improvement doesn't block improvements on other fronts

  27. Ge0rG

    Daniel: did you make any progress on the per-MUC push registration?

  28. Daniel

    i even removed the code again

  29. Daniel

    because it was causing some weird edge case bugs

  30. Daniel

    where the server was maybe at fault

  31. Daniel

    and nobody had implemented it

  32. Ge0rG

    "weird edge case bugs" is the long form of "MUC"

  33. Daniel

    well i think it wasn’t even muc related. but a server announced push and muc on the primary domain and then it registered itself 100 times or something

  34. pep.

    What happened of 410 on s2s?

  35. Daniel

    I don’t recall the details. but i didn’t feel like fixing it or finding work arounds for it when it was dead code anyway

  36. pep.

    MattJ, any hint about my first questio nbtw

  37. pep.

    MattJ, any hint about my first question btw

  38. Daniel

    not on s2s but having your server do it?

  39. Daniel

    because 410 on s2s would just be regular ping, no?

  40. pep.

    yeah, servers doing "410"

  41. pep.

    Whatever it takes to keep the link on. Which is also gonna be a problem for MIX anyway

  42. Daniel

    didn’t eta want to come up with a thing

  43. pep.

    dunno

  44. Ge0rG

    MIX has the additional problem that there is no plan for recovering from s2s outages.

  45. pep.

    yeah

  46. Ge0rG

    MUC is self-healing, more or less, once you rejoin

  47. Daniel

    i've heard matrix has something for that

  48. Ge0rG

    MIX... good luck.

  49. Daniel

    well your joins/leaves are IQ based so you know if that was succesful

  50. Ge0rG

    We also have the same problem with MAM.

  51. Daniel

    and then you might just lose messages

  52. Ge0rG

    Because apparently, combining "give me everything I missed" and "enable live delivery" is a HARD problem

  53. MattJ

    pep., from example 131 in XEP-0045 it looks like the affiliation list includes the nick, so yeah

  54. pep.

    Ah, thanks. I missed it indeed

  55. Daniel

    MattJ, does prosody already include that?

  56. MattJ

    Yes

  57. MattJ

    Just found the test: https://hg.prosody.im/trunk/file/tip/spec/scansion/muc_register.scs#l397

  58. eta

    Daniel: yeah I have half a protoxep about automatically reconnecting to MUCs that I never finished

  59. pasis

    Hi, I've refreshed library in libraries.json, but xmpp.org hasn't been synced. Could someone update xmpp.org? The library is libstrophe.

  60. Seve

    pasis: It was merged though, right? I think I did

  61. pasis

    yes, it's merged, just doesn't appear at xmpp.org

  62. pep.

    Yeah docker things probably need a push

  63. jonas’

    they need a pull :-X

  64. pep.

    :P

  65. jonas’

    joking aside, will do

  66. jonas’

    updated

  67. vanitasvitae

    btw. there are some stale PRs on xsf/xeps that are ready for processing by the editors, namely https://github.com/xsf/xeps/pull/923 and https://github.com/xsf/xeps/pull/932

  68. vanitasvitae pokes editors with a stick

  69. jonas’

    vanitasvitae, ah, I missed your approval on those, as I was on vacation during that time

  70. jonas’

    will tag them so that I process them later tonight

  71. jonas’

    thanks for the ping

  72. vanitasvitae

    nice 🙂

  73. vanitasvitae

    thanks for being an editor ❤️

  74. pep.

    and I didn't really help :x

  75. pasis

    it works now, thanks :)

  76. dwd

    (Reading up) eta, I keep meaning to explore your ideas around group chats in more detail. I think I've read half your series on your blog, but keep meaning to read the rest.

  77. eta

    dwd, I mean it does peter out near the end; the only useful thing in that series is the whole "sponsoring server" model

  78. eta

    (which you could implement with XEP-0045 today and a custom MUC component, and I indeed might one day)

  79. dwd

    eta, Ah, yes - you had a hybrid model between XMPP's single authority and IRC's model of every server has to be trusted?

  80. eta

    dwd, exactly -- you have a quorum of trusted servers that use some consensus algorithm (Raft / Paxos)

  81. eta

    which is less crazy than Matrix's "fully decentralized" model (which I don't see being viable

  82. eta

    which is less crazy than Matrix's "fully decentralized" model (which I don't see being viable)

  83. pep.

    (blog link?)

  84. dwd

    eta, There's some work around on partially trusted consensus algorithms, too. Kind of like BFT algorithms but each server might have different legitimate aims.

  85. eta

    pep., https://theta.eu.org/2019/10/10/nea-federation-design.html

  86. pep.

    thanks

  87. dwd

    eta, You definitely don't want Paxos, though. And even Raft has some problems if we assume byzantine fault tolerance and dynamic group membership is desirable, I think.

  88. eta

    dwd, very potentially! I never really went anywhere with this; it was just an idea

  89. eta

    what could be interesting is just to piggyback of something like etcd, which already handles the distributed state

  90. dwd

    eta, Well, you could start with a simple Raft implementation, for exploratory purposes.

  91. eta nods

  92. Shell

    etcd is just a Raft implementation on top of a k/v store, isn't it?

  93. dwd

    eta, And I think you don't need to worry too much about formal consensus for most things, you can do elections and a tree-based fanout for messaging, for example.

  94. eta

    Shell, yeah

  95. eta

    dwd, yeah, I mean you only need consensus for things like the room member list

  96. Kev

    Am I misremembering Raft, or isn't it a leadership-election thing? So fails if you need to continue functioning in a split situation?

  97. dwd

    eta, ... maybe? I'm not sure that needs consensus until the point where a member is more than a mere passive observer.

  98. eta

    dwd, no, sure

  99. eta

    I meant more bans/voiced users/etc

  100. dwd

    Kev, It might have leadership, but it survives partition. It is not, however, eventually-consistent, so an orphaned member ceases to be available.

  101. Kev

    That is, I believe, what I intended to express. The majority split functions, the minority split doesn't.

  102. eta

    Kev, the thing is you can just queue messages in the minority split until it rejoins

  103. Kev

    eta

  104. eta blinks

  105. Kev

    (Sorry, dev version of client)

  106. eta

    aha :)

  107. dwd

    Kev, But eventual consistency and message ordering are, I think, problematic, unless your UX makes things very clear. FMUC solves this, I think, by making that UX affordance possible.

  108. Kev

    eta: By 'messages' here do you mean <message/> or the log entries from Raft? If it's <message/> then there's lots of other modifications that could happen during a split that are hard to deal with, I think.

  109. eta

    Kev, <message/>

  110. pep.

    I don't know Raft nor Paxos. Why would a minority not continue working on its own? redo elections etc.?

  111. pep.

    Resources I can read?

  112. eta

    pep., https://raft.github.io/

  113. Shell

    because the minority can't know that what it does is compatible with the state of the majority.

  114. Kev

    dwd: Yes. It's not even the message ordering that most worries me though, it's the other state changes, e.g. banning a user and making them an owner on different nodes (the old IRC netsplit op attacks are an extreme symptom of this).

  115. dwd

    pep., Have a look at "CAP Theorum", as a good general start point.

  116. Kev

    pep: In order to win an election you need to have a majority vote. If you're in a minority split you can never have a majority vote. The leader is the Source Of Truth for the replication, with it being responsible for ordering the log messages. So if you had two leaders each claiming to be the Source Of Truth, bad things happen. At least from memory, I've not looked at Raft properly in a few years.

  117. dwd

    Kev, Broadly, this is true for almost every consensus algorithm that eschews eventual consistency.

  118. Kev

    Right.

  119. pep.

    Kev, right, your use of minority / majority "only" makes sense when you try to merge back :p

  120. pep.

    Thanks for the pointers

  121. eta

    Kev, I don't see how you can do split riding though

  122. dwd

    Kev, But also, I don't think the netsplit op attacks are possible if you don't have an eventual-consistency style.

  123. eta

    if you're in the minority partition, disallow all state changes

  124. dwd

    eta, Not just state. Message reordering can lead to some interesting problems.

  125. Kev

    Right. That (eta) obviously avoids the complications, at the cost of degraded service. But note that 'state' here includes message publishing.

  126. dwd

    And when I say "interesting", I mean in the sense of people dying.

  127. eta

    ?!

  128. Ge0rG

    Just DAG everything?

  129. eta remembers dwd makes healthcare messaging software

  130. dwd

    eta, Not that context actually, but yes.

  131. eta

    wait, so explain how message reordering is problematic?

  132. dwd

    "Is the patient breathing?" "Yes" "Have you checked for spinal injury?" "No". Rearrange for much fun.

  133. eta

    right, but you can't flip the yes and the no here

  134. eta

    assuming the asker and the responder have a netsplit between them

  135. eta

    because the behaviour in event of partition is to queue

  136. Zash

    eta: is your blog on planet ?

  137. dwd

    No, but if the "Yes" follows "spinal injury" people get pretty confused very quickly.

  138. eta

    Zash, don't think so

  139. Ge0rG

    eta: you can get nice reordering effects if there is a third party reading, and they have intermittent s2s issues

  140. Kev

    My (not exhaustive, but I've spent more time on this than one might like) experience of things-that-can-go-wrong in situations like this is that whenever I think "But at least X can't happen", something else that's equivalent to X ends up being able to happen.

  141. eta

    Kev, hah

  142. dwd

    Ge0rG, As for DAG, that does deserve a special place in hell. People don't talk in DAGs, they talk linearly. The way to get DAGgy in conversations is to thread them and present a UX that then makes sense, and somehow encourage users to use that threading sanely.

  143. Kev

    dwd: Our conversation here is a DAG. It's just a very specialised case :)

  144. Kev

    ._._._._._._._.

  145. dwd

    Kev, Yes, yes. Have a sticker. :-)

  146. Ge0rG

    Kev: is it really? Each message is either a root or a reference to some earlier message.

  147. eta

    if you *really* cared about message ordering you could replicate messages in the raft log

  148. dwd

    Ge0rG, A linear graph is a proper subset of a DAG, I think he means.

  149. eta

    but that would add quite a deal of latency

  150. Kev

    dwd: Honestly, a decent sticker would brighten up my day Quite A Lot right now.

  151. pep.

    Some people want want ./·_./·_./·_. :x

  152. Ge0rG

    dwd: yes, that's what I conlcuded from the message as well. My point is that this conversation is not a linear graph.

  153. pep.

    Some people want ./·_./·_./·_. :x

  154. pep.

    And not a full dag

  155. Ge0rG

    pep.: is that morse code?

  156. Ge0rG

    drunk morse?

  157. dwd

    Ge0rG, Morse did drink a lot.

  158. Kev

    Ge0rG: I'm as surprised as the next person that my joke doesn't stand up to scrutiny.

  159. Ge0rG

    Kev: sorry

  160. dwd

    Kev, It's been rejected at peer review stage, but is still available as a pre=print, so I guess you win?

  161. Kev

    Academic journal peer review is a process I will be very happy not to go through again (on either side).

  162. Ge0rG

    I suppose normal people are not deep into partially ordered histories, right?

  163. Kev

    Georg - if you mean that it's only the 'mission critical' (for want of a better term) situations where ordering matters, I think 'normal' people care about it too, it's just less bad when it goes wrong.

  164. pep.

    Ge0rG, it's supposed to represent a special shape of tree :p Dunno if that has a name

  165. pep.

    https://ppjet.bouah.net/tree-foo.png

  166. Ge0rG

    Kev: what I intend to say is this: if message history is implemented as an eventually consistent DAG, people won't be able to grasp the concept and draw the right conclusions regarding message ordering. Not in normal times and especially not during a crisis

  167. Kev

    Ah. Yes, I think a UI needs to de-DAG it in order for it to be reasonably understandable.

  168. dwd

    eta, Anyway, if you always send messages to the top of a conceptual routing tree, so a single leader elected for the purpose, but handle state changes through a consensus algorithm, then I think we're good. If you use a Byzantine fault tolerant algorithm, and use it sparingly, then I think you more or less get reasonable performance with highly resilient state - as long as you're not connected to a node which loses connectivity with the others in the consensus group.

  169. pep.

    (Basically meant to say some people want to be able to reply to a single message)

  170. dwd

    Kev, Or as pep. suggests (I think), Slack-style "threading".

  171. Ge0rG

    Are we talking about threading or about message history consistency?

  172. Kev

    I note that me saying that things like Raft don't solve all aspects of CAP (yes), doesn't mean that using Raft for MUCs (which are currently entirely SPoF) would be bad.

  173. dwd

    pep., FWIW, I wanted to do Slack-style threading by spawning off a new chatroom as a child of the containing conversation.

  174. pep.

    dwd, I hear that's how rocketchat worksss-ish?

  175. pep.

    I've never used it

  176. dwd

    Kev, Also the whole point of CAP is that you can't solve all aspects of it.

  177. Kev

    I wanted to do Slack-style threading by spawning off a new message node within the channel.

  178. Kev

    dwd: Yes, thus the "(yes)" to denote that I realised.

  179. dwd

    pep., I... don't know. I do know some people who do a lot with Rocketchat though.

  180. dwd

    Kev, Ah, I thought so, but then I worried about people misunderstanding, but then after writing my last I realised that anyone familiar with "CAP" would know anyway.

  181. dwd

    Sometimes I should just not write things. :-)

  182. dwd

    eta, Oh, and before I forget to mention it, the bit of your design I thought was interesting - and have now largely forgotten - was your approach to permissions at the group chat level, which seemed well thought through.

  183. Kev

    You are probably right that I'm not expressing things in an observer-considerate way.

  184. Kev

    I've been thinking about ways to improve clustering within M-Link (and other things) recently, which is a very similar area, but with different requirements.

  185. dwd

    Yes - no need to conider Byzantine fault tolerance, but you probably so want to go the eventual consistency route for most things.

  186. Kev

    I hadn't realised it, but reading eta's article I'm starting to think I'm in the middle of re-inventing Matrix.

  187. dwd

    Kev, Thinking about the IRC netsplit attacks, were those entirely predicated on the combination of channels disappearing when nobody was in them (and thus being able to be recreated with different ownership) and a poor merge algorithm?

  188. eta

    dwd, yes

  189. Kev

    Yes.

  190. eta

    the fix was "improve the merge algorithm"

  191. eta

    so the channel with the earliest creation timestamp wins

  192. Kev

    Well, that was one of the fixes. I've also seen (maybe not recently) disallowing channel creation during a split.

  193. Zash

    Oh glob not merge algorithms

  194. Kev

    You can't *not* have merge algorithms of some sort, if you want eventual consistency.

  195. Kev

    You may as well say "Oh glob, not eventual consistency", which would probably be quite fair, it's a lot more of a brainfuck to reason about than something like Raft.

  196. eta decides people should just use single servers and avoid making a distributed system where possible

  197. dwd

    eta, We already have a distributed system in XMPP; we just have a fixed leader and all decisions are made by the leader. That's a valid solution to be problem; it's just that if you want to address the shortcomings (like the leader being a SPOF) then things get complicated.

  198. dwd

    eta, Most notably, because our distributed system is a heterogeneous distributed system already.

  199. clinton removed by Zash

    porn

  200. eta

    ah, that is porn

  201. edhelas

    yes, but uploaded using HTTP Upload !

  202. eta

    yeah, which meant I clicked on it

  203. eta

    and now it's on my screen and I can't delete it

  204. eta

    thanks dino

  205. pep.

    quick quick send new messages

  206. Zash

    Message Moderation to the rescue?

  207. dwd

    Scrolling, scrolling, scrolling.

  208. vanitasvitae

    luckily I waited before clicking

  209. dwd

    Get them wagons rolling.

  210. edhelas

    got a nice preview on Movim :D

  211. vanitasvitae

    but yeah, deleting images in dino would be nice

  212. pep.

    Zash, I think allowing a user to stop displaying a picture would be nice nonetheless :/

  213. eta makes more noise

  214. Ge0rG

    Some clients will just auto load images

  215. eta

    vanitasvitae, indeed

  216. edhelas

    actually it's the best way to wake up a channel it seems 🤔

  217. eta

    hahaha

  218. eta

    yay, it's off the screen now

  219. pep.

    Good, now we can go back to sleep

  220. Zash

    Pretty sure I enabled moderation here, so you can go in and delete it from the archives even

  221. edhelas

    pfieuw, now let's go back to normal

  222. dwd

    At least we're considered a worthwhile audience for porn spam.

  223. Ge0rG

    You can't solve social problems with technical means. If you block images, they'll send ASCII porn. Or illegal text content.

  224. eta

    Zash, but isn't it just converse that does that

  225. edhelas

    only XEP pr0n is allowed here, I like reading 0060 for example

  226. eta

    that's the good stuff

  227. vanitasvitae

    guys, follow my OnlyXEPs account!

  228. Ge0rG

    edhelas: you pervert!

  229. edhelas

    yeah, I love those Romeo & Juliet stories

  230. pep.

    While there is some activity in here: in 20 minutes we have a SCAM meeting in xmpp:scam@muc.xmpp.org?join and we're talking about Summit. Please join if you want to follow and/or have ideas on the organisation side of things

  231. vanitasvitae

    pep., thanks for the heads up

  232. JustLikeThat

    I have a proposal 🙂

  233. JustLikeThat removed by Zash

    porn

  234. mdosch

    Porn again? This time moving pictures?

  235. Guus

    Yes

  236. Guus

    Better moderating support in clients is going to become more desirable.

  237. Guus

    EG: make a room temporarily moderated, hand out voice, stuff like that.

  238. Zash

    and XEP-0425

  239. a moderator removed a message

    porn

  240. a moderator removed a message

    porn

  241. CognitiveDissonance

    Hi all

  242. CognitiveDissonance

    I was wondering, what is the *fundamental* difference between XMPP and Matrix?

  243. MattJ

    Great question :)

  244. Daniel

    Fundamental from an end user experience probably not

  245. MattJ

    I think the fundamental difference is that XMPP is based on routing stuff, and Matrix is based on distributing stuff

  246. CognitiveDissonance

    Like design principles, target use-cases, future roadmap etc..

  247. eta

    CognitiveDissonance, Matrix is fundamentally a database (a graph data structure that's replicated across servers; the spec defines how to replicate the structure), whereas XMPP is based on passing messages around (the spec defines how messages should be routed to different clients and optionally stored)

  248. CognitiveDissonance

    Ah, routing vs distributing seems like a core difference.

  249. CognitiveDissonance

    Also, is matrix completely web-based?

  250. eta

    this is why Matrix servers need more resources to run; because they spend a lot of energy doing this replication algorithm

  251. Daniel

    Define web based.

  252. eta

    whereas XMPP messages are simple and easy to route

  253. CognitiveDissonance

    Daniel http

  254. Daniel

    Yes it uses http.

  255. Kev

    > easy to route /me giggles

  256. CognitiveDissonance

    Daniel What about xmpp?

  257. moparisthebest

    but if "http" is your criteria then XMPP is also web-based

  258. Daniel

    c2s *can* be http

  259. moparisthebest

    because it can be used over http

  260. Daniel

    s2s is tcp

  261. Daniel

    or at least we haven’t speced out a way to use s2s over http

  262. Daniel

    i guess we could

  263. jonas’

    why tho

  264. CognitiveDissonance

    So in terms of design principles like modularity, simplicity and extensibility, what is the diff b/w them?

  265. CognitiveDissonance

    Daniel Hmm, I always seen only xmpp://foo and not http://foo, for c2s

  266. Daniel

    jonas’, well I was just answering a question.

  267. moparisthebest

    XMPP has proven to be very extensible based on continuing to work well 20+ years after it started, matrix on the other hand... they are asking users to migrate servers because they can't scale

  268. Daniel

    but you might want to run servers behind more restrictives firewalls

  269. Daniel

    on your raspi or something

  270. MattJ

    Running servers in the browser has been discussed in certain server projects before

  271. CognitiveDissonance

    Daniel: I see.

  272. CognitiveDissonance

    moparisthebest Point.

  273. CognitiveDissonance

    * good point

  274. eta

    CognitiveDissonance, so natively XMPP uses a TCP socket, but the BOSH (bidirectional streams over synchronous HTTP) extension lets you use it over HTTP

  275. CognitiveDissonance

    Sp matrix is a monolithic system right?

  276. eta

    (almost all popular servers expose that as an option for web clients)

  277. moparisthebest

    eta, CognitiveDissonance also websockets

  278. eta

    yep, that too

  279. moparisthebest

    I expect *very soon (tm)* xmpp c2s and s2s to start working over QUIC as well

  280. CognitiveDissonance

    eta Daniel Ah BOSH makes sense. Btw, xmpp server doesn't *require* web server right? matrix seems to rely on web server.

  281. eta

    CognitiveDissonance, not at all!

  282. eta

    you might want one for HTTP file uploading (XEP-0363), but it isn't a hard requirement

  283. CognitiveDissonance

    I see.

  284. CognitiveDissonance

    Also, in DNS records, I notices xmpp has only SRV records, where matrix has only A records.

  285. eta

    that's because matrix uses /.well-known

  286. jonas’

    CognitiveDissonance, you need A/AAAA records for SRV to make sense

  287. moparisthebest

    XMPP also has TXT records for some connection methods...

  288. CognitiveDissonance

    boon or a bane?

  289. Zash

    Matrix uses SRV too

  290. moparisthebest

    again I expect *very soon (tm)* XMPP to start using http-svc/srv2 records or whatever the latest name for those are

  291. Zash

    moparisthebest: don't you dare!

  292. moparisthebest

    Zash, required for sni+alpn encryption!

  293. Zash

    NOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO

  294. moparisthebest

    (plus other things etc)

  295. Zash

    Do not want

  296. CognitiveDissonance

    i am scared of http and web tech stuff.

  297. moparisthebest

    that's the best thing about XMPP, even if you don't want it I can still have it and vice versa :D

  298. CognitiveDissonance

    Too many security issues

  299. moparisthebest

    CognitiveDissonance, to be fair that's just "computers"

  300. dwd

    CognitiveDissonance, To be cleat, Matrix need not suffer from those since it's not like it runs in the browser.

  301. dwd

    CognitiveDissonance, To be clear, Matrix need not suffer from those since it's not like it runs in the browser.

  302. dwd

    CognitiveDissonance, It just uses HTTP and JSON as its substrate layer, like XMPP uses XML. And XML has its own set of amusing security issues.

  303. jonas’

    "laugh"able!

  304. CognitiveDissonance

    dwd hmm, AFAIK, even riot/element desktop client is a web stuff via electron.

  305. dwd

    CognitiveDissonance, Sure, the implementation of the clients is all web tech. But in principle it need not be.

  306. CognitiveDissonance

    dwd. I see. I have seem in wikipedia that xmpp is a text-based protocol like IRC. what does mean?

  307. dwd

    CognitiveDissonance, There are, I think, Electron-ish clients in the XMPP world. Certainly browser based ones.

  308. jonas’

    CognitiveDissonance, it means that some bytes are not allowed in the streams

  309. jonas’

    doesn’t matter in practice

  310. dwd

    CognitiveDissonance, XMPP operates by exchanging XML fragments over TCP, not text. Matrix operates by exchanging bits of JSON over HTTP.

  311. moparisthebest

    IRC operates by exchanging bytes over TCP, better hope you guess the encoding correctly!

  312. dwd

    (Favourite thing about IRC is that the case-folding for usernames is based on whatever keyboard layout the developer happened to be using).

  313. CognitiveDissonance

    I was confused about "text-based". XML is not a plain text?

  314. jonas’

    (dwd, wasn’t it the swedish 8-bit latin encoding thing?)

  315. Zash

    jonas’: finnish

  316. jonas’

    CognitiveDissonance, it is as much plain text as JSON is

  317. jonas’

    Zash, ah, right

  318. dwd

    CognitiveDissonance, It's generally textual in nature, but it's not plain text as such.

  319. jonas’

    (I mixed that up with mysql, that was swedish)

  320. moparisthebest

    if anything XMPP/XML and Matrix/JSON are much more "text-based" than IRC because the encoding is actually defined

  321. CognitiveDissonance

    dwd: Ah, I am getting the fundamental difference now.

  322. CognitiveDissonance

    Matrix exchanges JSON over http and XMPP exchanges XML over tcp.

  323. jonas’

    (over http over tcp/whatever atrocity google thinks of)

  324. CognitiveDissonance

    If I put a VS (versus) in between ....

  325. moparisthebest

    mainly yes, but more generically XMPP exchanges XML over (a stream)

  326. dwd

    CognitiveDissonance, Well, possibly not. XML+TCP and JSON+HTTP aren't that different, really. The biggest gain XMPP gets from XML is actually the namespacing rules, which give us clear extensibility - but in principle you can transcode between the two with some rules. The much bigger difference between XMPP and Matrix is the fundamental model.

  327. moparisthebest

    it's mostly TCP but it can be Websocket or a BOSH session today

  328. moparisthebest

    QUIC or whatever else tommorow

  329. dwd

    moparisthebest, In fairness, plain TCP is rare since we're always using TLS.

  330. moparisthebest

    right I didn't mean *plain* TCP but good clarification

  331. CognitiveDissonance

    http is stateless and tct is stateful?

  332. dwd

    CognitiveDissonance, Broadly meaningless. Matrix has state in its sessions.

  333. CognitiveDissonance

    I see.

  334. dwd

    CognitiveDissonance, I mean, IP is stateless and yet XMPP has state. TCP has state too. XMPP sessions can span multiple TCP sessions. So whether TCP happens to have state isn't important - and HTTP runs over TCP anyway.

  335. raghavgururajan considers xmpp like emacs. extend and extend and extend

  336. moparisthebest

    "HTTP runs over TCP anyway" eh not so much anymore, http up to 2 does, http 3 does not

  337. moparisthebest

    in a similar way that future XMPP won't have to :)

  338. CognitiveDissonance

    what?

  339. CognitiveDissonance

    http is application layer right? it requires tcp?

  340. moparisthebest

    http3 runs over QUIC which is like TLS but over UDP

  341. CognitiveDissonance

    quic??

  342. moparisthebest

    https://en.wikipedia.org/wiki/QUIC

  343. CognitiveDissonance

    nice

  344. jonas’

    (good thing we didn’t have DTLS…)

  345. moparisthebest

    https://en.wikipedia.org/wiki/HTTP/3 etc

  346. dwd

    Matthew Hodgson, of Matrix, once compared Matrix and XMPP as Mac vs Linux. I think perhaps iPhone versus Linux might be more apropos. Matrix gives you a bunch of stuff, and you get what you're given - though you can add more. XMPP gives you a low-level kernel, but almost everyone then gives you a distribution with a bunch of common software on top.

  347. CognitiveDissonance

    Thank you all folks!!!

  348. CognitiveDissonance

    I appreciate it.

  349. moparisthebest

    jonas’, DTLS is quite different than QUIC

  350. jonas’

    moparisthebest, I don’t care

  351. dwd

    I mean, finding a client or server that does XMPP and nothing else is *hard*.

  352. moparisthebest

    it's akin to UDP vs TCP, you can't just drop-in replace one with the other, different use cases

  353. Kev

    You mean RFC 6120 only? Impossible, I suspect.

  354. dwd

    Kev, I was thinking 6120+6121, which is in principle possible.

  355. raghavgururajan

    dwd: That is actually. Start minimal and extend. Keeps things simple and modular.

  356. Kev

    Sorry, I didn't mean it was technically impossible to do 6120 only (although I think it probably is, because you'd need to replace 6121 with something else for the routing rules), just that I would be amazed if there exists any such software. Although I'd be moderately surprised to find something doing 612[01] and nothing else, but not shocked.

  357. raghavgururajan

    *actually good

  358. moparisthebest

    try finding a browser that just does http

  359. dwd

    moparisthebest, Ooooh, I'm going to use that next time someone does the "Too many XEPS!!!111" argument with me.

  360. Zash

    HTTP has too many RFCs!

  361. raghavgururajan

    To rewrite my message: What XMPP does is good. It starts minimal and then extends. It keep things simple and modular. Don't you all agree?

  362. dwd

    Broadly, yes - I don't know how else we could have built it but start with a common base and add bits.

  363. CognitiveDissonance

    raghavgururajan I was looking for that difference in xmpp and matrix.

  364. dwd

    Matrix can avoid this by essentially being an open source project with a protocol spec - I'm not sure that there's any well-used third-party implementations, are there?

  365. moparisthebest

    matrix starts with the whole shebang and then you are stuck with it

  366. CognitiveDissonance

    Oh hey lovetox is here. I love gajim.

  367. moparisthebest

    and that brings things like "sorry our main server is so slow please move to another server"

  368. CognitiveDissonance

    gotta go though

  369. dwd

    But if there ends up being a bunch of independent implementations, then wholesale changes to the monolith spec are going to be much harder to arrange.

  370. moparisthebest

    vs xmpp's "we handle 10million+ simultaneous connections and 600+ messages per second no problem" https://www.process-one.net/blog/ejabberd-nintendo-switch-npns/

  371. Kev

    That does *heavily* depend on your traffic model, mind.

  372. dwd

    moparisthebest, Yeah, but I kind of got bored with scaling. I like that we can, but I'm more interested in smaller scale stuff now - you can do a lot more interesting stuff.

  373. moparisthebest

    sure it's just nice to know it *can*

  374. dwd

    Right, sure. And it's another strength - you can pick a server built to do massive scaling, or one built for low-bandwidth, or write your own in about 2,000 lines of Python.

  375. Kev

    Or one line of Perl.

  376. raghavgururajan

    > dwd‎: But if there ends up being a bunch of independent implementations, then wholesale changes to the monolith spec are going to be much harder to arrange. +1

  377. Zash

    or one looooooooong line of sed ;)

  378. moparisthebest

    if you find yourself rewriting your server in another language in the hope it might be faster you've probably made a wrong turn design-wise

  379. dwd

    moparisthebest, Oh, I think 2000 lines of Python will be slower. But I don't have to scale to 10 million users, so I have other criteria for success.

  380. raghavgururajan whispers LISP

  381. moparisthebest

    (that was a jab at matrix by the way)

  382. raghavgururajan

    ?

  383. raghavgururajan

    *ignore the ?

  384. moparisthebest

    wow I do have to give them props for this page https://matrix.org/docs/projects/try-matrix-now that's far superior to anything we have

  385. Kev

    They have a much less complex ecosystem than we have.

  386. Kev

    And that page is already looking pretty complex.

  387. Zash

    Kev: ORLY?

  388. Kev

    Z

  389. Kev

    I really need to fix that bug.

  390. moparisthebest

    really? if you only judged by https://xmpp.org/software/clients.html / https://xmpp.org/software/servers.html / https://xmpp.org/software/libraries.html you'd probably conclude we barely have an ecosystem at all

  391. Kev

    Zash: Unless I'm wrong (which is always possible), Matrix is led by Matrix. They don't have the complexities we do where our 'central org' (I don't have a better description) is community-run by competing projects etc.

  392. Zash

    http://screenshots.debian.net/packages?search=xmpp&show=with

  393. Zash

    Kev: Oh boy, let me tell you about Grid, the Matrix (protocol) fork (IIRC).

  394. Kev

    So I'm just wrong. Fair enough.

  395. Zash

    Kev: But yeah. It's probably comparable to back when Jabber Inc was a thing.

  396. Kev

    Jabber Inc was never the JSF, though.

  397. Zash

    Similar but different 🤷

  398. Kev

    Fair.

  399. MattJ

    > If I want to get a reliable xmpp client going on my Mac, I’m going to have to download and compile code, and I don’t actually know which code is the best bet, so chances are I’m going to have to download and compile multiple bits of code before I find one that works.

  400. MattJ

    Er, wrong chat but not I guess

  401. Zash

    Sure would be nice to have that nicer clients.html

  402. Kev

    Be nice to have nicer clients first :D

  403. Kev

    (Yes, I know Swift hasn't had enough love in recent times. Working on that at the moment, although what form it'll take is up in the air)

  404. wurstsalat

    Zash, very much +1

  405. dwd

    I am right in thinking Ted Lemon's talking bollocks there, surely? I mean, the Mac desktop clients aren't pretty, but there's a few that exist and they're all in binary form right?

  406. Kev

    I've no idea which room this is referring to.

  407. Kev

    But it is certainly true that XMPP clients on Mac tend to be precompiled.

  408. Zash

    Kev: IETF list

  409. dwd

    Kev, MattJ's quote is from a mail by Ted Lemon on ietf-disgust

  410. Kev

    Ah, ta.

  411. Andrzej

    BeagleIM is for macOS and is distributed in binary form but if you wish you can compile it yourself

  412. eta

    Zash, wait, what do you know about grid

  413. Zash

    https://mailarchive.ietf.org/arch/msg/tools-discuss/21-hD287xlxBkskiWxPl5vX6f8I

  414. Kev

    I wonder at what point I unsubbed from ietf-disgust. I used to be subbed, but realise now I've not seen a mail for years, so I probably made a (rare) Good Life Choice at some point.

  415. Zash

    eta: I know someone wrote a long rant about how insecure Matrix is, then went on to make their own fork of it.

  416. jonas’

    dwd, typo? :D

  417. eta

    Zash, yeah, Max

  418. moparisthebest

    hrm since when was XMPP considered part of the #Fediverse ?

  419. Zash

    https://the-federation.info/ lists XMPP, ergo XMPP is part of the Fediverse.

  420. moparisthebest

    yep that's where I noticed

  421. Zash

    Now proceed to accept this circular logic as truth. Thank you!

  422. moparisthebest

    I thought Fediverse was just ActivityPub based stuff TIL

  423. Zash

    I thought it dated back to OStatus, but I'm not so sure.

  424. vanitasvitae

    I think it depends how you define the Fediverse

  425. vanitasvitae

    Diaspora is also considered part of it and doesn't do AP either

  426. vanitasvitae

    So XMPP fits the definition of being federated and being able to do social networking

  427. Zash

    Is Email part of the Fediverse? :D

  428. moparisthebest

    if XMPP is then I'd have to argue SMTP is too

  429. Shell

    there is social networking stuff built on XMPP, it's probably fediverse

  430. dwd

    moparisthebest, EMail used to be before Google took it over...

  431. raghavgururajan

    Zash: Is there a homepage for Grid?

  432. raghavgururajan could find via searx

  433. raghavgururajan

    moparisthebest: Yes, SMTP and XMPP are fedearted protocols.

  434. moparisthebest

    yea I knew that, just not that every federated protocol was considered part of the "fediverse"

  435. dwd

    Please tell me they have usenet in there.

  436. Zash

    usenet distributed, not federated!!!!!!!eleven

  437. moparisthebest

    a ton I've never heard of but no usenet https://the-federation.info/#protocols

  438. moparisthebest

    zot? dfrn? all have more servers than the pitiful 53 XMPP servers and the 1 SMTP server

  439. moparisthebest

    strangely, not gmail.com

  440. dwd

    Zash, I don't think it described itself as federated - but nor did email.

  441. Zash

    possibly relevant xkcd https://xkcd.com/802/

  442. dwd

    Zash, But you can post a message to any NNTP server and it ends up on all of them somehow, right?

  443. Zash

    dwd: Just like Matrix!

  444. Zash

    dwd: "usenet is distributed, not federerated" was what I meant to write, before my sleepy brain decided to change the sentence structure in the middle

  445. Maranda

    To me it looks like a "much spinning" site

  446. Zash

    Maranda: https://www.youtube.com/watch?v=ldK1gQSSTSo

  447. Maranda

    😂

  448. Kev

    dwd: You know I'm interested in MAM-FC too ;)

  449. Zash

    dwd, Kev: Have you seen https://gist.github.com/mar-v-in/8a9a578d2137a0196744a32abf6aa0eb/ ?

  450. Kev

    If that's the same as the old mailing list post that suggested we must not know what we're talking about, yes.

  451. Zash

    I don't know about that

  452. Kev

    I thought there was some line in it about only understonding how we could have written it if we didn't understand the area.

  453. Kev

    I might misremember, or it might be a different post. Will see if I can read it tomorr.w

  454. dwd

    No, this one just says it's badly written, and was proibably written solely to prove him wrong.

  455. dwd

    In any case, *my* main problem with MAM-FC, currently - beyond it needing a lot of editing work - is that there's no way for a server to distinguish between a client updating its conversation view and a client paging through a conversation. RSM simply doesn't provide that, and the result is it gets a bit weird and broken, or else plain inefficient.

  456. lskdjf

    dwd, in that gist there is a whole page of arguments on what the person thinks are issues with MAM-FC. It's not fair to pick out one sencence you didn't like to be able to say "whatever". It would be good to hear why you think that the issues raised in the gist (e.g. paragraph 3+4) aren't actually issues.

  457. moparisthebest

    Kev: speaking of misunderstandings another reminder for the xep-0001 update :)