jdev - 2021-02-16

  1. debacle has left

  2. pasdesushi has joined

  3. pasdesushi has left

  4. pasdesushi has joined

  5. adityaborikar has joined

  6. SouL has left

  7. Vaulor has left

  8. pasdesushi has left

  9. belong has left

  10. kikuchiyo has left

  11. belong has joined

  12. Zash has left

  13. Zash has joined

  14. suohua has left

  15. Vaulor has joined

  16. paul has left

  17. Zash has left

  18. Zash has joined

  19. SouL has joined

  20. Yagizа has joined

  21. Vaulor has left

  22. fade123 has left

  23. suohua has joined

  24. fade123 has joined

  25. suohua has left

  26. SouL has left

  27. lovetox has left

  28. mikeye has joined

  29. Vaulor has joined

  30. suohua has joined

  31. suohua has left

  32. suohua has joined

  33. Stefan has left

  34. Stefan has joined

  35. SouL has joined

  36. lovetox has joined

  37. suohua has left

  38. SouL has left

  39. SouL has joined

  40. mac has joined

  41. Daniel has left

  42. Daniel has joined

  43. suohua has joined

  44. mac has left

  45. suohua has left

  46. mac has joined

  47. suohua has joined

  48. paul has joined

  49. mikeye has left

  50. tiaod has joined

  51. mac has left

  52. wurstsalat has joined

  53. Stefan has left

  54. suohua has left

  55. mac has joined

  56. mac has left

  57. mac has joined

  58. belong has left

  59. mikeye has joined

  60. belong has joined

  61. marmistrz has joined

  62. Stefan has joined

  63. Daniel

    sonny, here is the stacktrace https://paste.rs/adf

  64. Alex has left

  65. mikeye has left

  66. mikeye has joined

  67. Alex has joined

  68. mac has left

  69. Kev has left

  70. floretta has left

  71. jubalh has left

  72. goffi has joined

  73. Kev has joined

  74. belong has left

  75. sonny

    Ha it's a stream error, the server is closing the connection, not xmpp.js

  76. sonny

    Apparently because the server is waiting on xmpp.js and it's not doing what it expects

  77. sonny

    Can you reproduce with passing --unhandled-rejections=strict to node?

  78. sonny

    Also what server is that?

  79. oibalos has joined

  80. Daniel

    sonny, ok. nevermind then. but thanks for your help i got tasked to code this by someone else who needed this. but they don’t know anything about xmpp but it also had to be node because node is all they use

  81. Daniel

    i originally just tried to point them to a 'good' javascript library. and then this grew into more and more me hacking something i don’t understand :-/

  82. Daniel

    so i'm not seeing this error. only they do. and apperantly they are seeing this only when thy try this from a server on the other side of world

  83. Daniel

    so i thought i'd just increase the client side timeout and be done with it

  84. Daniel

    but now that it turns out to be something more weird i'm just gonna give up

  85. sonny

    Alright, please invite them to create an issue in the repository

  86. marmistrz has left

  87. marmistrz has joined

  88. belong has joined

  89. marmistrz has left

  90. mikeye has left

  91. debacle has joined

  92. marmistrz has joined

  93. fade123 has left

  94. fade123 has joined

  95. FireFly has left

  96. FireFly has joined

  97. alacer has left

  98. debacle has left

  99. pasdesushi has joined

  100. pulkomandy has left

  101. pulkomandy has joined

  102. pulkomandy has left

  103. pulkomandy has joined

  104. marmistrz has left

  105. alacer has joined

  106. Wojtek has joined

  107. belong has left

  108. kikuchiyo has joined

  109. mikeye has joined

  110. belong has joined

  111. pasdesushi has left

  112. pasdesushi has joined

  113. asterix has left

  114. asterix has joined

  115. lovetox has left

  116. pasdesushi has left

  117. mikeye has left

  118. lovetox has joined

  119. pasdesushi has joined

  120. marmistrz has joined

  121. floretta has joined

  122. pasdesushi has left

  123. fade123 has left

  124. fade123 has joined

  125. asterix has left

  126. asterix has joined

  127. asterix has left

  128. asterix has joined

  129. asterix has left

  130. asterix has joined

  131. asterix has left

  132. asterix has joined

  133. marmistrz has left

  134. asterix has left

  135. asterix has joined

  136. asterix has left

  137. asterix has joined

  138. belong has left

  139. tiaod has left

  140. belong has joined

  141. tiaod has joined

  142. mac has joined

  143. Wojtek has left

  144. Wojtek has joined

  145. asterix has left

  146. asterix has joined

  147. asterix has left

  148. asterix has joined

  149. Meta Bergman has joined

  150. belong has left

  151. mac has left

  152. jubalh has joined

  153. jubalh has left

  154. Stefan has left

  155. cyril has joined

  156. Stefan has joined

  157. cyril

    Hello, I am trying to use the slixmpp python library synchronously

  158. cyril

    Like connect() do_some_stuff() close()

  159. cyril

    is there some examples using it like this ?

  160. Zash

    I've no recent experience with slixmpp but I'd try prefixing every line with `await` and see what happens. :)

  161. cyril

    Oh I was talking about sleekxmpp sorry

  162. Stefan has left

  163. cyril

    The thing is that the code using the library is not aware of coroutines

  164. Sam Whited

    sleekxmpp is unmaintained and deprecated, if you're starting a new project in python I'd use slixmpp or aioxmpp

  165. cyril

    Ok, I was talking of slixmpp, sorry 😅

  166. cyril

    The interface exposed to client have to look like somehing synchronous

  167. Sam Whited

    Maybe just take the echobot example from the docs and remove the message processing and do whatever you want to do in session_start instead?

  168. cyril

    with the event loop running in a thread

  169. cyril

    It kind of work but outputs some errors when disconnecting, like "ERROR Task was destroyed but it is pending!"

  170. cyril

    I was thinking maybe someone has already done this

  171. Sam Whited

    /cc mathieui (who mostly works on this, I think?)

  172. moparisthebest

    cyril: https://github.com/moparisthebest/sendxmpp-py

  173. Stefan has joined

  174. cyril

    Ok thanks, might be the self.disconnect(wait=True) thing!

  175. moparisthebest

    It stopped working for me years ago and instead of wasting more time with python I just rewrote it in Rust, might help you as an example though

  176. flow

    speaking of, what is the go to rust xmpp library?

  177. belong has joined

  178. jonas’

    Link Mauve, ^

  179. Sam Whited

    flow: https://crates.io/crates/tokio-xmpp

  180. Wojtek has left

  181. Wojtek has joined

  182. flow


  183. Wojtek has left

  184. Wojtek has joined

  185. pasdesushi has joined

  186. pasdesushi has left

  187. pasdesushi has joined

  188. pasdesushi has left

  189. cyril

    mathieui: Here is a minimal example of what I am trying to do https://pastebin.com/0VTQ7qtg

  190. cyril

    If you have any insight

  191. cyril

    That was for some quick unit tests, learning rust would be a bit overkill but it is on the todo list!

  192. serge90 has left

  193. serge90 has joined

  194. pasdesushi has joined

  195. edhelas has left

  196. edhelas has joined

  197. Martin has left

  198. Martin has joined

  199. pasdesushi has left

  200. kikuchiyo has left

  201. kikuchiyo has joined

  202. mathieui

    cyril, I don’t have a lot of insight around using threads with asyncio, I know there are some barriers in place to prevent people from doing weird things

  203. mathieui

    you should keep all slixmpp interactions in the same threads otherwise things can get weird pretty fast

  204. cyril

    ok thanks

  205. mathieui

    but what you sent seems mostly correct, using process(forever=False) will run the event loop until the client gets disconnected

  206. mathieui

    we might get rid of it at some point as it is confusing for people coming from slixmpp

  207. cyril

    actually it tells me « ValueError: loop argument must agree with Future » when calling process(forever=False)

  208. cyril

    with the non minimal example it worked (with almost the same code…) but I got an error on disconnect

  209. mathieui

    cyril, it looks like a future was created on another thread, which then used a different event loop

  210. jubalh has joined

  211. mathieui

    (slixmpp tries to use one specific event loop and only one, but it will not help if things happen in a multithreaded environment)

  212. tiaod has left

  213. cyril

    yep, I will try again with everything in the same thread

  214. cyril

    that was the plan anyway

  215. southerntofu

    hey is this a good place to submit criticism of MIX spec? or should i wrote a mail/blogpost?

  216. mathieui

    FYI you can have something like def func(self): xmpp = ClientXMPP(…) xmpp.connect() xmpp.loop.run_until_complete(xmpp.wait_until('session_start')) do_stuff() # you need to run the loop for most network operations though loop.run_until_complete(xmpp.disconnect()) but it is not a very pleasant workflow

  217. mathieui

    southerntofu, I guess an email to standards@ might be a more persistent venue for this?

  218. southerntofu

    thanks, makes sense

  219. tiaod has joined

  220. belong has left

  221. mathieui

    I should finish the slixmpp implementation too, it is already working but lacks mix-admin or mix-misc

  222. oibalos has left

  223. oibalos has joined

  224. southerntofu

    hmmm i see no intent (yet) to support in dino, gajim, conversations according to bugtracker

  225. southerntofu

    is it because people are so happy with MUC? or because there's no momentum yet?

  226. southerntofu

    i'm asking because there's some privacy concerns i'd like to address with some MUCs (eg. occupant ID) and i was told MIX was probably the way to go for feedback on specs, instead of fixing/extending existing MUC XEPs

  227. pep.

    honestly if you're waiting for MIX to fix this you're gonna wait for some more time

  228. pep.

    There's two camps, choose yours :x

  229. Sam Whited

    I don't have any good reason, but I suspect it's a mix of MUC working "good enough" once the implementation is done (even though it's a mess under the hood), so there's no user pressure ot move, and MIX ending up being really complicated and requiring a lot of big dependencies (pubsub/pep).

  230. southerntofu

    pep., are you saying some people are not convinced by MIX and likely won't be?

  231. pep.

    southerntofu, that

  232. Sam Whited

    So you've got the double whammy of no user pressure and devs don't want to read 46 pages and implement a ton of new things.

  233. southerntofu

    well that makes sense Sam Whited

  234. Sam Whited

    That being said, there is slow uptake. There are experimental implementations in a couple of servers and clients, at least.

  235. southerntofu

    Sam Whited, where can i see what clients/servers implement it?

  236. southerntofu

    my 4 clients of choice apparently don't :P

  237. moparisthebest

    southerntofu: it's just tigase

  238. mathieui

    southerntofu, siskin is one

  239. Zash

    ITYM if you want MIX you should implement MIX

  240. southerntofu

    Zash, not sure i *want* mix, though building upon standardized building blocks for multi user chatting sounds appealing :)

  241. Sam Whited

    I think there were experimental implementations in ejabberd, prosody, and tigase (those devs will hopefully correct me if I'm wrong) and the tigase people had an implementation in their clients I think. Unsure if it's released or just on a branch somewhere though.

  242. mathieui

    But yes, there is MUC which has worked well enough for years despite all of its warts, which we can mostly work around by now, and MIX which requires both pubsub support, multiple (albeit quite short) specs, and home server support

  243. pep.

    And also has different trade-offs really

  244. southerntofu

    pep., what's the tradeoffs of MIX?

  245. pep.

    Not saying I'm against MIX or pro-MUC, but they're not equivalent

  246. mathieui

    pep., well, it is more modular and extensible, and the workflows aer nicer from a client dev perspective

  247. mathieui

    the main downside is the requirement of home server support

  248. southerntofu

    and do you folks know if there's some people working on client/server test suite for MIX? soundsl ike it could help imlpmenetaiton

  249. Zash

    We could make it *drumroll* presence-based, like MUC, then you don't need home server support

  250. mathieui

    Zash, please no

  251. Zash

    IIRC there was brief discussion on some summit about that

  252. southerntofu

    mathieui, what do you mean home server? my understanding is if your server doesn't support it a MIX node can provide a MUC compatibility layer, correct?

  253. mathieui

    southerntofu, it can be done, yes, but then from a client perspective you are not using MIX

  254. mathieui

    presence-based is my main peeve with 0045

  255. southerntofu

    yeah i also dislike the whole "presence" concept (to me it's an antifeature though i understand in corporate settings i can be useful)

  256. southerntofu

    presence-based chat sounds like IRC to me :)

  257. Zash


  258. mathieui

    southerntofu, presence is a potential privacy leak but also useful information

  259. mathieui

    and MIX does not remove presence, it just removes it from the workflow and makes its support optional

  260. southerntofu

    mathieui, i don't have a use case for it, but i'm not opposed to people broadcasting presence info.. but in my view opt-in is better :)

  261. Zash

    I can see how it was more useful back when you were online via dialup ~1h per day, but I don't see it as an anti-feature.

  262. Kev

    The presence of presence isn’t an anti-feature, in my view, but tying MUC’s workings to it is.

  263. Sam Whited


  264. mathieui


  265. Wojtek has left

  266. Zash


  267. Kev

    (Not because I don’t want presence in my rooms, I do, I just want it decoupled)

  268. southerntofu

    Zash, depends on the usecase, but it's a huge metadata leak (see discussion in #modernxmpp). i understand myhome server needs to know which clients are online to deliver messages, but i don't want anyone else to know about that, or whether i received a message or not

  269. Sam Whited

    It's a metadata leak that won't matter for 99% of people though.

  270. southerntofu

    but i can also understand people who want it

  271. Zash

    southerntofu: Presence should definitely be protected, and XMPP tries to do that.

  272. Sam Whited

    If you are that tiny percentage of people who need the absolute privacy you described, you need to be using a special service and clients and what not that are designed for that.

  273. Zash

    Why you need to add people and get their approval to see their presence

  274. southerntofu

    Zash, yes that's good, but that a MUC operator can see everyone's presence is a huge difference in the threat model (i.e. correlating identities) compared to simply subscription settings (at least when the convos are OMEMO-encrypted)

  275. Zash

    Yup. What Kev said.

  276. southerntofu

    Sam Whited, i don't understand your point.. should privacy just be for the 1%? i'm aware of the tradeoffs i make but most people are not..

  277. moparisthebest

    define "privacy"

  278. Sam Whited

    southerntofu: no, but we don't need to optimize for the tiny percent of people who are worried about the threat you outlined. Most people it's perfectly fine if they join a MUC and share their presence.

  279. Sam Whited

    We do a pretty good job of balancing that overall. Getting rid of all presence won't help the majority of people in any way, it will just annoy them.

  280. Kev

    I think it’s good to define protocols that can be used in a reasonably private manner.

  281. mathieui

    southerntofu, I don’t see a thread model that has critical protection requirements on presence data (although sure, protecting it as much as we can should be a goal)

  282. Kev

    Which MIX tries to do by allowing you to send or not send presence, but still participate in the chat, and it can be controlled on your server, so the room only needs to know when you send a message, or initially join.

  283. Kev

    Compared to MUC, where the MUC needs to know not only when you’re online, but how many (and which, practically) devices you have online at that moment.

  284. mathieui

    Kev, with MIX, a client only needs to not send presence to the MIX items in their roster, right?

  285. Kev

    OTOH, I think Sam is right that 99% (or some high proportion of people) would choose to have the benefits of sharing presence with the room of their friends or whatever, over the privacy.

  286. Kev

    mathieui: Yes, the client can just not set up presence and everything Should Just Work.

  287. southerntofu

    Kev, yes i think that's a huge improvement, however what started this discussion was in XEP 0403:

  288. southerntofu

    > A MIX channel MAY require that all participants publish presence, so that active channel participants are visible. It is not possible to enforce this in the server, so participants in a channel with this option MUST publish presence.

  289. Kev

    Practically, the important bit there for privacy is ‘It is not possible to enforce this’ :)

  290. southerntofu

    moparisthebest, defining "privacy" is a hard task.. there's different threat models, but i believe they should be more explicit in the specs :)

  291. Sam Whited

    To repeat my argument from there: if you need that level of privacy, you can't rely on the server to enforce it anyways, so if you're writing a privacy aware client you'll need to show the user the room policy and not join the room if it requires presence.

  292. southerntofu

    Kev, well i'm ok with a server mandating its rules, but why would the spec say my client MUST send my data??

  293. Kev

    southerntofu: We have the Security Considerations for that - but they’re only as good as the authors think of, and reviewers comment on.

  294. Kev

    southerntofu: Yes, there’s room for improving that wording.

  295. Sam Whited

    The client only MUST send the data if the room requires it. If the room doesn't require it (probably most rooms), you don't have to send the data. That's a pretty good privacy tradeoff.

  296. southerntofu

    Kev, wouldn't it be useful to have a dedicated mailing list / MUC to discuss privacy/security across the ecosystem and evaluate upcoming specs?

  297. mathieui

    southerntofu, that’s supposed to be part of the global process

  298. Sam Whited

    Yah, probably best to have them in public, no need for more low-traffic lists.

  299. Kev

    It would be useful to have people reviewing privacy/security. That’s not quite the same thing :)

  300. mathieui

    Kev, while you’re there, I’m unsure about the resource examples from mix participant jids in 0405

  301. southerntofu

    Kev, well i'm no expert at all, but i'm always happy to take some time to give a critical overview from an outsider's perspective (i don't have background on why all the technical decisions have been made)

  302. Kev

    The upcoming specs all get sent out on the standards list where people can review and make such comments. Realistically the number of people reviewing upcoming specs is relatively low.

  303. marmistrz has joined

  304. mathieui

    are they supposed to mean something or to be public?

  305. moparisthebest

    southerntofu, yea "impossible to define" because it means different things for different people is pretty much my point, I'm in private MUCs with my family hosted on the server in my closet, so presence is not a privacy problem at all (there)

  306. moparisthebest

    specs do have security concerns which is generally where privacy related stuff lives too I guess ?

  307. Kev

    mathieui - can you give a bit of text for me to search on, or section number please?

  308. mathieui

    Kev, e.g. https://xmpp.org/extensions/xep-0405.html#usecase-user-presence-receive

  309. Kev

    The UUID-a1j/7533 bit?

  310. mathieui


  311. mathieui

    is it the resource of the client on the other side?

  312. mathieui

    and do we want to do resource leak in JID hidden channels?

  313. southerntofu

    moparisthebest, that makes sense though i'm not the best person for formal processes :P

  314. Kev

    This is the client’s resource, and it’s a long debate. We need unpredictable resources for a bunch of reasons. But at the Summit where we discussed this some people claimed that (despite XMPP already explicitly not guaranteeing your resource is the one you asked for) it was vital that they got the resource they asked for. So a two-part JID where the first half was server-provided, and the second half was client-provided was invented. It’s unrelated to MIX, it’s just the style that was used for those examples.

  315. southerntofu

    so anyway thanks for all these answers i'll further read the spec and try to come up with a patch, maybe write for the "security considerations"

  316. Sam Whited

    I still *really* hate that. The only argument that I remember was that it was needed for logging, in which case it seems like your logging system should just be better and not force server changes that you can't guarantee anyways.

  317. Sam Whited

    Requiring a weird two-part JID just feels like solving the problem from the wrong end and asking for trouble when people inevitably start assuming that's always the format and splitting on the / or whatever. Then again, maybe there were other better reasons I'm forgetting.

  318. Kev

    I *do* think we want clients to be able to be uniquely identified by their own server, FWIW.

  319. Kev

    But that’s not the same as saying it should be encoded in the resource.

  320. moparisthebest

    that's the thing that's optimizing for making it easier for human admins to scan XML streams and logs without server modifications right?

  321. moparisthebest

    always seemed beyond silly to me also

  322. tiaod has left

  323. Zash

    There's a disagreement on whether resources identify clients (ie are stable for the long term) or sessions (short lived).

  324. mathieui

    Ok, thanks.

  325. belong has joined

  326. mathieui

    I’ll pretend that this is an opaque identifier

  327. Sam Whited remembers the horrors of HipChat encoding stuff in the resourcepart that the client and server both had to be able to parse

  328. Sam Whited

    (not that that's what was happening with this split JID thing IIRC, but letting the client think it can guarantee anything being set is just too close for comfort)

  329. Zash continues to ignore all the /Conversations.XXXX resources everywhere

  330. Zash remembers Dino being upset if given a different resource than asked for (fixed now tho)

  331. Sam Whited just ignores anything the client requests and gives them a random resource.

  332. Kev

    I think what people encode in resources is up to them in a closed system (or if they’re a server in an open system, for that matter).

  333. Kev

    I know GTalk used to encode cluster routing information into the resources, and that seemed fine (smart, even) to me.

  334. Kev

    Wouldn’t work so well these days in the increasingly post-resource world, but back then it made sense.

  335. Sam Whited

    Oh yah, HipChat did that too and that part was nice, it was the bits the client had to understand that made it a problem, made upgrades difficult, broke them if the server didn't have it for some reason, etc.

  336. Sam Whited

    But yah, server sets the resource so if it wants to sneak some information in there it probably can't hurt anything.

  337. Sam Whited

    (assuming it does like I said and doesn't accept the resource from the client, which could lead to issues if a malicious client tries to sneak stuff in)

  338. raghavgururajan has left

  339. Yagizа has left

  340. debacle has joined

  341. pasdesushi has joined

  342. flow

    > A MIX channel MAY require that all participants publish presence how does a client discover this requirement? and how does a server know that a client complies?

  343. Meta Bergman has left

  344. flow

    mathieui> and do we want to do resource leak in JID hidden channels? there is a resource like in MIX hidden channels? could you point me to it?

  345. mathieui

    flow, I was pointing out that MIX-Presence supposed that you keep the resource even in the "anonymized" JID you get from the service

  346. mathieui


  347. mathieui

    ah no, it is addressed in 3.2

  348. flow

    3.2 of which xep?

  349. mathieui


  350. mathieui


  351. mathieui

    These JIDs will be used to represent specific JID clients. The resource associated with the encoded JID can be either of the follipwing two options: The resource value from the associated client JID; or A mapped valued to an anonymized value. This approach MUST be used with MIX-ANON.

  352. mathieui

    not a fan of the encoded JID thing, to be honest, but I can see how that is something you may want for plugging it into existing presence-handling code

  353. Sam Whited

    Oh crap, when I thought that I didn't want to re-read thsi earlier and was complaining about the length I forgot there are actually 7 other XEPs some of which I'd need to also read in order to understand this. Yah, maybe I won't get to that this weekend.

  354. Sam Whited

    I guess technically it works with just core, so I'm going to pretend I didn't see that and try not to despair.

  355. Zash

    So summary XEP the broken down series is too long by itself?

  356. flow

    to be fair, I believe groupchat protocols are simply not short

  357. flow

    mathieui, which "encoded JID thing" exactly?

  358. Sam Whited

    Yah, it's true, it is a *big* thing no matter how you slice it, but that doesn't make me feel better about the work I'd have to do to support it :)

  359. flow

    hmm, if you have a decent PubSub/PEP support than creating a minimal MIX client implementation should be "easy". OTOH I did not do it yet in Smack, so I could be proven wrong.

  360. flow

    hmm, if you have a decent PubSub/PEP support then creating a minimal MIX client implementation should be "easy". OTOH I did not do it yet in Smack, so I could be proven wrong.

  361. mathieui

    flow, user JIDs are "<opaque id>#room@service"

  362. mathieui

    see any example in mix-presence https://xmpp.org/extensions/xep-0403.html#usecase-presence-leave

  363. flow

    mathieui, yeah right, you need to stuff the four-tule of (MIX service, room name, room-user id, and room-user session id) into a JID, which has at most three parts

  364. flow

    I would have design it similar, so I don't see the issue with it

  365. flow

    (I was considering adding the room-user id into the resource, but it appears all the same in the end)

  366. flow


  367. Sam Whited

    Guess I'll have to read pubsub (20 pages) and its smaller and simpler cousine because pubsub was too complex, pep (25 pages) as well. (not that I think mix would have been better of without using existing building blocks, I just don't think it's feasable for me to actually implement all of this any time soon)

  368. raghavgururajan has joined

  369. Zash

    20 pages?

  370. flow

    Sam Whited, you have to look at it this way: MIX motivates you to implement PubSub, then PEP, and with that you have the building block for another dozens XEPs like bookmarks, avatars, …

  371. Sam Whited

    ish. I know it's not a great measurement, half of that is probably just version and appendices

  372. Zash

    How are you measuring?

  373. Sam Whited

    printed to PDF

  374. Zash

    Did you forget a zero?

  375. flow

    he didn't specify the paper format

  376. flow

    for all we know, it could be A2

  377. Zash

    The PDF version of 0060 is 182 pages, PEP is 19

  378. Zash

    or is this about one of the MIX XEPs?

  379. Sam Whited

    huh, that would make more sense, let's see what this did weird

  380. flow

    While cool kids have posters of their favorite swedish speed-death metal band on their wall, flow likes to look at walls full of specifications to read

  381. Sam Whited

    PEP is 20 for me (I was looking at the wrong one) but XEP-0060 appears to have printed (in that the first/last pages are correct), and it's also roughly 20, weird

  382. Sam Whited

    I did think it would be a lot longer thoug, so that's weird

  383. Zash

    Surely you mean 20 *stacks* of paper?

  384. edhelas

    still shorter than LOTR

  385. Zash

    We should resume the cut&paste we started in ... like 2014?

  386. flow

    Zash, IIRC it was 2015

  387. Sam Whited

    oh yah, there it goes, now it's 223 pages. No idea what happened last time or how it left pages out of the middle.

  388. jubalh has left

  389. flow

    The main problem you want to avoid with large specs like PubSub and MIX is unclarity about the required and optional parts. Ideally a XEP tells you early about the required parts and extension points for optional features, so that a developer can stop reading after them when starting an implementation. I think PubSub fails to do so.

  390. Sam Whited

    Yah, I tend to agree

  391. Sam Whited

    I was having a discussion with someone earlier who was saying they hoped the last holdouts from XMPP would switch over to working on / using Matrix, and while I pushed back against that for all the usual reasons, one of the things someone else said was that the Matrix specs were just a lot clearer and easier to build on top of even if you already started with a basic protocol implementation and were just developing features. I don't know if they had pubsub in mind when they said that, but I do suspec they're not wrong when I look at things like this.

  392. Sam Whited

    I don't really know how to improve that situation though, unfortunately.

  393. Kev

    Matrix is a chat system though, isn’t it?

  394. Zash

    It's a distributed JSON graph database.

  395. Sam Whited

    It's a protocol that's used for a lot of things in the same way XMPP is.

  396. Sam Whited

    But mostly for a chat system.

  397. Zash

    It does get further with "simple clients, complex servers" than current XMPP does, I'll give them that.

  398. Sam Whited

    Amusingly, this discussion took place over IRC.

  399. Sam Whited

    (for me, I assume they were on Matrix)

  400. asterix has left

  401. asterix has joined

  402. fade123 has left

  403. jubalh has joined

  404. flow

    I find the Matrix specifications on https://matrix.org/docs/spec/ heavily API centric, I am missing a description of their basic principles

  405. mathieui

    flow, that was also my conclusion a few years ago, this is an API documentation at most (maybe that got better even if it is not what you would like, I don’t know)

  406. flow

    I just skimmed over it, it appears to be a bit better now

  407. Zash

    JSON Web API all the things.

  408. flow

    eventually I always wondered if that "distributed graph" thing matrix does could also be done on top of XMPP, I see no reason why not

  409. Zash

    Sure you can

  410. raghavgururajan has left

  411. flow

    that would allow you to exploit an existing federated network, but HTTP and JSON where probably to tempting to use

  412. Sam Whited

    Sure, but if we write another 200 page spec to do it and mock them for using json instead of getting our own house in order it's probably not going to be very helpful.

  413. flow

    mocking them for using json is not helpful, right

  414. flow

    but writing and implementing a spec similar to matrix's distributed graph thingy should be fine

  415. Zash

    flow: you thinking MAM replication instead of routing, or something?

  416. flow

    right, but then again, I am fine with centralized room management

  417. flow

    not sure if we need to have something that provides eventual consistency

  418. raghavgururajan has joined

  419. flow

    there are surely distributed systems where you want to have eventual consistency, but can't think of a reason why federated groupchats should be based on such a system

  420. flow

    if you want availability, use an server implementation that supports clustering

  421. flow

    I'd love to hear the take on a Matrix person on that

  422. mathieui

    flow, I don’t really have much insight over how matrix works, I assume the replication & eventual consistency allow a more "resilient" view over rooms, without additional requirements for the server "hosting" the room for real

  423. Zash

    they usually compare it to git

  424. mathieui

    You can probably interact with the people on your own server who are also in that room, and that counts as delayed writes when the "real" room gets back online

  425. mathieui

    I mean, that is an interesting property to have, against outages, censorship, etc

  426. mathieui

    (I am only speculating obviously, I don’t know if that is even possible, as reconciliation between all of that state must be a real pain in the neck)

  427. flow

    mathieui, use a high-availability cluster to reduce outages, planed maintainance is usually not an issue, not sure how it helps against censorship

  428. Zash

    flow: you can remove a server and the room lives on on other servers

  429. mathieui

    flow, in the event of a server being bleeped out of existence by a powerful actor, instead of a whole lot of nothing, you have split communities across the ecosystem

  430. flow

    also, what incentivies servers to keep state of rooms the don't "own"

  431. Zash

    flow: now that's a question

  432. Zash

    and how do you enforce moderation, bans etc?

  433. flow

    potentially someone will answer, "no server owns a room", but the question still applies

  434. mathieui

    flow, well for matrix, "because that’s the protocol", but otherwise, I don’t know

  435. southerntofu

    flow, my understanding of the matrix model is censorship-resilience.. if my server with high availability gets seized/disconnected/attacked other people can continue chatting as if nothing happened.. it's not in my threat model, but i understand the value of it eg. for a wikileaks chan :)

  436. Zash

    blockchain? probably blockchain.

  437. southerntofu

    (ah people just said the same sorry i was reading backlog)

  438. flow

    southerntofu, sure that is a very nice property in theory. but doesn't that somehow imply that all servers need to keep all the state for eternity?

  439. mathieui

    (and if that is a desired property, then you need to put it in the base protocol to have the incentive "this must work")

  440. flow


  441. Zash

    I tried reading the s2s spec once and it gave me a headache. Quite the opposite of XMPP s2s where you just open a stream and send stuff.

  442. southerntofu

    flow, not necessary for all eternity, but sure that's the main problem with matrix servers lol

  443. Daniel has left

  444. mathieui

    I don’t want to try to read the specs again, but I imagine you must have the latest state to be able to send writes to the upstream server

  445. southerntofu

    i mean i understand why you would support this use case in some circumstances, but making it the default takes hardware requirements through the roof

  446. mathieui

    which require you to effectively maintain the state internally

  447. flow

    mathieui, I don't think this is true

  448. flow

    if it is a dag, then you just need to have some dag node

  449. flow

    and eventually someone will reconcile

  450. flow

    but you should be able to write without having the latest state, I mean that is the core of eventual consistency

  451. southerntofu

    flow, though another interesting property is because so many beefy servers store the state, you can host your server on a very unstable uplink that often gets disconnected from the internet, without resulting in eg. netsplits alla IRC :)

  452. mathieui

    I don’t know how the state is managed and what is accepted and what isn’t so I will not try to extrapolate further

  453. flow

    southerntofu, that's actually a very good point

  454. flow

    XMPP links are very fragile if the reciving end is on a low-bandwith connection

  455. flow

    XMPP links are very fragile if the receiving end is on a low-bandwith connection

  456. flow

    IMHO one of XMPPs biggest weakness

  457. flow

    OTOH, MIX/MAM mitigates this (a bit)

  458. southerntofu

    flow, yes that's why FMUC was so interesting, though i still have to finish reading MIX spec because they said there was something equivalent :)

  459. Zash

    Still tricky with s2s outages

  460. flow

    Zash, right

  461. flow

    I think if I where to do XMPP again, I would base it on pulling, not pushing

  462. flow

    because with pulling, the receiver can determine the rate

  463. eta drops https://theta.eu.org/2019/10/10/nea-federation-design.html in here

  464. jubalh has left

  465. southerntofu

    flow, doesn't PubSub support pull modes?

  466. flow

    southerntofu, sure it does

  467. flow

    but I was talking about pulling a first-class principle

  468. Zash

    This reminds me, has anyone looked at MLS? I read the architecture doc and it talked about requiring guaranteed eventual delivery.

  469. flow

    e.g. sending a message is just putting the message into your server's archive, the server informs the recipient about the new message. once the recipients server has pulled the message, the "pending messages" flag is cleared

  470. southerntofu

    flow, isn't that what happens already when an s2s message can't be delivered?

  471. southerntofu

    isn't there like MAM for s2s? :P

  472. Zash

    southerntofu: there is not

  473. southerntofu

    oooh, my bad for assuming otherwise :)

  474. flow

    southerntofu, not exactly, and even if, not potentially

  475. southerntofu

    sounds like it could be useful

  476. Zash

    MAM for s2s would pretty much be the "matrix over xmpp" flow mentioned earlier

  477. southerntofu

    well matrix over XMPP would be that + guaranteeing eventual consistency between several nodes who all act as authority for a resources (which is way more comlpex)

  478. flow

    Isn't "MAM for s2s" a little bit to vague? Hmm, but yes, if it is vague, then it could be pretty much everything ;)

  479. Zash

    flow: You could also figure out s2s+198 with resumption.

  480. Zash

    Not graph stuff tho

  481. flow

    Zash, sure, but what I have in mind is much more radical

  482. Zash

    flow: wanna write a xep about `<previous id="mam id of previous message"/>` and how to use that to magic together archives?

  483. southerntofu

    eta, i love your article (not finished reading just yet), the "sponsoring servers" is precisely what MX/NS secondary servers are about :P

  484. flow

    Zash, once I get a better grasp about where this is exaclty going, why not. :)

  485. flow

    uh, and I would need to fall into a pot of gold first, so that I can take the required time to do so :)

  486. Zash

    flow: Lets you detect holes in archives at least, if you receive a message with a pointer to a previous message you don't have. Throw some graph theory at it and call it Matrix over XMPP.

  487. flow

    hmm, having an extension that points to the preceeding message in <{mam:2}result/> actually does not sound like a bad idea

  488. flow

    we don't we have that already?

  489. southerntofu

    eta, https://xmpp.org/extensions/xep-0289.html sounds a lot like what you're talking about (federated MUC) :)

  490. Zash

    flow: not unless you count rsm, but it's not excatly that

  491. flow

    thinking about it <{mam:2}fin/> is probably the better place

  492. Zash

    flow: I was thinking about this for live messages

  493. Zash

    For patching up after s2s outages

  494. Zash

    Remember how MIX and s2s outages is an unsolved problem :)

  495. eta

    southerntofu: indeed!

  496. eta

    shame nothing implements it though

  497. Zash

    eta: you don't happen to have markdown sources for that blog somewhere? I like to convert to .epub for reading and source files tend to be less messy than converting from html.

  498. eta

    Zash: sure https://theta.eu.org/lx/selif/zmc58s7b.md

  499. eta

    (can do other posts if you want)

  500. Kev

    Zash: Undocumented solution, but I’m sure I discussed how to address it at the summit last year or the year before. Indeed, just by signalling there’s a hole.

  501. southerntofu

    eta, it doesn't look over complicated to implement.. also i need to finished reading MIX spec but it looks like there's some stuff about that in there too :)

  502. fade123 has joined

  503. pasdesushi has left

  504. oibalos has left

  505. oibalos has joined

  506. asterix has left

  507. asterix has joined

  508. lovetox_ has joined

  509. eta has left

  510. eta has joined

  511. eta has left

  512. jubalh has joined

  513. eta has joined

  514. asterix has left

  515. asterix has joined

  516. marmistrz has left

  517. goffi has left

  518. goffi has joined

  519. marmistrz has joined

  520. jubalh has left

  521. stpeter has joined

  522. kikuchiyo has left

  523. mikeye has joined

  524. Ge0rG has left

  525. marmistrz has left

  526. Ge0rG has joined

  527. Ge0rG has left

  528. Ge0rG has joined

  529. raghavgururajan has left

  530. jubalh has joined

  531. jubalh has left

  532. oibalos has left

  533. oibalos has joined

  534. friendlyOtter has joined

  535. friendlyOtter

    On xmpp.org they mention stroke as client library. A quick look around in the java world and i found camel-quarkus-xmpp. Any advice/remarks on that one?

  536. Sam Whited

    I haven't used it, but that looks like it's just a simple thing to get XMPP messages into/out of their integration framework, not a full XMPP library?

  537. Sam Whited

    There's also babbler if you're just looking for something in Java: https://bitbucket.org/sco0ter/babbler/src

  538. friendlyOtter

    ok thnx i'll look at that too

  539. pasdesushi has joined

  540. pasdesushi has left

  541. pasdesushi has joined

  542. debacle has left

  543. friendlyOtter has left

  544. pasdesushi has left

  545. pasdesushi has joined

  546. oibalos has left

  547. goffi has left

  548. riau.sni has joined

  549. riau.sni has left

  550. pasdesushi has left

  551. 00 has joined

  552. asterix has left

  553. larma has left

  554. Vaulor has left

  555. asterix has joined

  556. mikeye has left

  557. pasdesushi has joined

  558. 00 has left

  559. larma has joined

  560. tiaod has joined

  561. pasdesushi has left

  562. pasdesushi has joined