XSF Discussion - 2019-09-14

  1. rion

    > [11:26:06] <Zash> Dear lazyxmpp, I wish for a PRECIS implementation suitable for use in C I need one too

  2. flow

    join forces and consider starting one :)

  3. Ge0rG

    Patch libidn?

  4. Daniel

    What is the threat analysis for 'resource/presence leak'? Since for ever the community has treated that like something really bad. I never really understood why. I know that it makes some IQ based attacks easier. But really those can also be done over MUC and we generally don't put huge warnings signs up before someone joins a muc

  5. ralphm

    Security Considerations are there to allow people to make informed choices when implementing features. There are always tradeoffs between functionality, UX, and security. Something being called a presence leak doesn't mean it is 100% bad and you MUST NOT allow for it ever.

  6. Daniel

    Right. I'm just trying to figure out if I'm missing something

  7. flow

    Daniel, could potentially drain your mobiles battery if I know your resource

  8. flow

    (not saying that it isn't possible otherwhise too)

  9. flow

    besides that, joining a MUC is an explicit decission to reveal your presence and potentially your resource. Leaks are typically unintentional

  10. ralphm

    Right, but for MUC you could decide to only send availability once and not sync it with your away/xa/etc

  11. ralphm

    As an example

  12. ralphm

    That then leaks less information

  13. Daniel

    To be clear. (our terminology is a bit confusing here) I'm talking about leaking the resource. Not leaking availability

  14. ralphm

    Through MUC?

  15. Daniel

    Mostly Auto responding to certain messages

  16. jonas’

    Daniel, it can be used to scan for accounts

  17. jonas’

    normally we take some care to avoid that being possible unless you know a resource

  18. ralphm

    I think it is not so much about leaking resources per se, as you generally reveal them to your contacts anyway, but making them unpredictable

  19. pep.

    jonas’: vcard-temps and public omemo keys allow that

  20. pep.

    jonas’: vcard-temp and public omemo keys allow that

  21. flow

    I think it is about leaking the resource per se

  22. pep.

    (Scanning accounts)

  23. flow

    Remote entities shouldn't know that there is a connected client if they are not explicitly allowed to

  24. flow

    andt hen even be able to send data directly to that client

  25. Daniel

    flow, I get that I guess… That is a slightly different argument than leaking the resource (as in that specific string)

  26. Ge0rG

    Daniel: I agree with you that the leaking is overhyped as a security issue

  27. Daniel

    https://www.businessinsider.de/ihr-koennt-bald-offenbar-nachrichten-von-threema-auf-whatsapp-verschicken-2019-9 IM regulation wrt to interop is happening and nobody has asked us for our expertise. is there a bank account we need to wire money to to get heard? google translate link: https://translate.google.com/translate?hl=&sl=auto&tl=en&u=https%3A%2F%2Fwww.businessinsider.de%2Fihr-koennt-bald-offenbar-nachrichten-von-threema-auf-whatsapp-verschicken-2019-9

  28. pep.

    I wonder if Matrix also tried and also got ignored

  29. ralphm

    It reads to me that this is a paper goal, and I don't think that half 2020 is realistic if that's the case.

  30. Daniel

    obviously 2020 is not realistic. but that's not the point. the point is that they seem to be somewhat serious about attempting it

  31. ralphm

    The politicians or the companies?

  32. Daniel

    the politicians

  33. ralphm


  34. ralphm

    I'm curious if Ge0rG has heard back from his contacts.

  35. Ge0rG

    ralphm: no news yet. But I'll try to reach out again, I was told to remind them some months in

  36. Ge0rG

    That article is a good trigger for a reminder, actually

  37. ralphm

    This article seems like a good reason to. Was that contact in one of those parties?

  38. Ge0rG

    ralphm: I don't think so, he's an employee of one of the agencies

  39. Ge0rG

    ralphm: I'd love to be able to propose a way forward for E2EE

  40. Ge0rG

    This is something he was most concerned about

  41. Ge0rG

    MLS is probably something we need to evaluate properly in advance

  42. pep.

    > There is a lobby battle ahead Thank you article, I wasn't expecting that.

  43. Daniel

    also no sources to when and where they said this

  44. Daniel

    but who needs sources in 2019

  45. Ge0rG

    pep.: sharpen your axes, we are going full in

  46. ralphm

    Also curious if governments want 'in' on the conversations.

  47. pep.

    Ge0rG: the XSF has no resources for this kind of battle. Companies using XMPP would, more than us already anyway

  48. Daniel

    that's only a problem if you believe that democracy is about who can bring the most resources to a battle

  49. pep.

    I believe that's a harsh reality

  50. ralphm

    Companies are people now, in some jurisdictions, so arguably the are part of the demos.

  51. Daniel

    maybe we need to go the indirect route and convince 'our' lobby groups (CCC, FSFE, …) first

  52. Ge0rG

    Daniel: we've seen recently in the copyright debate that it's not about "resources" in general, but about money to politicians. Having 200k people demonstrating on the streets is meaningless

  53. pep.

    Daniel: that might help. EDRi also maybe

  54. ralphm

    I was also thinking about IETF

  55. Ge0rG

    ralphm: is IETF a political lobby org?

  56. Daniel

    i actually tried to talk to a representative from the FSFE about that and he was like: no I don’t want that i just want free software; to which i replied but that's not the point and also having them interop would actually strengthen free software because network effect; and he was like: but we just want to convince everyone to use Conversations; to which i was like: …

  57. Daniel

    i don’t think the IETF is a lobby group in the sense that the CCC is one

  58. pep.

    Daniel: can we put up a talk for CCC about this?

  59. Daniel

    IM interop? yeah i still have one in the pipeline for the 'privacey week' but they haven’t accepted it (yet?)

  60. pep.

    What do we know about it already? Maybe we should also contact matrix

  61. Daniel

    the idea was if it goes well at privacy week i'll do the same one at ccc

  62. pep.


  63. ralphm

    Ge0rG: no but they, under ISOC, might have ideas on how to approach this.

  64. ralphm

    And I'm not surprised at all about FSF(E)

  65. pep.

    I hear FSF/FSFE can be quite different btw, I haven't experienced that myself though

  66. Ge0rG

    Maybe we need to find someone else from the FSF then?

  67. ralphm

    Yes, there's also recently been considerable friction between them

  68. ralphm

    And I think neither cares about protocols

  69. Ge0rG

    Anyway, we need to act now.

  70. pep.

    I guess this kind of talk at POSS will fall in deaf ears

  71. Daniel

    maybe i should try calling those two people

  72. Ge0rG

    Daniel: keep us up to date please. I'm going to write to the ministry employee early next week

  73. Daniel

    I need to figure out what point I'm trying to make and how to get that across

  74. Ge0rG

    Daniel: "please keep moving this, good work! Demand open standard APIs! Hire me to make E2EE work!"

  75. flow

    > Daniel> flow, I get that I guess… That is a slightly different argument than leaking the resource (as in that specific string) I think it is the same argument assuming the leaked resource(string) is one of a "currently" connected client

  76. moparisthebest

    Ge0rG, Daniel: https://www.theregister.co.uk/2019/05/28/german_government_encryption/

  77. moparisthebest

    Maybe they are excluding open standards on purpose

  78. ralphm

    Now, what did I say?

  79. moparisthebest

    Nice chat network you got there, sure would be a shame if some regulation happened to it, maybe if you were to give us plain text though...

  80. Yagiza

    *GIRL CRY*

  81. moparisthebest

    What you don't trust the govt with everything? I mean in all the history of the German gover... Oh right...

  82. ralphm

    Don't mention

  83. larma

    Daniel > i actually tried to talk to a representative from the FSFE about that Who did you talk to? Also I doubt interop will work with encryption, as there has to be an official, gov-supported encryption standard for that. So if any, we get unencrypted interop and encryption will only work within a messenger network.

  84. larma

    What could make sense though is to require messengers to have easy data export in a standard format that could then be imported in other messengers, making it easier to move to a different network

  85. Daniel

    right. because nobody has ever created standardized encryption before that works across different providers and vendors

  86. larma

    Daniel, we don't even have it within a single network, how should that work across networks?

  87. larma

    We have about 5 different e2e-encryption standards implemented by clients in XMPP

  88. larma

    (only counting those in well-known open source implementations)

  89. larma

    And it makes sense to have multiple, because they have different properties.

  90. ralphm

    And also, you'll probably want the ability to upgrade to newer/better approaches in the future.

  91. Daniel

    and six part time open source developers not being able to agree means that you can’t do it?

  92. larma

    Even TLS is a monster of different encryption protocols. It just works because connections are short living and only between two entities.

  93. ralphm

    If you get parties to federate based on XMPP, even if it just unencrypted, you already have a connected network. Whatever e2ee can then be put on top.

  94. moparisthebest

    Within just a few years here omemo was invented and adopted by all major clients, so I really don't know what you mean larma

  95. larma

    Not by all major clients, just by a subset for a specific niche (personal im)

  96. ralphm

    But omemo is from the end-all-be-all, though.

  97. moparisthebest

    Also larma TLS is just for short lived connections????

  98. moparisthebest

    Are you talking about something else

  99. ralphm

    And I purposely turn it off because it invariably fails to work properly with my collection of multiple clients.

  100. larma

    Have you seen a TLS session running for several months as do OMEMO sessions?

  101. moparisthebest


  102. larma


  103. Zash

    XMPP s2s connections would

  104. larma

    Zash, several months? I hope you install updates, don't you?

  105. moparisthebest

    Wait until you hear about quic where sessions can survive reboots and network changes etc

  106. ralphm

    I've been mentioning (IETF) QUIC a couple of times.

  107. ralphm

    I think it definitely deserves an experiment to have an XMPP binding for.

  108. Daniel

    also i'm afraid that the people who want to regulate IM won’t stop because larma thinks E2EE (a subset of the entire thing) can’t be done. so I'd rather be part of the debate and help them do a slightly less shitty job than they would do if they'd only listen to facebook and google

  109. Zash

    larma: that's besides the point. and there can be months between updates to the servers or tls library that would mandate a restart

  110. moparisthebest

    ralphm: yea and I suspect you could do away with stream management inside quic too

  111. ralphm

    Daniel: to be honest, I would already settle for just the XMPP based federation, even without e2ee.

  112. ralphm

    moparisthebest: but it might be useful for file transfer

  113. moparisthebest

    Yes as long as they don't also ban e2e in the process

  114. ralphm

    Oh, I get what you meant, sorry. Yes, definitely.

  115. moparisthebest

    Cause as you said that can just be done over top of federation later

  116. ralphm


  117. ralphm

    moparisthebest: and about QUIC, yeah, there are quite a few benefits.

  118. larma

    QUIC probably won't survive reboots in practical scenarios as that would require to persist session keys I am talking about how TLS works in practice. Sure you can keep connections running longer, but ideally you don't so you can upgrade protocol version and ciphers for higher security at some point. And again TLS is between two entities. A group chat could consist of hundreds of different client versions if there was cross messenger interop...

  119. larma

    I guess I will be able to find a set of 5 TLS clients/servers that don't share a single common cipher compatibility

  120. moparisthebest

    So maybe you have Rock solid e2e between 2 entities and not as good in group chats?

  121. moparisthebest

    In fact that's what we have

  122. moparisthebest

    Is your point don't bother at all or?

  123. Daniel

    i don’t get why "it's going to be difficult" is an argument for "let's not even attempt to try this"

  124. moparisthebest

    Also lessons have been learned from TLS, not everything should be an option etc, have you seen 1.3 ?

  125. larma

    I just think it's unlikely to get to a common standard for e2e. There are a lot of voices against OMEMO in the XMPP community alone (for some good reasons). That said, for unecrypted interop at least I see potential. It could even be that we can require unencrypted interop and messengers can voluntarily do encrypted interop if they want to.

  126. moparisthebest

    It vastly limits options, and http2 requires a specific cipher for instance

  127. larma

    > Daniel > > i actually tried to talk to a representative from the FSFE about that > Who did you talk to?

  128. moparisthebest

    larma: right, and the voices against omemo didn't matter did they? All main clients support it

  129. Daniel

    larma, the guy who gave the free your android talk at gpn

  130. moparisthebest

    It's not perfect, but perfect is the enemy of good, and it's good

  131. ralphm

    moparisthebest: except omemo is only about text

  132. moparisthebest

    Right, so it covers 95% of the use case for IM then? Great

  133. larma

    Daniel, I will be at the FSFE european community meetup in November, so I might be able to bring up that topic. I know that the president of FSFE use(d) Dino, so at least there is a certain interest present.

  134. Daniel

    fwiw I personally see greater challenges when it comes to group chats. because different IMs do them vastly different. however that is all besides the point. the point is that THEY want to do this and 'we' have 20 years worth of experience on the *how*

  135. ralphm

    Well, one of the arguments here is that people needing to use this also want to hide who's talking to whom, and I don't see how to do this in XMPP.

  136. ralphm

    And another indeed groups and MAM.

  137. ralphm

    We do indeed have 20 years worth of experience on federation.

  138. ralphm

    And I would probably be a bit sad if for groups we'd have to settle on MUC.

  139. Daniel

    i mean we could explain to them what parts are more easily federatable (is that a word?) (1:1, unencrypted, simple media sharing); and I'd rather have us explaining that to them than letting Facebook Inc do the explaining

  140. ralphm


  141. ralphm


  142. flow

    > Daniel> i don’t get why "it's going to be difficult" is an argument for "let's not even attempt to try this" me neither

  143. ralphm

    Right. My reservations around e2ee are orthogonal to attempting to get services to federate.

  144. jonas’

    +1 Daniel

  145. moparisthebest

    Actually group chat is a really good example

  146. moparisthebest

    Everyone agrees it's hard, everyone agrees muc isn't ideal, still how many years later here we are

  147. Ge0rG

    We rather should try to support the involved law makers in ensuring that things aren't botched from the start.

  148. moparisthebest

    Mix is an attempt to make it more perfect, where is that

  149. ralphm

    moparisthebest: in progress

  150. ralphm

    And I'd love to have it sooner rather than later, but some of it is not easy

  151. moparisthebest

    I'm just saying it would have been silly to have avoided muc forever while waiting for mix

  152. Daniel

    MIX (and that's changing topics now) is pretty sad. I think it somehow got off to a wrong start and there is big resentment in the open source community toward it

  153. ralphm

    moparisthebest: sure

  154. moparisthebest

    Which is essentially what the "e2e is too hard" argument is

  155. ralphm

    Daniel: I mostly draw it up to ignorance, which is ok

  156. Daniel

    ralphm, maybe. but how can we make progress in that situation

  157. ralphm

    moparisthebest: no, MIX is achievable. I don't know about e2ee.

  158. Ge0rG

    Ignorance is also what I've experienced when trying to improve MIX. 🤷‍♂️

  159. ralphm

    I mean, I'm not discouraged by people looking at and going AARGH

  160. moparisthebest

    ralphm: except the opposite has happened in practice, we have working e2e, no working mix though

  161. Daniel

    i'm somewhat discouraged by people trying to "fix muc" with self ping, occupant id, and 100 others hacks like allowing members to read the affliations just to get something simple as a list of people who are in a room

  162. ralphm

    MIX needs to cater for a world with multiple devices, with a saner join model, and then some. I believe we can get it to be much easier for client devs. For servers there might be an impact that relies on server side support.

  163. Ge0rG

    ralphm: did you write your suggestions to standards@?

  164. Daniel

    or let me clarify that. i'm not discouraged by people trying to "fix muc" but by the fact that we have somehow given them the impression that this is the more achievable path than implementing MIX

  165. ralphm

    So various ideas there are solid, some need work. I'm going to take some time to look at that in the coming weeks

  166. Ge0rG

    ralphm: write a blog! 😉

  167. ralphm

    Ge0rG: I might

  168. Ge0rG

    Daniel: maybe the underlying problem isn't "MIX is too complex to tackle" but "server developers don't have enough time to make large changes at once"

  169. ralphm

    moparisthebest: as I said, I always disable e2ee because it doesn't work for me. Maybe it is just me.

  170. Ge0rG

    So that a path of small improvements of what's there already is working out better

  171. ralphm

    Ge0rG: but they do for each inventing their own

  172. Ge0rG

    Even if we are bound to end up in a local maximum.

  173. Ge0rG

    ralphm: I also always disable E2EE.

  174. Ge0rG

    ralphm: none of those parties have provided reasonable specifications yet.

  175. ralphm

    My point is that I don't buy that argument

  176. ralphm

    (from them)

  177. Ge0rG

    ralphm: I was mainly talking about the OSS implementations out there.

  178. ralphm

    Sure, that's a valid point

  179. Ge0rG

    And as MUC isn't going anywhere, it's worth fixing it.

  180. moparisthebest

    Doesn't change the fact that omemo "just works" for the majority of people

  181. ralphm

    I have no metrics to make that argument

  182. jonas’

    moparisthebest, ejabberd has a partial MIX implementation in the current release

  183. moparisthebest

    Most people don't use as many clients as you do etc

  184. jonas’

    I’m going to start a MIX implementation in aioxmpp against that

  185. moparisthebest

    Most people just have a phone only, the ones that have a desktop also use a client that supports it on there

  186. Daniel

    well the MUC that is not going to go away is this kind of muc; the muc that we are in right now. however that kind of muc doesn’t suffer that much under the limitations of MUC

  187. Daniel

    so it does’t have to go away

  188. moparisthebest

    Again, not perfect, but good, waiting for perfect gets you in the waiting for mix forever mode

  189. Daniel

    however the "private group chat" can easily be switched over. and will also benefit from what mix has to over

  190. Daniel


  191. Ge0rG

    Daniel: I agree. Still, somebody needs to write the code, then to find out the places where the MIX spec is broken, get them fixed and rewrite the code finally.

  192. Ge0rG

    I'm still not convinced that MIX is solving all the known technical problems of MUC.

  193. ralphm


  194. Daniel

    i kinda wish someone would address my MIX feedback so i could move on implementing it…

  195. Daniel

    or even invite me to do a PR i guess

  196. Ge0rG

    ralphm: my pet issue is message loss during s2s interruptions

  197. ralphm

    It would be good to have a list of those and document how such issues are, or are not (yet?), addressed by MIX.

  198. Ge0rG

    ralphm: I don't remember my other points raised around 2016.

  199. jonas’

    Daniel, you got a +1 and no -1nes on the PEP node thing. Please just do a PR

  200. ralphm

    Ge0rG: yeah, that needs work

  201. jonas’

    not sure where Steve Kille is

  202. ralphm

    Enjoying his weekend?

  203. Daniel

    that might be a tiny bit undeserverd but I never got the impression that MIX is welcoming to PRs

  204. Ge0rG

    ralphm: one of my points, the incompatibility of the roster with MIX entries, has been addressed two or three years later, it seems to me

  205. jonas’

    ralphm, for three months?

  206. jonas’

    Daniel, true, in a way

  207. ralphm

    As I said, I'm going to try and dislodge things

  208. Ge0rG

    Daniel: not just PRs, but any kind of change requests?

  209. jonas’

    Daniel, FTR, I added a +1 to that thread, maybe that gets it rolling again

  210. Ge0rG

    ralphm: feel free to make a list from what was written on standards@ over the last years

  211. Daniel

    one more than the other maybe

  212. ralphm

    Ge0rG: various comments I raised have been addressed by Steve last time around, though.

  213. ralphm

    And after that I was less involved, so didn't follow up

  214. Ge0rG

    ralphm: so you say I should re-read the XEPs and what I wrote in 2016, and just submit the open points as new feedback? I'd do that, but I lack the time.

  215. Daniel

    assuming that I would go ahead and make a PR. what is it that we want to do exactly? private pep node, multiple items, one item per channel, client can’t modify it. last item notify is off (so a client can +notify that node but not get bothered with the last item for no reason on connect)

  216. Ge0rG

    I wouldn't mind at all if somebody else did it.

  217. ralphm

    Understood. Time is precious. But I do have some, so am going to use it for MIX

  218. moparisthebest

    And to be clear my only point was no one is saying "group chat too hard we should give up" so why say the same about e2e that's all, sorry for starting mix discussion :)

  219. jonas’

    Daniel, all of what you say sounds excellent

  220. Ge0rG

    ralphm: > ralphm: feel free to make a list from what was written on standards@ over the last years

  221. Daniel

    and MIX-PAM will automatically send presence to all the items

  222. jonas’

    Daniel, if and only if presence is enabled for that MIX, IIRC you can turn that off?

  223. jonas’

    (that type of metadata should maybe be included in that PEP node?)

  224. ralphm

    Daniel: I am not following. What are you talking about?

  225. Daniel

    yes… so a client would have to be able to modify that paramater

  226. jonas’

    Daniel, wouldn’t that work via the existing PAM-IQ-flows?

  227. jonas’

    Daniel, wouldn’t that work via the existing PAM/MIX-IQ-flows?

  228. Daniel

    jonas’, i would have to look into that again

  229. jonas’

    let’s start it simple though

  230. Ge0rG

    Daniel: weren't there issues with multi item PEP?

  231. Daniel

    ralphm, moving channels out of the roster

  232. ralphm

    Daniel: I missed that was an issue. Can you explain why that is a problem?

  233. Daniel

    ralphm, didn’t you raise the same issue during the summit?

  234. jonas’ goes and fetches the standards@ link

  235. jonas’

    ralphm, https://mail.jabber.org/pipermail/standards/2019-June/036172.html

  236. Daniel

    mostly because it is mixing concerns

  237. Daniel

    i might not want a roster

  238. ralphm

    Daniel: my primary concern was that it requires servers to know about MIX. I'm not sure if I still think that is a problem. Sure it means that if your server doesn't you can use MIX. A counter argument is that at some point progress means you have to let that go.

  239. jonas’

    ralphm, servers need to know about MIX anyways, that’s '405

  240. ralphm

    Daniel: not having a roster indeed is a good point

  241. ralphm

    And I did raise that.

  242. Daniel

    i remember that

  243. Ge0rG

    ralphm: From my 2016 memory: non MIX clients need to get a copy of the roster without the rooms. This means servers need to disco a client before sending roster, and roster versioning becomes a mess.

  244. ralphm

    Thanks for bringing that up again.

  245. Daniel

    i also see virtually no argument in favor of putting it in the roster

  246. Daniel

    and a lot of (some bigger than others) arguments against doing that

  247. jonas’

    (FTR, the roster mail I linked was what I was referring to when I was ironically asking whether Steve was enjoying his weekend for three months now ;))

  248. ralphm

    Ge0rG: wait huh?

  249. Ge0rG

    Also you might not want automatic presence delivery to all rooms, but this is a minor issue.

  250. jonas’

    Ge0rG, the 2016 design still had only those MIXes where you’d send presence in the roster

  251. Ge0rG

    jonas’: which is also bonkers, because how do you know of the others?

  252. jonas’


  253. Ge0rG

    I remember the bad things that happen if you add a MUC to your roster.

  254. ralphm

    Knowing which channels you have joined is orthogonal from which channels you send presence to and the current roster integration conflates that? Yes I see that argument.

  255. jonas’


  256. Ge0rG

    jonas’: fun for an outside sadistic watcher, yeah

  257. Ge0rG

    ralphm: it would also require roster annotations to be inside the respective room elements to know those are not people

  258. jonas’


  259. jonas’


  260. ralphm

    Ge0rG: I disagree with that point

  261. Ge0rG

    Because, surprisingly, a client needs to know that when opening a chat window

  262. Ge0rG

    ralphm: it's not a MUST, but the alternatives are worse

  263. ralphm


  264. Ge0rG

    Say, disco#info to all roster entries after connecting.

  265. ralphm

    Or doing that right when while opening a chat.

  266. ralphm

    And caching it.

  267. Daniel

    not just opening the chat. but take the simple task of learning what channels you are joined

  268. ralphm

    Obviously CAPS are in issue for this.

  269. ralphm

    An issue

  270. Daniel

    because when starting the client you might want to have open tabs for all channels

  271. Daniel

    how would i do that if the roster items are not annotated

  272. ralphm

    Daniel: right

  273. jonas’

    (something about inbox)

  274. ralphm


  275. Ge0rG

    ralphm: you also need to know that when receiving a message from MAM. Which is why I've insisted on adding <x/> elements into MUC PMs.

  276. Daniel

    also you might want to learn what nodes you are subscribed to for a specific channel

  277. Daniel

    in MucSub there is a get me my 'channels' command. and then each items lists the nodes

  278. Daniel

    *currently subscribed nodes

  279. Ge0rG

    ralphm: making the UI depend on another roundtrip is a horrible idea. What if you just lost your network connection, or are on EDGE?

  280. ralphm

    Sure, I see that argument, but once you know it is a MIX Channel, that's forever, likely.

  281. Ge0rG

    ralphm: think of thin clients.

  282. ralphm

    Daniel: having to query the full list all the time is an issue, too, so you need some versioning there too?

  283. jonas’

    or web clients, for that matter

  284. Ge0rG

    Do they have to store a local list of MIXes now?

  285. Daniel

    also MIX-PAM already has those annotations

  286. Ge0rG

    ralphm: how is all that outweighing a simple child element / a dedicated list in PEP?

  287. ralphm

    I'm not saying it is

  288. ralphm

    Just exploring what's being said here. Happy to see all these points.

  289. Ge0rG

    ralphm: you said you disagree with that point.

  290. Ge0rG

    ralphm: I've argued all that back in 2016. It's nice to see we have finally moved beyond it.

  291. ralphm

    I disagreed that it has to be in the roster

  292. Ge0rG

    > ralphm: it would also require roster annotations to be inside the respective room elements to know those are not people ralphm [16:43]: > Ge0rG: I disagree with that point I read that as you disagreeing with annotations.

  293. ralphm

    Ge0rG: sure, to get anywhere we need to reevaluate all of it

  294. Ge0rG

    ralphm: please do, and blog it.

  295. ralphm

    Yes, sir. 🤣

  296. Ge0rG

    ralphm: or make a spreadsheet of problems and their respective solutions

  297. Ge0rG

    Because I'm honestly tired of arguing all that, over and again.

  298. ralphm

    I definitely want this to be easy and thin.

  299. ralphm

    E.g. I think having to ask every channel for MAM is not good.

  300. ralphm


  301. Ge0rG

    ralphm: I agree with the general idea. Just ping me once you've taken over authorship... 😉

  302. Daniel

    what is the counter argument of putting it in a private pep node roughly in i style of: > private pep node, multiple items, one item per channel, client can’t modify it. last item notify is off (so a client can +notify that node but not get bothered with the last item for no reason on connect)

  303. Ge0rG

    Daniel: it sounds sensible to me, but I haven't really worked with PEP, so don't know the pitfalls

  304. Daniel

    (note that servers could implement it as a virtual pep node)

  305. Ge0rG

    Daniel: Zash recently mentioned issues with having multiple items in one node. But those might be practical and not protocol limitations, so can be fixed in a MIX supporting server

  306. ralphm

    Daniel: might work, indeed. Was just mostly interested in the issues, rather than looking at possible solutions.

  307. jonas’

    the only pitfall I see is that it might not scale to enormous numbers of channels

  308. ralphm

    jonas’: I'm curious what is enormous

  309. Daniel

    can you RSM through pubsub items?

  310. ralphm


  311. jonas’

    Daniel, in theory, yes

  312. Daniel

    so i don’t see the issue

  313. jonas’

    ralphm, anything which breaks the stanza limit

  314. jonas’

    Daniel, you have to fetch the channel list on each reconnect, don’t you?

  315. Ge0rG

    Can you receive a delta of all PEP changes, given a certain initial state? From MAM?

  316. Daniel

    that's why it is multiple items

  317. jonas’

    Daniel, but if those are 1k items, the fetch will still be annoying on each connect, especially on mobile.

  318. Daniel

    also if you are afraid it will break your stanza size limit it will also break your roster fetch

  319. ralphm

    When I was in VEON, our Slack had many channels. I was in 40+, and then there are the unnamed 'channels' for talking to an immutable set of people.

  320. ralphm

    My daughter uses WhatsApp quite a bit

  321. ralphm

    Many many channels

  322. ralphm

    None of them ever closed

  323. ralphm

    (though many go unused)

  324. ralphm

    My current thinking is that PEP might not be the best model. If a server manages all your subscriptions, why would a client need to touch it? Maybe just a plain old iq would be good.

  325. Daniel

    so the argument against PEP would be that we that we don’t benefit from roster versioning

  326. jonas’

    ralphm, the PEP node is read-only to the client

  327. jonas’

    Daniel, we cannot benefit from roster versioning in any serious way anyways because for non—MIX clients the entries must not be in the roster

  328. jonas’

    it would at least make roster versioning much more complex on the server side

  329. Daniel

    i'm not overly attached to PEP. i just don’t like to use roster

  330. ralphm


  331. Daniel

    ralphm, there would need to be some kind of subscribe to changes mechanism. and +notify would _one_ solution to signal that

  332. ralphm


  333. Daniel

    also reusing syntax. my client might already have PEP notification handling

  334. Ge0rG

    > Can you receive a delta of all PEP changes, given a certain initial state? From MAM?

  335. jonas’

    Ge0rG, there are rumors about MAM for PubSub

  336. Daniel

    that probably doesn’t help you with PEP

  337. Ge0rG

    jonas’: you can't implement a rumor

  338. ralphm

    Just to be clear, this is hardly PEP, but node-on-account, but I digress.

  339. Ge0rG

    Whatever it is, I want a delta mechanism.

  340. ralphm

    jonas’: without MAM for pubsub, I'm unsure how we could do other nodes in MIX

  341. ralphm

    Ge0rG: noted

  342. Daniel

    > Just to be clear, this is hardly PEP, but node-on-account, but I digress. fair enough

  343. jonas’

    #undef PEP #define PEP PubSub-on-user-accounts

  344. Zash

    PEP Plus

  345. Zash

    XEP-0222, XEP-0223

  346. ralphm

    But but

  347. Daniel

    in an attempt to get to a point where I could actually make a PR. if we didn’t use a pubsub node on the account we would have to specificy: 1) querying the list; (possibly with RSM) 2) subscribing to the list (can be done with putting a mix namespace into caps) 3) new item notification 4) retract item notification am i missing anything?

  348. Daniel

    are those four tasks close enough to pubsub to justify borrowing the syntax

  349. Daniel

    or is it better to create own syntax for that

  350. Ge0rG

    Daniel: receiving actual deltas if you were offline for some weeks

  351. jonas’

    Daniel, provisions which would allow us to introduce versioning later

  352. jonas’

    ha, what Ge0rG said

  353. ralphm

    I'd not put in a PR at this point. Rather reiterate the issues and requirements.

  354. jonas’

    ralphm, the issues have been re-iterated on the ML since 2016

  355. jonas’

    nobody has objected to Daniels bringing it up three months ago

  356. jonas’

    I think we can move forward with proposing an update to this *Experimental* XEP to finally achieve some progress

  357. Daniel

    it doesn’t have to be a PR. and i'm happy not having to create one

  358. ralphm

    Ok, if you don't want to wait for me, maybe it'll work.

  359. Daniel

    but i would like to get to a point where i'm not being ignored

  360. jonas’

    ralphm, what timeframe do you anticipate? MIX has been "waiting" for years now.

  361. Daniel

    ralphm, wait for you? what are you planning to do?

  362. jonas’

    a few more weeks probably won’t hurt if you’re going to do an in-depth evaluation which might actually move us forward

  363. jonas’

    what Daniel is saying, effectively

  364. ralphm

    I have time on my hands and will try and dislodge the whole thing

  365. jonas’

    my dictionary does not list a sensible translation of `dislodge`

  366. jonas’

    my dictionary does not list a sensible (in this context) translation of `dislodge`

  367. ralphm

    Get it unstuck.

  368. jonas’

    I see

  369. Ge0rG

    I'm sure the underlying conditions have changed significantly since I tried the in depth evaluation.

  370. jonas’

    ralphm, I don’t mean to attack you. It seems just frustrating to re-iterate all the arguments we’ve had yet another time.

  371. ralphm

    Consider the discussions on MAM2 in light of MIX.

  372. jonas’

    which reminds me that certain Reactions-interested folks also don’t feel heard with their feedback on XEP-0422

  373. ralphm

    jonas’: well, I don't think you have to. I'll try to dig things up.

  374. ralphm

    jonas’: I still owe responses to people reacting to my post. Anything specific you mean?

  375. jonas’

    all in https://mail.jabber.org/pipermail/standards/2019-September/036422.html

  376. ralphm


  377. Daniel

    there is also some underlying meta debate one could go into. so assuming I'm company X and I had some money to develop a group chat thing. I'll start looking into MIX but it doesn’t yet do 100% of what i want to achieve. so i provide feedback (like the roster thing); my feedback is being ignored. however my customers needs a group chat now and not in 5 years. so what do I do; do I fork MIX internally and use a "fixed MIX" which due to other constraints in MIX might still only be 99% of what i'm trying to do. or do I start from scratch and develop something new that fits my needs and my needs alone

  378. ralphm

    I think Kev did a good job of attempting to help out with his document. People not immediately responding doesn't mean they are being ignored. What do you think are reasonable expectations from people putting in volunteer efforts in our standards process?

  379. jonas’

    ralphm, I carefully chose my words to say "feel"

  380. Daniel

    seems like a lot of companies in the past have gone down the second road. P1, erlang solutions, redsolution etc

  381. jonas’

    I understand as well as anyone that volunteer standards process will have delays and everything

  382. jonas’

    ralphm, I’m not sure how we can fix this, but this is in a similar vein as what Daniel says

  383. ralphm

    Yep, this stuff is not easy. Some people want something now because they have business constraints, and some others have other priorities.

  384. ralphm

    I'm not opposed to companies doing their own private thing in the meanwhile.

  385. Daniel

    i don’t have an answer to that (otherwise i wouldn’t be asking) but is there anything we can do to help people who need solutions now and keep them in the community

  386. Daniel

    as opposed to what some companies are currently doing with "let's develop our own XEPs"

  387. ralphm

    Well, doing your own is a valid choice if you have internal pressure and don't require interop.

  388. Daniel

    you say "in the meantime" as if they were coming back once MIX is done. but i highly doubt that

  389. ralphm

    Even if we fix the protocol to most desire, it will still take a while to get to a place where we can migrate away from MUC.

  390. ralphm

    Daniel: I'm not sure how to help that other than my offer to move things forward.

  391. ralphm

    So in the short while that is fastening/references and MIX

  392. ralphm

    I also have opinions on calling, but I don't want to overcommit.

  393. ralphm

    And larma's comments are valid, so I'll try to get to that next week.

  394. Daniel

    > Daniel: receiving actual deltas if you were offline for some weeks Yes. Being able to retrieve deltas is probably a good argument for using custom syntax. I've re-read parts of 60 to see if we can build something like that on pubsub, rsm or mam but it doesn't seem to be the case. Not without either being super hacky or having massiv overheads

  395. Daniel

    I mean you don't actually want to get pubsub notifications from MAM. Not if your desire is to safe bandwidth

  396. Ge0rG

    Can a smart server aggregate those?

  397. Daniel

    Well I guess it would be fake mam in practice. Because you don't actually want to store the individual notification messages. It would just use a time stamp as a replacement for a version identifier and the your server would play out the notifications wrapped in pubsub wrapped in mam to you

  398. Daniel

    But that's not worth the overhead imho

  399. Daniel

    Compared to a custom syntax that would probably work more similar to how roster versioning works

  400. ralphm

    But you can also use RSM directly on pubsub?

  401. Daniel

    ralphm, that won’t give you the deletes

  402. Daniel

    also what is your after id

  403. Daniel

    if id (like in bookmarks 2) is the jid

  404. ralphm


  405. Ge0rG

    Can we generalize that into something to properly synchronize multi item PEP / PubSub?

  406. Zash

    pubsub since?

  407. Ge0rG

    I have no idea

  408. Zash


  409. ralphm

    That also doesn't send retracts.

  410. ralphm

    Problem is that PubSub is not designed to store retractions.

  411. Daniel

    and it's time based…

  412. ralphm

    So servers don't (currently) have that information.

  413. ralphm

    Daniel: yeah, that's not ideal either

  414. Ge0rG

    Can we design something new around MAM then?

  415. ralphm

    What MAM really means in pubsub has been debated before, but I don't think there's been a conclusion:

  416. ralphm

    a) you expose item storage as the archive

  417. Daniel

    a) you still don’t have a good ID for use in after b) syntax wise that would mean pubsub notification inside forward inside mam results (=massive overhead)

  418. ralphm

    b) you store the notifications as the archive

  419. Daniel

    and the ideas with the deltas is to avoid overhead

  420. ralphm

    a) clearly doesn't help for this.

  421. ralphm

    And by the way, you could have a mode where replays are *not* enveloped in forwarded.

  422. ralphm

    But item or retract directly in result.

  423. ralphm

    But I'm not sure.

  424. Daniel

    i mean what you really want is 'replay me events since x' and then you just get vanilla pubsub events; without any wrapper

  425. Daniel

    but at that point you are so far away from what pubsub can do that it is probably better to just use a custom syntax for that

  426. Ge0rG

    And x should be an ID, not a timestamp

  427. ralphm

    Daniel: maybe, but then how to differentiate between life events

  428. ralphm

    Ge0rG: yes

  429. ralphm

    Live events

  430. Daniel

    do you have to? roster versioning doesn’t use anything different for live events

  431. ralphm

    Race conditions and sync are not fun

  432. Daniel

    seems to work for roster versioning

  433. Ge0rG

    With roster you know when und sync is complete

  434. Daniel

    do i?

  435. Ge0rG

    With roster you know when the sync is complete

  436. ralphm

    And you don't get new events until you send presence, no?

  437. Ge0rG

    Isn't it all in one iq response?

  438. Daniel

    no you send ver to server. server sends you result (empty) and then a bunch of pushes

  439. Daniel

    and pushes are of the same syntax like normal pushes would be

  440. Ge0rG


  441. ralphm

    Ah, right

  442. Ge0rG

    But it does have merit to know when it's over, at least in a new design

  443. ralphm

    But we currently don't have notification ids

  444. ralphm

    For pubsub

  445. Daniel


  446. Daniel

    that what i mean with "by the time you have hacked all that in you are probably better of just doing something custom"

  447. ralphm

    Roster versioning has strictly increasing version numbers, right?

  448. Daniel

    the xep doesn’t say i think

  449. ralphm


  450. Ge0rG

    ralphm: strictly opaque

  451. Daniel

    i think there are cheap implementations where they just use a hash; and on reconnect if the hash doesn’t match anymore you just get the entire thing

  452. Ge0rG


  453. ralphm

    Right, and otherwise strictly increasing

  454. Ge0rG

    Still, I'm sure the general problem is worth solving

  455. ralphm


  456. Daniel

    i mean pubsub notifications having an event-id/version-id and then being able to say give me all events since version x would be nice

  457. ralphm

    Yes, that seems doable. Besides probably having to redesign storage in implementations. 🤣

  458. pep.

    > ralphm> It would be good to have a list of those and document how such issues are, or are not (yet?), addressed by MIX. Yes please, can we have a wiki page or something. If I ever take the time to read MIX and attempt giving feedback on it I don't want to repeat what everybody said already

  459. pep.

    > moparisthebest, [..] sorry for starting mix discussion it's fine, it seems some people (not you) love to jump on the opportunity to push the subject whenever there's a slight relation (or not even) :)

  460. pep.

    (that alienates me as much as what Matrix making IRC people's lives harder. Even if they deserve it somehow, let's be honest :))