XSF Discussion - 2018-06-08

  1. daniel

    Link Mauve: to answer your question from yesterday. I'm not interested in bookmarks 2 per se but I'm interested in moving bookmarks to pep asap.

  2. daniel

    And I'm afraid bookmarks 2 is uncharted territory for a lot of reasons and harder to pull of then 48-on-pep

  3. daniel

    With only marginal improvements over 48-on-pep

  4. Andrew Nenakhov

    Do bookmarks have any other use besides storing user's MUCs?

  5. jonasw

    Andrew Nenakhov, I think some web folks put website bookmarks in there. Movim and SáT maybe

  6. Link Mauve

    jonasw, Movim and SàT put PubSub URIs AFAIK.

  7. Link Mauve

    jonasw, Movim and SàT put PubSub URIs there AFAIK.

  8. Link Mauve

    Not web ones, but xmpp: ones.

  9. Link Mauve

    daniel, that’s also my view of it.

  10. edhelas

    I moved my pubsub uris back to a "non standard" pep node for now

  11. edhelas

    because it was creating conflicts with the clients that are starting to put bookmarks in PEP (all the list in one item) clearing out the Pubsub URIs that Movim saved

  12. jonasw

    ah yes, that’s also a nice issue with everything-in-one-item

  13. jonasw

    it becomes tricky to deal with elements you don’t know, because normally in XMPP you can just drop them

  14. jonasw

    it becomes tricky to deal with elements you don’t know, because normally in XMPP (as a client) you can just drop them

  15. jonasw

    in this case you need to preserve them

  16. edhelas


  17. jonasw

    this would be much easier with one-item-per-bookmark, because you can just ignore items you don’t know and only update those you do know

  18. daniel

    jonasw: if it's just about that you can very easily do that with regular bookmarks as well

  19. daniel

    Conversations just ignores anything it doesn't know

  20. jonasw

    it /can/ be done, but it’s not straightforward

  21. Ge0rG

    I want to have bookmarklets, which can execute code in the context of the current MUC.

  22. jonasw

    yeah, I’m pretty sure that aioxmpp drops everything not in XEP-0048

  23. Wiktor

    Ge0rG: what would they exactly do? send some text?

  24. jonasw

    so if you’re using neither <url/> nor <conference/>, it gets lost

  25. Link Mauve

    Wiktor, for instance your password.

  26. Link Mauve

    I remember both Jappix and Empathy were victims of such RCE.

  27. Wiktor

    Oh, I didn't get the joke.

  28. Link Mauve


  29. Wiktor

    I was thinking about mIRC scripts ;)

  30. jonasw

    do I want to know, Link Mauve?

  31. jonasw

    (yes, I do)

  32. Link Mauve

    jonasw, https://usn.ubuntu.com/1250-1/

  33. jonasw

    ah, but that wasn’t about bookmarks

  34. Link Mauve

    Basically, /nick <img src=invalid onerror="alert('Hello world!')">

  35. Link Mauve

    Ah, no.

  36. jonasw

    good ol’ HTML nicknames

  37. Link Mauve

    I wouldn’t exclude bookmark names from being vulnerable to that kind of things, in web clients. ^^'

  38. Link Mauve

    … I should have such a bookmark in my test account!

  39. jonasw

    gotta love the web

  40. jonasw


  41. goffi

    https://github.com/git-federation/gitpub ==> and now they re-invent something on activityPub, it's a pitty to see coming a missed occasion for XMPP, once again.

  42. jonasw

    goffi, is it missed already though?

  43. goffi

    if gitlab, gogs and gittea are following, pretty much yes.

  44. jonasw

    although, I can’t blame web applications for not wanting to run another daemon

  45. jonasw

    we’d need a proper solution for stabily running a client or server or component in common web frameworks

  46. jonasw

    goffi, maybe you could contribute there and turn it around?

  47. goffi

    jonasw: I'm already working on XMPP based web framework, and I'm doing it on my freetime, I can't fight at all fronts.

  48. Ge0rG

    Dark corners of XMPP. Lection #23: prosody will put 0184 ACKs from some clients into MAM, but not from others. It depends on whether they are type=chat.

  49. Bunneh

    Ge0rG: Submit first draft of Mobile Considerations #23 https://github.com/xsf/xeps/pull/23

  50. Ge0rG

    Bunneh: Uhm, no.

  51. jonasw


  52. Ge0rG

    Same for Carbons. At least it's consistent.

  53. jonasw

    now that’s

  54. jonasw

    now that’s news

  55. daniel

    Ge0rG: are carbons ever type Chat though?

  56. Ge0rG

    daniel: dunno. Weren't you sending chat carbons?

  57. daniel

    Ge0rG: why would I send carbons of any type?

  58. Ge0rG

    if not(orig_type == "chat" or (orig_type == "normal" and stanza:get_child("body")) ) then you_shall_not_carbon_mam(); end

  59. Ge0rG

    Oh, wait.

  60. Ge0rG

    daniel: I mean: Weren't you sending chat *acks*?

  61. Ge0rG

    Same holds true for markers and any other bodyless messages.

  62. Holger

    Type chat without body should be carboned but not MAM-stored right?

  63. daniel

    Ja deliberately make them type chat yes

  64. Ge0rG

    Holger: who said that=

  65. Holger

    The XEPs?

  66. Ge0rG

    CSNs are defined as `chat`.

  67. Ge0rG

    Holger: now you make me sad.

  68. flow

    Ge0rG, have a cookie

  69. Ge0rG

    Holger: sad because of https://github.com/xsf/xeps/pull/434

  70. jonasw

    Ge0rG, do some work on https://xmpp.org/extensions/xep-0409.html

  71. Ge0rG

    jonasw: sure, when MIX is finished.

  72. jonasw

    since when do you care about MIX?

  73. Holger

    Ge0rG: Ah, routing 2.0 will fix everything.

  74. Ge0rG

    Holger: 0280 rules tl;dr are "a server may do anything, or not do it, or make demons fly out of your nose and be compliant"

  75. Ge0rG

    0313 only asks for <body> messages to be archived

  76. Ge0rG

    I'm not sure why anybody would want to archive any <chat> messages.

  77. jonasw

    I am reading that xep for the first time now, and I like how it handles the NG vs. non-NG interop

  78. Ge0rG

    jonasw: it's really intriguing, I'll need to think it through thoroughly

  79. jonasw

    Kev has done some good work there, I expected much less given the time frame in which it was done.

  80. Holger

    Ge0rG: > I'm not sure why anybody would want to archive any <chat> messages. You mean why people would *not* want to archive bodyless chat messages?

  81. Kev

    "Kev ... I expected much less" Story of my life.

  82. jonasw

    c’mon :)

  83. jonasw

    do you think we can get a <stanza-id/> in the mandatory reflection to all resources?

  84. jonasw

    (on the sender side)

  85. jonasw

    that would be truly amazing

  86. Kev

    Yes, but is that MAM or routing?

  87. Ge0rG

    Holger: No, I mean why people *would* want to archive bodyless chat messages.

  88. Holger

    Read markers for example?

  89. Ge0rG

    Holger: also CSNs

  90. jonasw

    Kev, good question; I would like to have it mandatory for every service implementing MAM though. this avoids us having to namespace-bump MAM.

  91. jonasw

    i.e. make it mandatory in IM-NG for a service which also offers any XEP-0313 namespace to put <stanza-id/> in the reflections

  92. Ge0rG

    Holger: not sure if you heard of it, but MAM storage of CSN made my MAM DB explode and yax.im melt down, just recently

  93. Ge0rG

    Kev: mandatory stanza IDs for "important" messages are good enough for me, so MAM > routing

  94. Holger

    Ge0rG: I haven't, but not storing read markers sucks UX-wise and I'm sure there are more examples.

  95. Ge0rG

    Holger: so we have two examples now of things that are of the same type but needs to be handled differently. What now?

  96. Kev

    Holger: I have thoughts on that too, but I've not written them up yet.

  97. Ge0rG

    ACKs are of type=undefined, so even more meh.

  98. Kev

    But I think that you want your archive to be collating such things rather than storing verbatim.

  99. Ge0rG

    Kev: that sounds like a database synchronization problem. Send deltas, update internal state.

  100. Holger

    Yes such optimizations would be nice. I'm just saying that throwing this stuff away is not a good solution.

  101. jonasw

    oh, it’s in there already: > When reflecting an IM-NG client's outbound bare-JID messages, the server SHOULD reflect the archived version (i.e. after any transforms have taken place).

  102. jonasw

    IIRC the archived version contains the <stanza-id/>

  103. Ge0rG

    jonasw: yes, it should

  104. jonasw

    it should or it SHOULD?

  105. Ge0rG

    jonasw: don't ask me these questions please. It's friday and I'm having tremendous "fun" at work already.

  106. Ge0rG

    I only started this because it bothered me that I didn't see ACKs of messages on my other client.

  107. jonasw

    here, have a cookie

  108. jonasw


  109. Ge0rG

    And there is no easy solution.

  110. jonasw

    and you should please appreciate that I pasted you a cookie because you know how much that makes my poezio behave weird!

  111. Ge0rG

    jonasw: sorry, but cookies trigger me since the GDPR faux-compliance of ignorant americans made cookie dialogs look like good UX

  112. Ge0rG

    really sorry.

  113. jonasw


  114. jonasw

    some cool water melon maybe?

  115. Ge0rG

    jonasw: since running emoji_ascii the situation has considerably improved for me. Except for the jabber❌namespace.

  116. jonasw

    Ge0rG, you’re not alone, github does it too: https://github.com/xsf/xeps/pull/664

  117. Ge0rG

    it's a rainy 18°C here. But I had to change my munin cpu temp warning threshold from 70° to 80°

  118. jonasw

    right, I forgot that only Dresden seems to be shielded from the bad^Wgood and plant-saving weather

  119. Ge0rG


  120. jonasw

  121. jonasw

    then that maybe^?

  122. Ge0rG

    that made my day.

  123. jonasw

    Ge0rG, finally found something good for you \o/

  124. Ge0rG

    next time somebody complains, I'll point at that issue

  125. jonasw

    I need to re-write the commit message, because I’m going to be complaining a lot ;-)

  126. Ge0rG

    I've had a 1.5h conference call today, where the application owner and me tried to explain to the indian chief developer that exposing an SQL-like API over the internet, with some regex filters strapped on top for access control, just doesn't cut it.

  127. Ge0rG

    The final words of the app owner were "I hope we are aligned now"

  128. Ge0rG

    not sure if it was a command or a threat.

  129. Ge0rG

    But back to the topic. Even with IM-NG, we'll still have to define rules for handling of legacy messages

  130. Ge0rG

    So back to https://github.com/xsf/xeps/pull/434

  131. Ge0rG

    Would it make sense to make 0184 and others use type=chat?

  132. Ge0rG

    or type=<type of what you respond to>?

  133. Ge0rG

    MUC ACKs should go to the MUC, so that we get O(n²), right?

  134. jonasw

    Ge0rG, the rules are defined in there: > In order for interoperability with other entities (contacts, remote servers etc.) that don't support IM-NG, old-style full-JID messages also need to be handled. When a server receives a message with type other than than 'groupchat' or 'headline' that does not contain an <im-ng xmlns='urn:xmpp:im-ng:0'> element it is to be routed by the above rules as if they were sent to the bare JID

  135. jonasw

    plus: > Any message that is routed to all IM-NG clients is stored in the MAM archive, where available, and any message that would not be routed to all IM-NG clients is not stored in the MAM archive

  136. jonasw

    so if to bare JID => archived and copied. if to full JID and not groupchat and not headline => archived and copied. if to full JID and (groupchat or headline) => routed to full JID

  137. Ge0rG

    What about CSNs?

  138. jonasw

    are they of type headline or groupchat?

  139. Ge0rG


  140. jonasw

    archived and copied.

  141. Kev

    At the Summit we discussed that they should really be presence rather than messages, FWIW.

  142. jonasw

    I’ll make you a flow chart, Ge0rG.

  143. Ge0rG

    jonasw: no thanks

  144. jonasw


  145. jonasw

    Kev, that, too

  146. Ge0rG

    jonasw: my question wasn't "what happens to CSNs under these rules". I was able to figur that out on my own, I think. My intention was "what *should* happen to them, as the rules are obviously inadequate"

  147. jonasw

    Ge0rG, ah, put them into presence

  148. jonasw

    that seems like a good solution for me

  149. Ge0rG

    so the IM-NG server will translate directed presence with a CSN into a message on the border, and back?

  150. jonasw

    need to think about implications of sending directed presence all the time, but on the first order this seems like a good thingf

  151. jonasw

    Ge0rG, uh, interesting idea :)

  152. Ge0rG

    directed presence will get cached on the server, right?

  153. jonasw

    (but I was actually talking generally. clients should put them into presence)

  154. jonasw

    directed presence is tricky

  155. Ge0rG

    Yeah, so while you are constructing the IM-NG New World Order, I'm not receiving Carbons of my ACKs!

  156. Kev

    The other idea I have floating around is having <transient/>, which is rougly <no-store/>, and carefully speccing that a CSN is <transient/>, and that anything that is <transient/> and "shouldn't be" shouldn't be processed by a client.

  157. Kev

    So that you can't e.g. inject <transient/> in a chat stanza in order to maliciously evade archiving.

  158. jonasw

    that sounds quite fragile

  159. Ge0rG

    > maliciously evade archiving. That is very evil.

  160. Kev

    Because <no-store/> that your server listens to and your client doesn't care about is obviously bad.

  161. jonasw

    but it might be worthwhile to have that

  162. jonasw

    I mean provision this in IM-NG so that servers support it right away.

  163. Ge0rG

    Ephemeral Messages are doomed now.

  164. Kev

    jonasw: Which that. My floaty idea, or the thing I say is bad?

  165. jonasw

    and so that clients drop it right away

  166. jonasw

    your floaty idea

  167. Kev

    Third idea is that we requisition headline.

  168. jonasw

    lemme rephrase: The <transient/> thing sounds a bit fragile, but it might be worthwhile to put that into IM-NG right away because it’s likely that we’ll need such a thing down the road.

  169. Kev

    But now we're getting into even dodgier territory, potentially.

  170. Ge0rG

    What about we define a list of rules which messages are to be handled as transient, like in https://xmpp.org/extensions/xep-0280.html#which-messages

  171. jonasw

    I’m not 100% sure about CSN-in-<presence/> either, because that has interesting implications regarding directed presence and things which do things with directed presence (MUC, Invisible Command, …)

  172. Kev

    Ge0rG: Because it's important that we don't have a situation where a server's MAM implementation needs complete knowledge of all standard and non-standard extensions.

  173. jonasw

    I’m not 100% sure about CSN-in-<presence/> either, because that potentially has interesting implications regarding directed presence and things which do things with directed presence (MUC, Invisible Command, …)

  174. jonasw


  175. Kev

    But if we go with 'floaty idea', you have more or less that, without the server problem.

  176. jonasw

    then only the client needs to know about it, and that is probably fine because it needs to know about the message payload anyways to do something sane with it

  177. Kev

    The server listens to the <transient/> flag, and the client is responsible for checking that a message that should be archived wasn't received without it.

  178. Kev

    jonasw: Right.

  179. Ge0rG

    Kev: so that client-side message processing has two stages. Stage 1 is for all messages, and may not add anything to the database, and stage 2 is for non-transient messages only and may add new items?

  180. jonasw


  181. Ge0rG

    And there is a huge fat `// WARNING! NO PERSISTENCE ABOVE THIS COMMENT` line in between?

  182. Kev

    Ge0rG: There's a few ways one could implement it, but something like that would work.

  183. jonasw

    I’d probably go more with: a general stage which works like the servers MAM (i.e. ignore anything with <transient/> on it) and down the road in the pipeline some component might add something to the database in response to a message which was <transient/> if and only if it makes sense to do so

  184. jonasw

    but in general you wouldn’t persist <transient/> stuff anyways

  185. jonasw

    that’s the point of transient. I mean you could have been offline at the time and wouldn’t have gotten the message, so storing it seems not needed in any case

  186. Kev

    I'd probably go with an initial screen process that rejects any stanza that doesn't verify the right transientness for the present payloads, but yes, any of these work.

  187. Kev

    (Lot a literal forked process, obviously)

  188. Kev

    (Not a literal forked process, obviously)

  189. jonasw

    Kev, I’m going to dump you a bit of commentary on XEP-0409 on the list in a minute btw

  190. Kev

    Ok. Be kind :)

  191. jonasw

    will be

  192. jonasw

    I’ll add a few points I already raised in this discussion so that we have them on-record somewhere

  193. Ge0rG

    I hope nobody ever asks *me* to be kind on-list :D

  194. Ge0rG

    jonasw: excellent!

  195. jonasw

    Ge0rG, why would anyone? you’re the personified kindness.

  196. jonasw

    Ge0rG, but not all points, I wasn’t paying attention consistently

  197. Ge0rG

    jonasw: you just grilled my sarcasm detector.

  198. jonasw

    Ge0rG, double-strike

  199. jonasw