XSF Discussion - 2018-06-20


  1. flow

    labdsf, caps still work with routing-ng (or why shouldn't they) and you are still able to send messages to the full jid

  2. jonasw

    labdsf, please close PRs which shouldn’t be merged

  3. jonasw

    you can make a new one later

  4. jonasw

    open PRs clutter the view

  5. labdsf

    ok, closed

  6. labdsf

    I can even reopen it

  7. jonasw

    yupp

  8. jonasw

    thank you d)

  9. labdsf

    flow: i mean, when I send messages to bare JID, I cannot know which capabilities the receiving client has

  10. jonasw

    labdsf, there are use-cases where you want to send to a single device, for example for IQ-based protocols and stuff like Jingle

  11. labdsf

    So ok, I can simply send message to all full JIDs that I think have required capabilities

  12. labdsf

    And attach no-copy hint

  13. jonasw

    except that they’re not archived

  14. labdsf

    Not archived by MAM is ok

  15. jonasw

    awful

  16. labdsf

    For ephemeral messages that is exactly what I want

  17. jonasw

    I am strictly against introducing another reason why messages don’t appear on all devices.

  18. jonasw

    OMEMO itself is bad enough already.

  19. labdsf

    If I send to all full JIDs, then it will appear on all online devices

  20. labdsf

    I think we need a better way to deliver messages to offline devices

  21. Kev

    We have it, it's MAM.

  22. labdsf

    It does not deliver messages to full JID

  23. jonasw

    labdsf, except that the device may be going offlnie exactly in that moment you send the message

  24. jonasw

    and come back online a secodn later

  25. jonasw

    without the user noticing

  26. jonasw

    (badly timed SM resumption can cause this behaviour)

  27. labdsf

    How about updating XEP 0160 to say that messages send with no-copy should be postponed and delivered to exactly that full jid, not some random with positive priority?

  28. labdsf

    sent*

  29. labdsf

    Or return service-unavailable at least

  30. jonasw

    labdsf, also, what if: 1. you check device support 2. you send a message to all full JIDs you want to 3. your network is slow and the message is delayed 4. one of your target devices goes offline 5. another device from that user comes online which does *not* have the feature you were looking for, but uses the same resource 6. your message was delivered to the wrong device

  31. labdsf

    jonasw: do not use the same resource

  32. jonasw

    you can’t force your peer to not do that

  33. jonasw

    using the resource for that kind of stuff is inherently racy because it’s an inherently temporary thing

  34. labdsf

    I think using unique resources is the best practice

  35. Kev

    And remember that clients don't get to choose their resources, the server does.

  36. labdsf

    Otherwise we cannot do anything but say that capabilities are totally unreliable

  37. jonasw

    labdsf, that’s what we’ve been doing for a while now :)

  38. jonasw

    for IQ based stuff it’s okay, and for things like jingle where there’s constant feedback, but using it for messaging is not a good idea

  39. Kev

    It's generally ok, but not foolproof.

  40. labdsf

    Except that capabilities XEP mentions XHTML-IM in the introduction, which is messaging use case

  41. jonasw

    labdsf, it’s also old

  42. labdsf

    I would rather slowly improve offline message delivery then declare that nothing is reliable in XMPP and we should just send everything to bare JID with store hint and body and save it in MAM forever

  43. jonasw

    labdsf, why not?

  44. labdsf

    Because it is impossible to implement any extension that relies on client support on top of that

  45. labdsf

    Lets say, replies, editing messages other than the last etc.

  46. labdsf

    OMEMO has to put encryption keys for *all* devices except the one it is intended for

  47. labdsf

    Adding body to OMEMO messages is a kludge also

  48. labdsf

    There are two problems with full JIDs: 1. No way to get list of offline resources 2. No reliable delivery to offline resources

  49. labdsf

    Both are fixable on server side

  50. Kev

    (1) That's what a presence subscription gives you (2) That's because there's no such thing as an offline resource.

  51. jonasw

    Kev, how does presence subscription give you a list of *offline* resources?

  52. Kev

    Sorry, I read *online* resources. It also gives you a full list of offline resources - it's an empty list because they don't exist :)

  53. jonasw

    heh

  54. Kev

    We do need to track offline clients for assorted reasons, but those are offline clients, not offline resources.

  55. labdsf

    Ok, "offline resource" means a resource that offline client will use when it reconnect

  56. MattJ

    There is no way to know what resource a client will use when it reconnects

  57. MattJ

    A resource is a session identifier for an online session only

  58. MattJ

    As much as it would be nice to imagine it is a device identifier, it is not - there is no standard for that, and it's not what most clients do

  59. jonasw

    (yet)

  60. MattJ

    I would like to change both of those things

  61. labdsf

    If it randomizes resource every time, it should be fixed IMO, or don't expect to get messages other than those sent to bare JID

  62. jonasw

    (I don’t expect messages except those sent to the bare JID.)

  63. jonasw

    (and I don’t think we should send normal IM to anything *but* the bare JID.)

  64. MattJ

    Agreed

  65. jonasw

    labdsf, the right way to handle stuff like XHTML-Im etc. is to provide a sensible fallback for clients which do not support it

  66. jonasw

    the protocol must be designed in such a way that legacy clients do the right thing

  67. labdsf

    MattJ: agreed to me or jonasw?

  68. MattJ

    jonasw, but I do agree that clients should not request a random resource for every session

  69. MattJ

    I also agree that a client cannot assume it will get *every* message sent to a full JID, but that one is far less clear

  70. labdsf

    Instead of MAM, server can list resources used by offline client for a week or so, with cached capabilities, and store offline messages sent to them for some time

  71. MattJ

    and sending to full JID is, like random client-requested resources, something we now want to discourage

  72. MattJ

    I already have most of an implementation of that, FWIW

  73. MattJ

    But I still don't think that means what you think it means

  74. MattJ

    Just because the server can track devices, doesn't necessarily mean they will always have the same resource

  75. MattJ

    It's potentially something you don't want to leak to third parties

  76. flow

    yet we do it as soon as a client replies with a message

  77. Kev

    We leak resources, we don't necessarily leak devices, because resources and devices aren't a 1:1 mapping over time.

  78. flow

    true, I still wonder if we shouldn't try to address that by adding a way to send messages from the bare JID. although I fear that this would probably break some (receiving) clients

  79. labdsf

    flow: or just introduce a virtual "proxy" resource maintained for you by the servet

  80. labdsf

    server*

  81. labdsf

    MattJ: what are the drawbacks of "leaking" a list of resources. Third party will know how many devices you have, anything else?

  82. MattJ

    Not that I can immediately think of

  83. MattJ

    It's not just how many, it's when you use certain devices also

  84. MattJ

    e.g. someone can track you between work and home if you have a device in each location

  85. MattJ

    People used to explicitly set resources like /Home and /Work (well, some clients defaulted to things like that also), but I think we know better than to share things like that by default in 2018

  86. flow

    labdsf, the resource string is the only thing between a remote entity and your device. if a remote does know it, they can send stanzas which will very likely reach your device (compared to "probably reach your device"). For example, someone knowing the resource of your mobile could write a script which sends "silent" messages every minute or so, in an attempt to drain your battery without you noticing it

  87. Seve/SouL

    MattJ, did you or someone else publish something regarding the newsletter? I've been not able to catch up lately and I wonder if I missed something

  88. Seve/SouL

    sorry, not the newsletter, where do I have my braind today...

  89. Seve/SouL

    The survey

  90. MattJ

    Seve/SouL, not yet

  91. jonasw

    flow, itym "reach that specific device" instead of "reach your device"

  92. jonasw

    because sending to the bare JID would normally cause the device to receive the message for sure at some point

  93. jonasw

    (in the ideal IM-NG world)

  94. Seve/SouL

    Ok ok, no rush MattJ, just worried about myself missing stuff :)

  95. flow

    jonasw, yes, if we did im-ng, but I deliberatly did not take it into condsideration because there are still many open questions

  96. jonasw

    flow, help to solve them :)

  97. flow

    jonasw, if it only where so easy ;)

  98. flow

    hmm "if it only would be that easy"

  99. labdsf

    flow, what if someone sents silent messages to bare JID?

  100. labdsf

    MattJ, where can I find your implementation of offline message delivery?

  101. MattJ

    labdsf: on my laptop

  102. labdsf

    so it is not in https://hg.prosody.im/trunk/ yet?

  103. MattJ

    No

  104. MattJ

    It will probably be in prosody-modules first

  105. MattJ

    Too much of this is new and experimental

  106. labdsf

    is there any description of it, maybe in ML?

  107. MattJ

    But I am writing it with the full intention of it replacing the current code we have

  108. MattJ

    No, and to be honest I have mostly avoided talking about it until recently

  109. MattJ

    There is not really anything to discuss

  110. MattJ

    For the initial implementation it will need no protocol changes, and no client changes

  111. labdsf

    but still, you will keep disconnected resources around, they will be seen in presence and it will be possible to send messages to them, right?

  112. MattJ

    No, as I said earlier a resource is not a device

  113. MattJ

    I am not trying to change that

  114. MattJ

    Prosody will track devices, but that does not necessarily mean sessions will have predictable resource strings

  115. labdsf

    if some client uses the same resource string on every connection, and no other client uses the same resource string, the message will be delivered to that device only?

  116. MattJ

    Send a message to a full JID, use no-copy and Prosody will deliver to that device only

  117. labdsf

    and if there is no no-copy, it is delivered to some other resource, as per https://xmpp.org/extensions/xep-0160.html ?

  118. MattJ

    As I wrote, I am not trying to change any protocols or behaviour beyond what should reasonably be expected

  119. MattJ

    If the device is online when you send the message, it is not an offline message

  120. MattJ

    It will go to that device

  121. MattJ

    If you send to a resource that is not online, core routing rules apply as always, it will be redirected

  122. MattJ

    If all devices are offline it's an offline message

  123. labdsf

    what happens if there is some device online, message has no no-copy and resource it is sent to 1) was used before 2) was never used before

  124. labdsf

    and what if there is no-copy and resource was never used before, message is lost?

  125. MattJ

    You're over-thinking it... the answer to all the questions is essentially: "exactly the same as happens now"

  126. MattJ

    Whether a resource was used before is completely irrelevant - resources are temporary, and they belong to a session

  127. MattJ

    Prosody is not tracking resources, it is not making resources persistent in any way

  128. MattJ

    Prosody is tracking devices. Devices != resources

  129. labdsf

    how does it track devices, besides the resource string they request on connection?

  130. MattJ

    XEP-0198 session token is another way, for example

  131. MattJ

    A client could try to resume a session, fail (because the session timed out), and then it goes on to bind a completely random resource

  132. MattJ

    Same device, different resource

  133. Zash

    Except not, because the device was killed by the OS and all that was lost

  134. labdsf

    the device can try to save SM id in some persistent database

  135. MattJ

    Zash, tangential to the devices != resources point :)

  136. Zash

    Possibly

  137. MattJ

    labdsf, I would eventually like bind2 or some alternative to be implemented, and allow the client to unambigously send a unique id

  138. MattJ

    But as I mentioned, my current scope involves no protocol changes

  139. MattJ

    After it's done, bind2/something will become more of a priority

  140. daniel

    saving the id only wont help you. you'd have to save the entire state

  141. MattJ

    Yeah

  142. daniel

    but on mobile the push target is another nice id for the device

  143. daniel

    clients will enable that in every session

  144. MattJ

    Ah, good point, thanks :)

  145. daniel

    but maybe too late depending on what you are trying to do

  146. MattJ

    No, it's easily extended to that

  147. MattJ

    Though I would like the push target to become part of bind as well

  148. MattJ

    Kinda "this is another way to reach me" metadata associated with the session

  149. labdsf

    MattJ, how is it different from mod_smacks_offline?

  150. labdsf

    it just places message for offline delivery, and you try to deliver it to the correct device instead?

  151. MattJ

    mod_smacks_offline is not ideal, it doesn't work with multiple resources online

  152. labdsf

    it just delivers to the resource with highest priority?

  153. MattJ

    If deviceA and deviceB are both online, if deviceB goes offline and does not resume, and there is an unacked message in the queue, there is a question about what to do

  154. MattJ

    It can't save it to the offline message store, because there is another device still online

  155. MattJ

    it could send it to that other device

  156. MattJ

    but that device possibly already received it

  157. MattJ

    via bare-JID forking or through carbons

  158. MattJ

    etc.

  159. MattJ

    so you end up with duplicated messages or bounced messages. In the case of a single resource going offline, with no other resources, it's simple to just save it to the offline store

  160. MattJ

    Next resource to log in receives it

  161. labdsf

    ok, current default is that it is just lost

  162. labdsf

    and what will you do instead with the new code?

  163. MattJ

    Current default (mod_smacks) is to send an error reply to the sender

  164. MattJ

    Which works fine, and is the safe option

  165. MattJ

    ...except when the sender has gone offline

  166. Zash

    or ignore all of that and Just Use MAM™

  167. MattJ

    labdsf, new code tracks devices and what stanzas/messages they need to receive

  168. MattJ

    which means the mod_smacks problem is solved, because the code can focus on delivery to a single device, and not worry about whether other resources are online/offline/etc.

  169. MattJ

    so at the time of forking/carbons/etc. that is when it gets decided what device receives what stanza

  170. MattJ

    Instead of 5-15min later "Oops, that failed to deliver, now where should we send it?"

  171. labdsf

    MattJ, so what will be different, you will try to identify deviceB even after timeout?