XSF Discussion - 2023-11-28

  1. debacle

    My xmas wishlist: Bring back "XEP-0071: XHTML-IM" and "XEP-0138: Stream Compression". Maybe both with updated Security Considerations. Is there any process for reviving the dead? Some kind of Lazarus procedure?

  2. Daniel

    Actually last week's discussion has convinced me that compression can't be done in a manner that is generally secure

  3. jonas’

    Daniel, well, if you flush after each stanza it should be mostly fine, shouldn't it?

  4. jonas’

    you don't compress across security boundaries in that case

  5. Daniel

    unless you can inject something into the stanza that you want to retrieve information from. to name one example; the attacker sends a room invite to a room with a password. that's somehow accepted automatically and said password ends up in bookmarks. now you can modify that password (update the bookmarks again and again) to guess other passwords in there

  6. Daniel

    yes it's far fetched and multiple bookmarks items are only in the same stanza on first connect; but you still crossed a security boundry

  7. jonas’

    Daniel, oof

  8. jonas’

    yeah, ok, let's not do that

  9. MattJ

    So what problem are we trying to solve with compression? Why do you want it, debacle? :)

  10. Daniel

    fwiw i’m more than happy to ignore that and still use compression on project where compression really matters. but i'd rather not write up the security considerations for general use

  11. debacle

    MattJ Daniel In my case it is for an IoT application. Devices are "in the field" and traffic *is* a problem. In my tests with zlib, I get a compression rate of 15.6.

  12. MattJ

    It seems to me that most IoT devices for which traffic matters are typically not powerful enough to really do zlib anyway

  13. debacle

    It's not about the IM use case of XMPP. There are no passwords or other special secrets in the traffic. Login happens via client certs anyway, not passwords.

  14. Guus

    You don't need a XEP to be revived for an implementation.

  15. debacle

    MattJ This assumption is very wrong for the products of my company :-)

  16. MattJ

    It's not only about passwords. It sounds like you have a closed environment where you're willing to trust everyone on the network, just use compression then.

  17. debacle

    Guus, sure, we will probably implement the XEP no matter if it is still a standard.

  18. moparisthebest

    I think not allowing compression state to cross stanza boundaries is good enough (with these security considerations spelled out of course) and using zstd with a custom dictionary should be investigated

  19. MattJ

    I think a custom encoding or fixed dictionary is the way forward

  20. MattJ

    for bandwidth optimization

  21. moparisthebest

    MattJ: hasn't been true in a long time, $2 (at quantities of like 4) chips can do WiFi and Bluetooth and have 2 cores powerful enough to run linux

  22. moparisthebest

    Esp32 is the specific one I'm calling out but by now there are many more

  23. MattJ

    ESP32 can run Linux and do zlib?

  24. MattJ

    I spent some time working on an XMPP library for ESP32, I will be surprised if you say "yes" :)

  25. moparisthebest


  26. MattJ

    Right, but there's "Linux" and there's "Linux"

  27. moparisthebest

    But it can do TLS and a web search finds zlib implementations for it

  28. MattJ

    The TLS implementation leaves a lot to be desired, but offloads some stuff to the hardware which is why it's tolerable for most applications. I don't know if there is similar support for zlib, but if there is it's going to be in a pretty similar state and would almost certainly require external RAM.

  29. MattJ

    I stand by what I said, such devices would perform better without this "optimization"

  30. MattJ

    I'm a fan of not prematurely optimizing, and measuring before optimizing. I'm very curious how traffic is a real problem in real deployments.

  31. Daniel

    MattJ: define real deployments

  32. MattJ

    Heh, that article you linked has Linux running atop a RISC-V emulator on the chip, fun, but almost certainly painfully slow :)

  33. MattJ

    Daniel, anything that isn't just people in a chatroom saying "we should support compression", but where there is a deployment struggling with the amount of traffic it is processing

  34. Daniel

    I agree that 'the network of publicly federating xmpp servers' also know as jabber should probably not turn on compression

  35. MattJ

    With today's tech capable of streaming videos to mobile battery-powered devices for multiple hours continously, I find it very hard to imagine how it can be a struggle to shuffle some text around

  36. MattJ

    Compression is not "free", either, it just trades bandwidth for CPU/RAM usage (and indirectly, battery life, for battery-powered devices)

  37. Guus

    MattJ: I have customers with high latency bandwidth issues. Things that fly and float far away, things like that.

  38. MattJ

    Compression doesn't solve latency

  39. Daniel

    > Compression is not "free", either, it just trades bandwidth for CPU/RAM usage (and indirectly, battery life, for battery-powered devices) Exactly and sometimes CPU and power is cheaper than bandwidth

  40. Guus

    MattJ: that's very real world, but more for reducing roundtrips than compression.

  41. Guus

    Although they are adjecently related

  42. MattJ


  43. Guus

    Isn't the 'compression is not free ' argument partly addressed by EXIs promise to allow devices work on the raw binary data? Not so much compression than an efficient mapping of the protocol/data?

  44. MattJ

    EXI has various problems of its own, so it is indeed mostly a "promise" but not a reality

  45. debacle

    Our customers are almost everywhere and at some places bandwidth is either scarce or expensive or both. E.g. satellite connections. That's why we want compression on that line. But we don't want any proprietary way of compression, because then we could not work with the standard tools. Ejabberd, fortunatley, supports 0138, so we only need to implement it on the device side. We think about sponsoring it for libstrophe. I hope, my $BOSS agrees :-)

  46. debacle

    In our application, the system is openly federated. Any JID can retrieve measurement data, if it has permission for the PEP node. But the server is closed, i.e. only our devices have accounts there and they are relatively safe thanks to client certs. I assume, that for such a use case stream compression is not a security problem, even without resetting after every stanza.

  47. debacle

    Daniel: > I agree that 'the network of publicly federating xmpp servers' also know as jabber should probably not turn on compression +1 (Gajim does not yet support XEP-0444: Message Reactions)

  48. emus

    ## Last XMPP Newsletter 2023 📬📧 ## Dear all, cal0pteryx was so nice to open the PR for the last Newsletter release this year! So if you have news please add them to the PR or the pad. We are happy for everyone to support again and help to prepare and review. Newsletter draft on Github: https://github.com/xsf/xmpp.org/milestone/3 (Navigate to the latest issue, then to 'Files changes', then 'Edit File') Online pad: https://pad.nixnet.services/oHnY_ZvLT8SoFyCqIC2ung Please remind that it requires also your activity to reach out and add news. Looking forward 👋

  49. jonas’

    since the discussion here the other day, the idea of a compact XML binary encoding geared toward XMPP crossed my mind

  50. meson

    jonas’, and then we call it FunXMPP? :P

  51. jonas’


  52. Zash

    jonas’, did you say mod_cbor?

  53. singpolyma

    with cbor tags specified for specific xmpp things! ;)

  54. jonas’

    Zash, decidedly not

  55. singpolyma

    Has it ever been discussed to add caps hash to disco#items ?

  56. Zash

    singpolyma, not impossible that it has.

  57. MattJ

    It has

  58. MattJ

    I very much want to do it

  59. singpolyma

    Yeah, it seems like it would make a lot of stuff more performant

  60. Zash

    Is there room in caps2?

  61. Zash

    I'd also like a way to pass a recursive caps thing to clients in one go

  62. lovetox

    how would that make it more performant?

  63. Zash

    How would what do what?

  64. lovetox

    > Yeah, it seems like it would make a lot of stuff more performant

  65. edhelas

    Like for example for Pubsub items to basically know if some node items changed or not ?

  66. lovetox

    a lot of stuff ..

  67. lovetox

    what stuff would make it more performant

  68. Zash

    You would be able to cache e.g. MUC component features and know on connect if it changed without doing disco#info again

  69. lovetox

    ah so disco#items returns the items but also a caps hash of the items

  70. lovetox

    this would indeed save a few disco#info queries

  71. Zash

    Currently there's a way to pass caps for your own host in <stream:features/>, but not for your account or any subdomains.

  72. Zash

    https://xmpp.org/extensions/xep-0390.html#usecases-stream-feature Huh, can't find the one for xep-30?