jdev - 2020-07-26


  1. Жокир

    Just wondering, is there any particular reason why xmpp isn't length prefixed and rather the whole stream is one never ending document? I mean if it were length prefixed than inband binary data could be transmitted without b64

  2. Жокир

    Just wondering, is there any particular reason why xmpp isn't length prefixed and rather the whole stream is one never ending document? I mean if it were length prefixed than inband binary data could be transmitted without b64, among other things

  3. lovetox

    sounds complicated

  4. lovetox

    sounds more complex to implement

  5. lovetox

    right now i just feed the stream to a parser

  6. lovetox

    i try to think about when i need inband data transfered

  7. lovetox

    and only avatars come to mind

  8. Жокир

    > sounds more complex to implement Quite the contrary. If stanzas were length prefixed standalone documents you could just feed them to a regular dom building parser of which there are plenty for every language.

  9. lovetox

    and what is the difference to the parser i push it now?

  10. lovetox

    do you mean the xml would just end in the middle, like <data>binary data<data>?

  11. lovetox

    do you mean the xml would just end in the middle, like <data>binary data</data>?

  12. lovetox

    if yes, this would of course be nice, but i maintain a decade old client, and as i said the only usecase come to mind are avatars

  13. lovetox

    for everything else, you would want to have a out of band transfer that you prefereably can do async

  14. lovetox

    and i was not there back then, but i guess avatars was not on the mind of xmpp creators, and even if it would be debateable if you design a whole protocol around transfering avatars which is like 1% of xmpp usage or less

  15. MattJ

    Data framing has some nice properties, but would essentially need everyone to write custom parsers

  16. MattJ

    XMPP over websocket is framed, FWIW, and supported by many servers

  17. flow

    Жокир, I think the simple answer is: because XMPP re-uses XML as building block

  18. flow

    also it is *not* a never ending document, there is an end (at least if you use a transport that closes the connection by sending a closing </stream> tag)

  19. Guus

    There are limits to max stanza lengths. I don't think that the spec defines a limit, only that one is allowed to exist.

  20. Guus

    (although I can't immediately find it. Am I wrong here?)

  21. Zash

    > A deployed server's maximum stanza size MUST NOT be smaller than 10000 bytes

  22. Zash

    https://xmpp.org/rfcs/rfc6120.html#rfc.section.13.12

  23. lovetox

    the whole domain name stuff is insane

  24. lovetox

    luckily python has a idna lib for that

  25. lovetox

    and i discovered just now luckily it has a UTS46 mapping mode

  26. Zash

    IDNA 2003/2008 issue?

  27. lovetox

    2008

  28. lovetox

    but imagine there would be no python lib for that

  29. lovetox

    and everybody who implements xmpp, needs to implement the whole idna spec with it

  30. lovetox

    probably would stop right there and do something different

  31. lovetox

    yeah easy to miss, the IDNA2003 spec required a mapping phase in the first stage

  32. lovetox

    for example XMPP.net -> xmpp.net

  33. lovetox

    IDNA2008 does not require or describe that phase anymore

  34. lovetox

    so if you do purley idna2008, then you end up with different JIDs here

  35. lovetox

    it seems they excluded that mapping spec to some other standard because its no strictly required anymore

  36. Zash

    IIRC our IDNA 2003 code didn't do any lowercasing.

  37. lovetox

    just discovered this now

  38. Zash

    But the current stuff seems to do it

  39. Zash

    "current" = future, actually

  40. lovetox

    apperently it does

  41. lovetox

    http://unicode.org/reports/tr46/#IDNA2003-Section

  42. lovetox

    or it did

  43. Zash

    $ idn <<< XMPP.net XMPP.net $ idn2 <<< XMPP.net xmpp.net

  44. Zash

    🤷

  45. lovetox

    then this is weird, because the standard says, IDNA2003 does require a mapping phase

  46. lovetox

    IDNA2008 does not

  47. lovetox

    but maybe this idn2 application just uses sane defaults

  48. lovetox

    and you can disable the mapping

  49. lovetox

    just to be clear, 2008 does not require it as part of the IDNA standard

  50. Zash

    Keep in mind that with JIDs you'd do some stringprep or precis operation on the host part first

  51. lovetox

    but of course application can do a mapping if wished

  52. Zash

    IDNA is only for when you do DNS lookups, and DNS is case-insensitive

  53. lovetox

    Zash, you dont do any precis operation on the domain part

  54. Zash

    Not?

  55. Zash

    This topic. Are you trying to make everyone cry themselves to sleep tonight?

  56. lovetox

    no Zash, it refers to the IDNA rfc

  57. lovetox

    so we dont need IDNA only for dns lookups

  58. lovetox

    we need it to know that a@XMPP.net = a@xmpp.net

  59. Zash

    So IDNA becomes the nameprep step then.

  60. lovetox

    yes

  61. lovetox

    to be precis IDNA2008 does no mapping, and requires no mapping

  62. lovetox

    you need a lib that implements IDNA2008, and the UTS46 mapping or some equivalent

  63. lovetox

    which is not specified in the idna spec

  64. lovetox

    hence, my first statement its all insane, and if there would be no lib that would just do that for me

  65. lovetox

    i probably would stop right there to develop my client :D

  66. Zash

    I know the feeling.

  67. Zash

    Hence the part about crying.

  68. Zash

    We don't have any PRECIS lib atm. We use libidn for stringprep and IDNA (2003). But libidn is deprecated and replaced by libidn2, which does *only* IDNA 2008. No stringprep or precis. Thus not an option.

  69. lovetox

    and just out of interest i looked up the docs of your application

  70. lovetox

    and indeed it uses the sane default

  71. lovetox

    but you can disable it

  72. lovetox

    --no-tr46 Disable TR46 processing

  73. Zash

    $ idn2 --no-tr46 <<< XMPP.net xmpp.net

  74. lovetox

    hm

  75. lovetox

    lovetox@lovetox:~$ idn2 --no-tr46 XMPP.net XMPP.net

  76. lovetox

    dont know what your "<<<" does

  77. lovetox

    but nothing good :)

  78. Zash

    bash magic, dumps that string to stdin

  79. Zash

    heredoc without eof marker

  80. Zash

    pretty handy imo

  81. lovetox

    yeah but not if it ignores your options

  82. Zash

    $ idn2 --no-tr46 XMPP.net xmpp.net

  83. Zash

    Weird

  84. lovetox

    thats indeed weird

  85. lovetox

    i use version 2.2.0

  86. Zash

    https://gitlab.com/libidn/libidn2/-/blob/master/NEWS#L25

  87. lovetox

    great :)