jdev - 2021-02-28

  1. Martin

    What would you consider a reasonable timeout for connecting to an xmpp server?

  2. lovetox

    in total?

  3. lovetox

    or to one connection point before you try the next

  4. lovetox

    actually i dont think i have a timeout

  5. lovetox

    not sure why i should, if i cant connect it does not change anything if i abort after a timeout, its not like i can suddenly connect after trying again

  6. lovetox

    but i guess there is a timeout usually from your network libs

  7. lovetox

    so i take whatever they do and when the return with "could not reach remote" or whatever

  8. Sam Whited

    lovetox: presumably you want to go back to the start screen and say "Something's wrong, please wait and try again later" or something eventually right, and not just say "Connecting" forever even though it will never happen?

  9. Sam Whited

    eg. if something was broken and the server stops responding, or the TCP connection hangs, or any number of other things happen.

  10. lovetox

    hm its not like it will be stuck there forever though

  11. Sam Whited

    Why not?

  12. Sam Whited

    The point of a timeout isn't in case you hit an actual error, it's in case it gets stuck forever :)

  13. lovetox

    because usually if you open a socket to some server, the network lib tells you after some time that something does not work

  14. lovetox

    so yeah maybe these libs have timeouts implemented already

  15. Martin

    lovetox: The timeout before trying to connect to the next port (SRV records).

  16. Sam Whited

    That's fine if it's at the TCP layer, not if you're connecting to something that's accidentally slow loris-ing you because it's overprovisioned and is returning 1 byte per minute or something very slowly by mistake

  17. lovetox

    dont know how they determine that a host is not reachable

  18. lovetox

    sorry Martin i dont do this right now

  19. Sam Whited

    ah okay, those are two different things though. Between SRV records I'm not sure

  20. Sam Whited

    I don't actually know if people treat different ports on the same host as different resources for rate limiting purposes, I would assume not. So maybe exponential fallback starting with 10ms or something would be a good baseline?

  21. Martin

    I just set it to 10 seconds while playing right now. Prior to setting it I blocked outgoing traffic on 5222 and it didn't continue for a minute, so whatever the underlying network libs use for a timeout, it seems to be terribly high.

  22. Martin

    But I think I'd rather set it to 1s or so.

  23. Sam Whited

    Your network library waits 10 seconds before trying the next thing in a set of SRV records? That does seem like a long time.

  24. Martin

    No, I set it to 10 seconds to speed things up. According to mattn/go-xmpp there is no timeout when you don't set it. But I don't know if there will apply a timeout from the underlying network libs.

  25. Sam Whited

    Oh, you mean it will jus tinstantly reconnect to the next one by default?

  26. Martin

    So it seems I would be stuck forever if I can't connect to the first record and don't set a timeout.

  27. Sam Whited

    oops, too soon, not instantly, stuck forever. Gotcha.

  28. Martin

    But just trying all xmpp-client records, then all xmpps-client records is surprisingly easy. I'm not yet sure whether I should try xmpp on 5222 and xmpps on 5223+443 if no SRV records are provided. How is the common sense?

  29. Sam Whited

    That's technically what people do as a fallback if no srv records are provided, but there was some discussion recently that it was a mistake. Right now I do that in my library too, but I keep going back and forth on whether it's worth it.

  30. Sam Whited

    See https://tools.ietf.org/html/rfc6120#section-3.2.2

  31. Martin

    You can still specify it manually with `--jserver=example.com:5222` so maybe I should not bother and just use SRV records and don't care about fallbacks.

  32. Sam Whited

    That sounds sane to me, FWIW

  33. Sam Whited

    Or if there are no SRV records it would make sense to default to using that

  34. Martin

    I'll think about it. Thanks for your input. :)

  35. Sam Whited

    Sure thing; let us know what you decide, I keep changing my mind on how I should do this too so I'll be curious what you do.

  36. Martin

    I tend to use 5222 if there are no srv records as I think servers either use the standard port or provide srv records.

  37. defanor

    I think it's nice to let the underlying system to decide on TCP timeouts, since those can be tweaked per-system and system-wide, depending on its connection or requirements. But as reference values, RFC 1122 suggests at least 100 seconds, and Linux's tcp(7) mentions the default of about 13 to 30 minutes.

  38. Sam Whited

    I thought he was asking how long to wait between connection attempts to different SRV records, not for TCP timeouts, but maybe I'm still confused.

  39. Martin

    I really don't want to wait for minutes while my tool tries to connect. Rather try the next srv record.

  40. defanor

    Oh, after a failure, and before trying the next one? Why to wait between those at all?

  41. Sam Whited

    Maybe you shouldn't, I'm not really sure. But if all the SRV records are just different ways to connect to the same service you may want to wait to avoid triggering rate limiting (this is why the old xmpp.net scanner waited, IIRC)

  42. Martin

    When I locally blocked outgoing traffic on 5222 and had no timeout set it didn't go on to the next srv record for one minute. That's why I added a timeout of 1s now as I want to try the next port quickly if I can't connect there.

  43. defanor

    One solution may be to extend "happy eyeballs" to cross SRV records. To neither give up on connections too quickly, nor wait too much if others are available.

  44. Sam Whited

    This doesn't include timeouts, but I think this is up to date if you're curious how we're connecting: https://mellium.im/issue/2#issuecomment-735241044

  45. lovetox

    defanor, yeah but happy eyeballs does not do this

  46. lovetox

    and i think its even for a xmpp lib kind of advanced to implement happy eyeballs themself, not to speak from extending it to multipls srvs

  47. lovetox

    im in luck that GLib supports happy eyeballs with just passing a srv record

  48. lovetox

    but they fixed multiple bugs over the years on that code

  49. lovetox

    i also think there is not much use for this cross srv thing in the wilid

  50. lovetox

    i also think there is not much use for this cross srv thing in the wild

  51. Martin

    > This doesn't include timeouts, but I think this is up to date if you're curious how we're connecting: https://mellium.im/issue/2#issuecomment-735241044 You look up xmpp and xmpps records in parallell?

  52. Sam Whited

    I look up both records in parallel, then try xmpps first if they existed

  53. Martin

    I look up xmpp-client records, try them and if I can't connect look up xmpps-client records and connect. Should xmpps-client have higher priority?

  54. defanor

    The XEP suggests to mix them, so that priorities would be defined by a hostmaster. I was lazy to do that, and just prioritising xmpps too for now (after querying them in parallel as well).

  55. Sam Whited

    The XEP is wrong IMO. It's breaking the SRV RFC and just makes no sense with how SRV records work.

  56. Sam Whited

    I personally try xmpps-client first, but I'm not sure that it matters. In theory if xmpp-client is first you could negotiate without STARTTLS when TLS is available (if you support plain connections, which you shouldn't or should at least hide it behind some kind of flag or option)

  57. Zash

    There's prior art in the SRV for email RFC IIRC

  58. Zash

    Whether it's a good idea not is a separate issue

  59. Sam Whited

    TIL: email discovery with SRV.

  60. Sam Whited

    huh, fair enough, it recommends mixing imap, imaps, pop3, and pop3s.

  61. Sam Whited

    This RFC makes it *much* clearer how priorities and weights are supposed to be used, it's somewhat convincing. Although it still seems like it's just unnecessary difficulty because libraries that query and connect aren't going to support this most likely.

  62. moparisthebest

    How can xep368 be more clear in that aspect?

  63. moparisthebest

    Crap libraries exist is not a reason to change a spec I think

  64. Sam Whited

    This isn't crap libraries, this is the standard library of every programming langauge I've ever used, I think.

  65. moparisthebest

    It's ok, standard libraries can be crap too

  66. Sam Whited

    Actually, maybe not the standard library, but everything that does SRV.

  67. Zash

    Sam Whited, hold on, you're saying there's languages with SRV support? Other than Go?

  68. Sam Whited

    Zash: that's why I said "maybe SRV libraries", I realized I'm not sure outside of Go if I had to download something else or not :)

  69. Sam Whited

    But still, LookupSRV(service, proto, name) seems sane and simple. Having to provide multiple isn't something I've ever seen done because the SRV RFCs don't work that way.

  70. Zash

    The Go network lib is the only I've ever seen, with actual SRV aware connection support. Not counting pure DNS libraries.

  71. moparisthebest

    This is the only way to do it with SRV, but SRV2 fixes this by letting you specify *how* to connect to each endpoint

  72. Sam Whited

    moparisthebest: or don't do it and just say "Prefer this one if supported or then look up this other one"

  73. Sam Whited

    Or leave it entirely up to the clients which one to do first.

  74. moparisthebest

    "never do new things" is not a good process to follow, but in this case, this wasn't even a new thing

  75. moparisthebest

    That's what it says

  76. Sam Whited

    I didn't say "never do new things" just "don't do things that the SRV RFC didn't anticipate because it makes it needlessly difficult for no reason"

  77. moparisthebest

    You should mix them, if you don't want to, do whatever, have fun

  78. moparisthebest

    It might even be a may now?

  79. Sam Whited

    It did have a MAY I think.

  80. Sam Whited

    Well, it's a SHOULD, but then had a "MAY" for "or you can ignore this"

  81. Sam Whited

    NOt really sure what that means

  82. moparisthebest

    It's not needlessly difficult, it's not difficult at all actually

  83. moparisthebest

    Just because some go library programmer didn't anticipate it

  84. Sam Whited

    It is because I have to re-implement all the sorting and stuff instead of just calling LookupSRV (or the equivalent in your DNS Library of choice)

  85. Sam Whited

    It's not a Go thing, it's the SRV RFCs and literally every library.

  86. moparisthebest

    If you have your own srv code it's a 2 minute change

  87. moparisthebest

    I know because I made the change in Conversations

  88. Zash

    2 minutes to what?

  89. Sam Whited

    I don't want my own SRV code, I want to download a DNS library and make queries and get back to writing XMPP stuff.

  90. Sam Whited

    Even if it is 2 minutes it's 2 minutes I shouldn't have had to do.

  91. moparisthebest

    2 minutes to look up another record, and mix them

  92. Zash

    and keep track of if it's the 's' variant

  93. Sam Whited

    And not accidentally be buggy and do the wrong thing, or mix the ports wrong, etc. and write tests for all of it, and… I'm not saying it's impossible, or even hard, just that it's extra stuff that could have been done for me.

  94. moparisthebest

    Sam Whited: I could say I don't want to implement my own styling library, after all, that's much harder than this

  95. moparisthebest

    Guess we should have just used html since that would have been easier :D

  96. Sam Whited

    Lots of things are harder than this, we can talk about saving time there too maybe, but right now we're talking about this and the way literally every library does things and the way the SRV RFC makes it sound.

  97. Sam Whited

    That's a strawman, we discussed that and "it's easier" was in fact a valid argument for HTML.

  98. moparisthebest

    Except the ones mentioned

  99. Sam Whited

    Obviously SRV records does not have the same other concerns though.

  100. Sam Whited

    What ones mentioned?

  101. moparisthebest

    And the author of srv took a look and said he thought it was fine

  102. moparisthebest


  103. Sam Whited

    Right; just one other thing does it this way, but no library does and as far as I can tell the SRV RFC sounds like it defeats the point.

  104. moparisthebest

    I've never seen a library that does SRV at all

  105. moparisthebest

    But regardless, it's optional, feel free to do whatever you want

  106. Sam Whited

    Sure, and I do, I'm just not sure the XEP should confuse people into doing something that adds a ton of edge cases either way. Though like I said, the email RFC at least has some justification for it so we'll see, maybe I'll come around.

  107. moparisthebest

    The reason it ends up being optional is "client doing whatever they want" and "client being behind restrictive firewall" is indistinguishable to the server operator anyway

  108. Sam Whited

    makes sense

  109. moparisthebest

    Like I said if you already write your own (trivial) srv code mixing these in is equally trivial

  110. Sam Whited

    "trivial" is never a good argument for "add more code".

  111. Sam Whited

    I mean, if there's some compelling reason to do it that's fine, but "it's easy so why not?" just isn't that. That's how bugs get introduced.

  112. Sam Whited

    Not to mention that now it's a lot of SRVs to read and code and tests to write and edge cases to handle, so I don't believe for a second that this is "trivial" just because it's easier than some other things.

  113. Sam Whited

    (FWIW, I did also implement this at one point and then changed it, it wasn't the most difficult thing in the world, sure, but it's not "trivial" by any means)

  114. Sam Whited

    moparisthebest: ignoring all this for a minute, what is the purpose of letting the server decide between STARTTLS and implicit TLS? It just occured to me, I don't even know why it would make a difference (assuming both are supported, if one isn't it will be a '.' record either way or have no response so it won't matter regardless of mixing/connection strategy)

  115. moparisthebest

    Sam Whited: just because SRV let's server set priority and weight, that made sense to carry forward

  116. lovetox

    Zash‎: The Go network lib is the only I've ever seen, with actual SRV aware connection support

  117. lovetox

    > Zash‎: The Go network lib is the only I've ever seen, with actual SRV aware connection support

  118. lovetox

    does this count in GLib

  119. lovetox


  120. lovetox

    Gio.SocketClient.connect_to_service_async(example.net, "xmpps-client", ....)

  121. Zash

    Never used GLib, so hadn't seen that. But yeah, that's two.

  122. lovetox

    it resolves, afterwards does happy eyeballs

  123. lovetox

    thats the reason i dont do the mixing, this method exists its a oneliner for me

  124. lovetox

    doing the mixing, means i cant use that and now have to implement a lot myself

  125. lovetox

    fuck that

  126. lovetox