jdev - 2021-02-28

What would you consider a reasonable timeout for connecting to an xmpp server?

in total?

or to one connection point before you try the next

actually i dont think i have a timeout

not sure why i should, if i cant connect it does not change anything if i abort after a timeout, its not like i can suddenly connect after trying again

but i guess there is a timeout usually from your network libs

so i take whatever they do and when the return with "could not reach remote" or whatever

lovetox: presumably you want to go back to the start screen and say "Something's wrong, please wait and try again later" or something eventually right, and not just say "Connecting" forever even though it will never happen?

eg. if something was broken and the server stops responding, or the TCP connection hangs, or any number of other things happen.

hm its not like it will be stuck there forever though

Why not?

The point of a timeout isn't in case you hit an actual error, it's in case it gets stuck forever :)

because usually if you open a socket to some server, the network lib tells you after some time that something does not work

so yeah maybe these libs have timeouts implemented already

lovetox: The timeout before trying to connect to the next port (SRV records).

That's fine if it's at the TCP layer, not if you're connecting to something that's accidentally slow loris-ing you because it's overprovisioned and is returning 1 byte per minute or something very slowly by mistake

dont know how they determine that a host is not reachable

sorry Martin i dont do this right now

ah okay, those are two different things though. Between SRV records I'm not sure

I don't actually know if people treat different ports on the same host as different resources for rate limiting purposes, I would assume not. So maybe exponential fallback starting with 10ms or something would be a good baseline?

I just set it to 10 seconds while playing right now. Prior to setting it I blocked outgoing traffic on 5222 and it didn't continue for a minute, so whatever the underlying network libs use for a timeout, it seems to be terribly high.

But I think I'd rather set it to 1s or so.

Your network library waits 10 seconds before trying the next thing in a set of SRV records? That does seem like a long time.

No, I set it to 10 seconds to speed things up. According to mattn/go-xmpp there is no timeout when you don't set it. But I don't know if there will apply a timeout from the underlying network libs.

Oh, you mean it will jus tinstantly reconnect to the next one by default?

So it seems I would be stuck forever if I can't connect to the first record and don't set a timeout.

oops, too soon, not instantly, stuck forever. Gotcha.

But just trying all xmpp-client records, then all xmpps-client records is surprisingly easy. I'm not yet sure whether I should try xmpp on 5222 and xmpps on 5223+443 if no SRV records are provided. How is the common sense?

That's technically what people do as a fallback if no srv records are provided, but there was some discussion recently that it was a mistake. Right now I do that in my library too, but I keep going back and forth on whether it's worth it.

See https://tools.ietf.org/html/rfc6120#section-3.2.2

You can still specify it manually with `--jserver=example.com:5222` so maybe I should not bother and just use SRV records and don't care about fallbacks.

That sounds sane to me, FWIW

Or if there are no SRV records it would make sense to default to using that

I'll think about it. Thanks for your input. :)

Sure thing; let us know what you decide, I keep changing my mind on how I should do this too so I'll be curious what you do.

I tend to use 5222 if there are no srv records as I think servers either use the standard port or provide srv records.

I think it's nice to let the underlying system to decide on TCP timeouts, since those can be tweaked per-system and system-wide, depending on its connection or requirements. But as reference values, RFC 1122 suggests at least 100 seconds, and Linux's tcp(7) mentions the default of about 13 to 30 minutes.

I thought he was asking how long to wait between connection attempts to different SRV records, not for TCP timeouts, but maybe I'm still confused.

I really don't want to wait for minutes while my tool tries to connect. Rather try the next srv record.

Oh, after a failure, and before trying the next one? Why to wait between those at all?

Maybe you shouldn't, I'm not really sure. But if all the SRV records are just different ways to connect to the same service you may want to wait to avoid triggering rate limiting (this is why the old xmpp.net scanner waited, IIRC)

When I locally blocked outgoing traffic on 5222 and had no timeout set it didn't go on to the next srv record for one minute. That's why I added a timeout of 1s now as I want to try the next port quickly if I can't connect there.

One solution may be to extend "happy eyeballs" to cross SRV records. To neither give up on connections too quickly, nor wait too much if others are available.

This doesn't include timeouts, but I think this is up to date if you're curious how we're connecting: https://mellium.im/issue/2#issuecomment-735241044

defanor, yeah but happy eyeballs does not do this

and i think its even for a xmpp lib kind of advanced to implement happy eyeballs themself, not to speak from extending it to multipls srvs

im in luck that GLib supports happy eyeballs with just passing a srv record

but they fixed multiple bugs over the years on that code

i also think there is not much use for this cross srv thing in the wild ✏

> This doesn't include timeouts, but I think this is up to date if you're curious how we're connecting: https://mellium.im/issue/2#issuecomment-735241044 You look up xmpp and xmpps records in parallell?

I look up both records in parallel, then try xmpps first if they existed

I look up xmpp-client records, try them and if I can't connect look up xmpps-client records and connect. Should xmpps-client have higher priority?

The XEP suggests to mix them, so that priorities would be defined by a hostmaster. I was lazy to do that, and just prioritising xmpps too for now (after querying them in parallel as well).

The XEP is wrong IMO. It's breaking the SRV RFC and just makes no sense with how SRV records work.

I personally try xmpps-client first, but I'm not sure that it matters. In theory if xmpp-client is first you could negotiate without STARTTLS when TLS is available (if you support plain connections, which you shouldn't or should at least hide it behind some kind of flag or option)

There's prior art in the SRV for email RFC IIRC

Whether it's a good idea not is a separate issue

TIL: email discovery with SRV.

huh, fair enough, it recommends mixing imap, imaps, pop3, and pop3s.

This RFC makes it *much* clearer how priorities and weights are supposed to be used, it's somewhat convincing. Although it still seems like it's just unnecessary difficulty because libraries that query and connect aren't going to support this most likely.

How can xep368 be more clear in that aspect?

Crap libraries exist is not a reason to change a spec I think

This isn't crap libraries, this is the standard library of every programming langauge I've ever used, I think.

It's ok, standard libraries can be crap too

Actually, maybe not the standard library, but everything that does SRV.

Sam Whited, hold on, you're saying there's languages with SRV support? Other than Go?

Zash: that's why I said "maybe SRV libraries", I realized I'm not sure outside of Go if I had to download something else or not :)

But still, LookupSRV(service, proto, name) seems sane and simple. Having to provide multiple isn't something I've ever seen done because the SRV RFCs don't work that way.

The Go network lib is the only I've ever seen, with actual SRV aware connection support. Not counting pure DNS libraries.

This is the only way to do it with SRV, but SRV2 fixes this by letting you specify *how* to connect to each endpoint

moparisthebest: or don't do it and just say "Prefer this one if supported or then look up this other one"

Or leave it entirely up to the clients which one to do first.

"never do new things" is not a good process to follow, but in this case, this wasn't even a new thing

That's what it says

I didn't say "never do new things" just "don't do things that the SRV RFC didn't anticipate because it makes it needlessly difficult for no reason"

You should mix them, if you don't want to, do whatever, have fun

It might even be a may now?

It did have a MAY I think.

Well, it's a SHOULD, but then had a "MAY" for "or you can ignore this"

NOt really sure what that means

It's not needlessly difficult, it's not difficult at all actually

Just because some go library programmer didn't anticipate it

It is because I have to re-implement all the sorting and stuff instead of just calling LookupSRV (or the equivalent in your DNS Library of choice)

It's not a Go thing, it's the SRV RFCs and literally every library.

If you have your own srv code it's a 2 minute change

I know because I made the change in Conversations

2 minutes to what?

I don't want my own SRV code, I want to download a DNS library and make queries and get back to writing XMPP stuff.

Even if it is 2 minutes it's 2 minutes I shouldn't have had to do.

2 minutes to look up another record, and mix them

and keep track of if it's the 's' variant

And not accidentally be buggy and do the wrong thing, or mix the ports wrong, etc. and write tests for all of it, and… I'm not saying it's impossible, or even hard, just that it's extra stuff that could have been done for me.

Sam Whited: I could say I don't want to implement my own styling library, after all, that's much harder than this

Guess we should have just used html since that would have been easier :D

Lots of things are harder than this, we can talk about saving time there too maybe, but right now we're talking about this and the way literally every library does things and the way the SRV RFC makes it sound.

That's a strawman, we discussed that and "it's easier" was in fact a valid argument for HTML.

Except the ones mentioned

Obviously SRV records does not have the same other concerns though.

What ones mentioned?

And the author of srv took a look and said he thought it was fine

Email

Right; just one other thing does it this way, but no library does and as far as I can tell the SRV RFC sounds like it defeats the point.

I've never seen a library that does SRV at all

But regardless, it's optional, feel free to do whatever you want

Sure, and I do, I'm just not sure the XEP should confuse people into doing something that adds a ton of edge cases either way. Though like I said, the email RFC at least has some justification for it so we'll see, maybe I'll come around.

The reason it ends up being optional is "client doing whatever they want" and "client being behind restrictive firewall" is indistinguishable to the server operator anyway

makes sense

Like I said if you already write your own (trivial) srv code mixing these in is equally trivial

"trivial" is never a good argument for "add more code".

I mean, if there's some compelling reason to do it that's fine, but "it's easy so why not?" just isn't that. That's how bugs get introduced.

Not to mention that now it's a lot of SRVs to read and code and tests to write and edge cases to handle, so I don't believe for a second that this is "trivial" just because it's easier than some other things.

(FWIW, I did also implement this at one point and then changed it, it wasn't the most difficult thing in the world, sure, but it's not "trivial" by any means)

moparisthebest: ignoring all this for a minute, what is the purpose of letting the server decide between STARTTLS and implicit TLS? It just occured to me, I don't even know why it would make a difference (assuming both are supported, if one isn't it will be a '.' record either way or have no response so it won't matter regardless of mixing/connection strategy)

Sam Whited: just because SRV let's server set priority and weight, that made sense to carry forward

> Zash‎: The Go network lib is the only I've ever seen, with actual SRV aware connection support ✏

does this count in GLib

https://lazka.github.io/pgi-docs/#Gio-2.0/classes/SocketClient.html#Gio.SocketClient.connect_to_service_async

Gio.SocketClient.connect_to_service_async(example.net, "xmpps-client", ....)

Never used GLib, so hadn't seen that. But yeah, that's two.

it resolves, afterwards does happy eyeballs

thats the reason i dont do the mixing, this method exists its a oneliner for me

doing the mixing, means i cant use that and now have to implement a lot myself

fuck that

:)