XMPP Service Operators - 2021-04-27

What is o.j.n ?

observe.jabber.network

tom, I’m not going to make monitoring data of domains public without their consent, buuut: https://github.com/horazont/xmppobserve-web/issues/3

Problem is with exposing that data without also exposing the monitoring data of all my private services, as the infrastructure is currently shared

I have another box I could use to help with that, but its availability is questionable, so I’m hesitant there

I need to check the memory use on the new monitoring box and whether it’d allow me to split the TSDB in two instances there

I already have a closed beta where ojn users can access their own monitoring data in a grafana instance ... but that still has too many issues (so registrations for that are currently closed and that’s also why it’s not advertised anywhere)

hoping to improve on all that in the future, but Prometheus is not really made for tenant separation and running a separate instance for each user is going to be too expensive. So I’ll most likely have to roll my own proxy in front of that which correctly isolates the data

Hi everyone. As a reminder (just in case) I am the operator of the chapril.org server. I have had user feedback that audio/video calls do not work when both parties are using 4G (LTE). They can only connect if at least one of them is behind a NAT (for example using wifi or ethernet). What could be the problem? (We use ejabberd)

add a TURN server

I think your observation is incomplete; 4G/LTE is very likely to be a CGNAT which tend to be worse for peer to peer connectivity than "classic" NATs.

so while "classic" NATs can sometimes be worked around either via proper IPv6 support or hole punching, CGNATs are nasty

and a TURN server helps with that

when you deploy a STUN/TURN server pair, please pick a random port number (e.g. from the range 8192 -- 32768); the standard port number is being abused in DDoS attacks.

neox, ^

jonas’, well... I didn't know about CGNAT, thank you. We already activated the ejabberd internal TURN server, and it relays data sucessfully, so perhaps is it misconfigured for that special case ?

neox, how do you know that it is relaying data successfully?

jonas’, i see that in the ejabberd logs

what do you see?

jonas’, an amount of data relayed

I don’t know how and what exactly is measured there

have you run the tests described in https://gist.github.com/iNPUTmice/a28c438d9bbf3f4a3d4c663ffaa224d9 ?

or more specifically: https://gist.github.com/iNPUTmice/a28c438d9bbf3f4a3d4c663ffaa224d9#testing

I had a subtle misconfiguration in my TURN setup which made everything look great except when used via XMPP (typo in the prosody configuration---so probably not directly relevant to you)

jonas’, thank you very much. I did not check with that tools, I'm doing it right now

good luck! :)

> when you deploy a STUN/TURN server pair, please pick a random port number (e.g. from the range 8192 -- 32768); the standard port number is being abused in DDoS attacks. Ah, that's what it is. I see strange traffic on 3478/udp ~15-25kbps from time to time...

balabol.im, yeah, that seems related

jonas’: i'll try to change portnumber, thanks for advice :)

balabol.im, make sure to also change the "alt" port number and set it to the main port number plus one

jonas’: hmm...i don't remember setting the _alt_ port...you mean turn tcp/5349?...maybe you're talking about prosody?

no, alt port is 3479 by default

if you’re using coturn, it is literally called `alt-xyz-port`

Oh that's good to know, I only changed the regular port

No, i use ejabberd with built-in turn Afair there wasn't _alt_ port

> when you deploy a STUN/TURN server ... the standard port number is being abused in DDoS attacks. TURN is authenticated, isn't it?

yeah

and that makes it worse

or well, the protocol around authentication makes it worse

the problem here is that you can send a 20 byte UDP packet to a STUN server and get 100 bytes back

so for every byte of traffic sent to the STUN server, it emits 5 bytes.

Even without access?

yes

that’s just STUN

with TURN and auth it gets worse, because the "not authorized" message is even larger

That is concerning

exactly

and UDP being UDP, an attacker can spoof the source address if their ISP is crappy (or suitable, depending on how you look at it) and thus make the reply go wherever they want

and there is no way to track them down except with very sophisticated techniques basically requiring cooperation of the entire internet.

then again, STUN is pretty good still with an amplification factor of 5:1 ✏

given that any random authoritative DNS server will probably have a better yield

and you can’t move *those* to a random port.

Is it viable to disable UDP? Is voice over TCP that inefficient?

not viable, ICE works over UDP

and yeah, voice over TCP is bad. a single dropped packet will cause a massive delay and will take a while to recover from also bandwidth wise

while a single dropped packet with UDP may not even be noticeable if the codec does forward error correction.

Thanks for the clarification

but the actual media stream is irrelevant here, because that only happens post-auth

the problem are the parts needed during ICE, before anything has been negotiated ✏

this is clearly a design flaw in STUN/TURN.

"not authorized" should not cause any reply in UDP-based protocols.

but here we are

jonas’: thank you for explaining, it was very interesting to know

you’re welcome

> Oh that's good to know, I only changed the regular port I forgot I didn't open that port so it doesn't matter in my case

I know I mostly lurk here, but this ☝ is exactly why I continue to lurk. I love how informative and educational it is. ♥

jonas’: > this is clearly a design flaw in STUN/TURN. > "not authorized" should not cause any reply in UDP-based protocols. Yeah, the problem is how STUN/TURN auth is challenge-response. The server won't know whether the response will be valid before sending a challenge to the possibly spoofed IP address.

Mmm

Edumacayshun

Gosh, my computer keeps hanging, despite the oboard intel watchdog being armed

It locks up where not even the numlock key works on a PS/2 keyboard, despite the keyboard being directly wired to the second highest interupt

Below timer

I have to press reset on the physical board, of which it doesn't do anything for 15 seconds then kills power

During the hang POST code is still 00

Do intel watchdogs have to be tickled a certain way or something?

Could this be due to bad ram timings?

Establishing a secure connection from muc.tigase.org to lebihan.pl failed. Certificate hash: 6309c033094e3d8fb71d4dea59197ac81bd38240a4c03a68c10fee75fc09ac47. Error with certificate 0: certificate has expired.