XSF Discussion - 2020-12-01

dwd, spreadsheet of doom?

jonas’, https://docs.google.com/spreadsheets/d/1-IBY4NhXbdZ_TqCCZkXrMgqq63yytaxXqJKhh47J1vI/edit?usp=sharing - but poke it about to see if I've cleared the old data off it correctly first. I think I have, but ENOQA.

dwd, I need your approval for +w, I sent a request

mathieui> whoops I forgot to look at pending PRs before submitting my MIX patches, but it looks like there’s no overlap at least with which of the current PRs do you see a conflict?

jonas’, no conflict, but I saw a few MIX-related ones after looking

dwd, nevermind, I forked it

mathieui, which ones?

jonas’, Oh, sorry, didn't think of that.

jonas’, only #856 actually

right, that one’s not gonna happen soon anyway

yup

there are deeper changes in other ones that I didn’t want to get into, e.g. reusing 0424 in 0407 section 4 instead of providing an ad-hoc mechanism+namespace

Ge0rG, can you obtain approval by the author of '401 for: https://gitlab.com/xsf/xeps/-/merge_requests/33

cc @ marc

jonas’: okay for me if that's the MR Ge0rG was talking about last time

marc, please check if it really is that PR :)

jonas’: yep, later. I'm on mobile atm

marc, thank you

it's the one splitting out IBR with token from 0401. I'm not so sure any more about the title, maybe I overdid it a bit

Re https://github.com/xsf/xeps/pull/1008 Does anyone actually enforce or even handle that?

I do in my implementation (just by virtue of using a uint16 which in Go wraps around if you overflow it)

I can't imagine it's likely to be something that's hit often (or ever) though.

Yeah, it goes into "don't use IBB for files that large" territory

Given a maximum block size of 65535 that's 4 GB. So, plz don't.

I did look around and found several other implementations that explicitly handled it (Smack is the only one I remember looking at now though, but I think there were some of the python ones too). They used uint16

I just looked at two and saw no trace of any overflow handling

this one seems to do stuff tho

https://i.imgflip.com/4oiu0k.jpg

yah, maybe it wasn't Smack, I don't see it now

it is wrapped around in slixmpp afaik

ah no, here it is: https://github.com/igniterealtime/Smack/blob/master/smack-extensions/src/main/java/org/jivesoftware/smackx/bytestreams/ibb/InBandBytestreamSession.java#L759

Is that off-by-one?

And here it is for aioxmpp: https://github.com/horazont/aioxmpp/blob/devel/aioxmpp/ibb/service.py#L195-L196

Yah, that looks off by 1 to me

Haven't found anything in nbxmpp and Swiften tho

Nobody so far seems to have accidentally read it as a the positive subset of a signed integer, which I think is a thing somewhere else.

Zash, good catch

np

Yah, good eye; this is why I never use weird ternary constructs, I would never have noticed it in that mess of symbols and what not

I've just looked at the origin of this: it was introduced by henning in 2010

so it's a 10 year old bug you discovered \o/

:D

Nice, Zash gets all the points

of course i wonder if this ever caused an troubles :) but that doesn't mean that this shouldn't get fixed

Heh, yah, I doubt it

Zash, oh, slixmpp is also off by one as well I guess

FWIW, java does integer wraparound too IIRC, so that could probably just be an "unsigned short" or whatever it would be called in java land

Nice; multiple off by ones fixed today! As they say, the three hard problems in computer science are naming things and off by one errors

tricked by a modulo

SamWhited, java doesn't have unsigned anything

oh, nevermind then. Been a while since I did any Java

SamWhited: nice chain of events you set in motion here :)

Yah that worked out nicely; I thought it was just going to clarify a typo fix, good teamwork turning it into bugfixes

I just hope nobody ran into that, as it would reject stanzas with the 65535 sequence number

SamWhited, the problem is that there is no uint16_t in java land

I can't imagine anyone actually sends multi-gig files (assuming a reasonable blocksize) and doesn't get bored and cancel or have some other hickup

But who knows

The best file transport!

I'm sure one crazy person somewhere has done it

Doing 4k livestreams over IBB

Absolute madlad

Tangentially related: has anyone ever experimented with different blocksizes to see what a good default is? I just kind of guessed that ~2k was probably a good default, but I really have no idea

For IBB? 🤷

somewhat related to stanza size limits I reckon

I imagine there's some sweet spot wrt filesystem block sizes, tcp/ip/ethernet packet sizes etc

I assumed ethernet MUT would be smaller than TCP and would be too small to actually fit a stanza in (without drastically increasing the overhead just from the XML), but maybe wifi would be a better thing to lookup

MTU, even

4k block gets you a ~5.5k message stanza, that's about half of the 10k minimum stanza size limit. should be safe, even with longer jids.

a JID can be up to 3k in theory...

That sounds good to me, maybe I'll up mine which I think was at 2k

that would be a case of "some people get what they deserve" (re: 3k JIDs)

dns limits you to 255 or something for the hostpart, so more like 1023@255/1023

Could be a hostname though

No idea if there are limits there

A message stanza with 2x 1023@255/1023, 2x UUID and 4k IBB block turns into 12.5k

heh, with 2k block it looks like 9.8k

so if you're absolutely want to stay clear of the strictest stanza size limits then 2k blocks would do.

meh, if you have a 3k JID you can manually set the block size, that's definitely not a thing I'm interested in optimizing for

Would be cool to have real network troughput stats ;)

jonas’, +1 for the PR from Ge0rG

marc: thanks ✏

Path-MTU!

Mmmmmmm. I've observed MUC participants in loops where they request the vcard of someone, then get their connection closed because the avatar was larger than the stanza size limit

Much fun.

big avatar crashes MUC's you say ? >:)

khekhekhe

fun fact: you cannot kick MUCs off the s.j.n list that way (except the one with the evil avatar) because s.j.n shuffles the list of MUCs during each scan :)

(even if it caused a longer outage of the s2s connection or killed the c2s connection in a bad way or somesuch)

Was there some point in advertising stanza size limits in stream-features or somesuch?

That would be really nice; at the very least it would let clients show a rough max-message-size while you're typing.

FWIW, I assumed that stanza sizes were reasonable huge these days. Like 64k would be an absolute minimum. Didn't we say 10M in RFC 6120?

10k

10k

Prosody ships with a default of 10M by default, ejabberd with 256k for c2s and 512k for s2s IIRC

Dunno about others

Eh, default defaults ftw

I think Openfire does 2M.

So we need Path MTU Discovery! Will s2s be terminated on oversized stanzas? Is there an error sent to the originating node?

With bandwidth and RAM sizes these days, I'd think IBB at 10k blocksizes would be fine. That'll yield a 1ms choke on a 80Mbit link, so you could probably increase that by a factor of ten without worrying.

Ge0rG: yes. maybe, but probably no.

Push harder for s2s-198

dwd: not on german mobile "broadband"

Zash: won't help on stream errors

Why?

Ge0rG, So you don't want to much a PMTU discovery, you actually want a IBB blocksize negotiation.

You want PMTU discovery for vcards/avatars!

I think my implementation for blocksize "negotiation" was to just request the default, then cut it in half until they accept the transfer or it gets under some small minimum where we deam the XML encoding overhead is too high to justify bothering

Zash, But if you're on a slow link, do you even want the jumbo-avatar?

Why you want the multiple-choice 0084 allows

dwd: how would you do that for the involved servers?

Ge0rG, Hmmm, yes. But you probably want a path characteristics request, plus a stream feature for stanza limits. We talked of a path characteristic command for TLS a while back; one for stanza limits and 198 might be more useful.

dwd: all of the above, and a way to discover each hop on the way