jdev - 2020-07-26

Just wondering, is there any particular reason why xmpp isn't length prefixed and rather the whole stream is one never ending document? I mean if it were length prefixed than inband binary data could be transmitted without b64, among other things ✏

sounds more complex to implement ✏

right now i just feed the stream to a parser

i try to think about when i need inband data transfered

and only avatars come to mind

> sounds more complex to implement Quite the contrary. If stanzas were length prefixed standalone documents you could just feed them to a regular dom building parser of which there are plenty for every language.

and what is the difference to the parser i push it now?

do you mean the xml would just end in the middle, like <data>binary data</data>? ✏

if yes, this would of course be nice, but i maintain a decade old client, and as i said the only usecase come to mind are avatars

for everything else, you would want to have a out of band transfer that you prefereably can do async

and i was not there back then, but i guess avatars was not on the mind of xmpp creators, and even if it would be debateable if you design a whole protocol around transfering avatars which is like 1% of xmpp usage or less

Data framing has some nice properties, but would essentially need everyone to write custom parsers

XMPP over websocket is framed, FWIW, and supported by many servers

Жокир, I think the simple answer is: because XMPP re-uses XML as building block

also it is *not* a never ending document, there is an end (at least if you use a transport that closes the connection by sending a closing </stream> tag)

There are limits to max stanza lengths. I don't think that the spec defines a limit, only that one is allowed to exist.

(although I can't immediately find it. Am I wrong here?)

> A deployed server's maximum stanza size MUST NOT be smaller than 10000 bytes

https://xmpp.org/rfcs/rfc6120.html#rfc.section.13.12

the whole domain name stuff is insane

luckily python has a idna lib for that

and i discovered just now luckily it has a UTS46 mapping mode

IDNA 2003/2008 issue?

2008

but imagine there would be no python lib for that

and everybody who implements xmpp, needs to implement the whole idna spec with it

probably would stop right there and do something different

yeah easy to miss, the IDNA2003 spec required a mapping phase in the first stage

for example XMPP.net -> xmpp.net

IDNA2008 does not require or describe that phase anymore

so if you do purley idna2008, then you end up with different JIDs here

it seems they excluded that mapping spec to some other standard because its no strictly required anymore

IIRC our IDNA 2003 code didn't do any lowercasing.

just discovered this now

But the current stuff seems to do it

"current" = future, actually

apperently it does

http://unicode.org/reports/tr46/#IDNA2003-Section

or it did

$ idn <<< XMPP.net XMPP.net $ idn2 <<< XMPP.net xmpp.net

🤷

then this is weird, because the standard says, IDNA2003 does require a mapping phase

IDNA2008 does not

but maybe this idn2 application just uses sane defaults

and you can disable the mapping

just to be clear, 2008 does not require it as part of the IDNA standard

Keep in mind that with JIDs you'd do some stringprep or precis operation on the host part first

but of course application can do a mapping if wished

IDNA is only for when you do DNS lookups, and DNS is case-insensitive

Zash, you dont do any precis operation on the domain part

Not?

This topic. Are you trying to make everyone cry themselves to sleep tonight?

no Zash, it refers to the IDNA rfc

so we dont need IDNA only for dns lookups

we need it to know that a@XMPP.net = a@xmpp.net

So IDNA becomes the nameprep step then.

yes

to be precis IDNA2008 does no mapping, and requires no mapping

you need a lib that implements IDNA2008, and the UTS46 mapping or some equivalent

which is not specified in the idna spec

hence, my first statement its all insane, and if there would be no lib that would just do that for me

i probably would stop right there to develop my client :D

I know the feeling.

Hence the part about crying.

We don't have any PRECIS lib atm. We use libidn for stringprep and IDNA (2003). But libidn is deprecated and replaced by libidn2, which does *only* IDNA 2008. No stringprep or precis. Thus not an option.

and just out of interest i looked up the docs of your application

and indeed it uses the sane default

but you can disable it

--no-tr46 Disable TR46 processing

$ idn2 --no-tr46 <<< XMPP.net xmpp.net

hm

lovetox@lovetox:~$ idn2 --no-tr46 XMPP.net XMPP.net

dont know what your "<<<" does

but nothing good :)

bash magic, dumps that string to stdin

heredoc without eof marker

pretty handy imo

yeah but not if it ignores your options

$ idn2 --no-tr46 XMPP.net xmpp.net

Weird

thats indeed weird

i use version 2.2.0

https://gitlab.com/libidn/libidn2/-/blob/master/NEWS#L25

great :)