jdev - 2020-04-13

so for domainnames normally i use stringprepare

nameprep

but in python there is module available that does the idna2008 standard

I don’t like where this is going

can i just switch to that, or is this going to be problematic?

Me neither

no, those are different things

really?

you need to do nameprep first, then you can IDNA-encode the name before handing it to the DNS server

IDNA is an encoding (unicode -> DNS-compatible ascii bytes)

like UTF-8 is an encoding

problem is: there are two incompatible versions of idna, and nobody knows which one to use

im talking about validating domainnames

nameprep

in python there is a idna standard module

that’s all validation you need

it offers a method thats called nameprep

there is no idna module in the python standard library

according to https://docs.python.org/3/library/

its a submodule of stringprepare

there is no stringprepare module either

or stringprep dont know

stringprep doesn’t have submodules AFAIK. most certainly not an `idna` submodule

It hasn't been long enough since some user put http://example.com:5280/ as their XMPP server name and everything was fine with that.

I’m wondering what you’re talikng about. LTIC python only supported IDNA2003

ok https://github.com/python/cpython/blob/e42b705188271da108de42b55d9344642170aa2b/Lib/encodings/idna.py

its a encoding

as I said, yes

and it’s IDNA2003

not 2008

again, IDNA* are encodings like UTF-8 are encodings. has nothing to do with validation or normalisation like nameprep does

yes and it has a method call nameprep

neat

but it’s not public API

seems as if nameprep is a precondition for IDNA2003 and they’re doing that for you

oh ok so thats what confused me

you can do a call to codecs.encode(some_domain, "idna") and if it doesn’t raise UnicodeEncodeError, then you know that it passes nameprep

soo can is witch to nameprep -> IDNA2008

?

no

it’s idna2003

not idna2008

also note that nameprep has been deprecated in XMPP

you just said nameprep has nothing to do with idna, its just a precondition

so what is it now

TIL that nameprep is a precondition to IDNA2003.

it doesn’t matter tho

IDNA2003 does more than just nameprep

if you want nameprep, you should do nameprep and not IDNA2003

You actually want IDNA too, nameprep isn't enough to validate an XMPP hostname.

domain. hostpart. thing.

Zash, IDNA doesn’t buy you much though: >>> codecs.encode("http://foo:5082", "idna") b'http://foo:5082'

See previously mentioned incident with someone having a HTTP URL in their config

wut

yeah.

it’s just nameprep plus some mapping of characters

plus length restrictions

> print(util.encodings.idna.to_ascii("http://foo:123/")) nil

is that IDNA2003 or IDNA2008?

Probably 2008

might be the difference

python3 only has 2003

haha

yeah

but don't python also have ... whatsitcalled, precis?

the issue is just open for seven years now: https://bugs.python.org/issue17305

Zash, not built-in

there’s a third party module

better than nothing

true

there also seems a third-party IDNA module which does things

and then again: I’m not sure XMPP software should make assumptions about what DNS allows.

: is not allowed in IDNA2008

let DNS deal with the weird things we put in the domainpart.

it’ll tell us to f* off. encoding (too many) assumptions about how DNS works and what it allows seems like it can only lead to a world of pain

normalisation makes sense for comparision and stuff, but beyond that…

`idn -a <<< "http://foo.bar:123/"` spits out its input

and that would be idna2003

so i still dont see no problem to run a host through idna 2008?

probably ok

not sure what that gains you tho

and if it returns an exception, tell the user its not a valid domain name

I hate that type of stuff

that’s the type of stuff which breaks new things

you’ll notice that it’s not a valid domain name when you ask the DNS about the name

(though, you need to do IDNA2008 or IDNA2003 before you ask the DNS)

(but you can’t know which one is right \o/)

The least painful answer for us: IDNA2008 in IDNA2003 compat mode

IDNA2003 library support status: deprecated.

so what does the xmpp standard say about domainpart

libidn v1 deprecated libin2 dosen't do nameprep & co, can't manage xmpp parts

no validation at all?

it does for node and resource part have precis modules

lovetox: "the", there are like 3 of them

3 versions

the lastest

idna2003, idna2008, precis

so wtf it definitly says a domainpart has to conform to idna2008

so i certainly run it through idna2008 and be finished with it

Latest is https://tools.ietf.org/html/rfc7622

yes thats what iam refering to

I love how we still haven’t figured out how to do unicode release interop

This be the IDNA2008 + PRECIS thing righht?

yes

and idna2008 has no nameprep as precondition

I dropped that in #debian-til and someone got nerdsniped by that and dug out that the unicode releases also don’t really have guidelines on compatibility

so actually i can throw out all nameprep stuff

jonas’: No way we're doing that while in the middle of a pandemic, can't afford to waste painkillers and anti-fever meds on this horror

:D

luckily, the next unicode release will be postponed due to the pandemic, too

Praise Glob

lovetox> so i certainly run it through idna2008 and be finished with it then you potentially disallow ipv4/ipv6 addresses in the domainpart, which are allowed

after i check if its an ip :D

lovetox, bonus points for allowing ipv6 scope IDs

i do

then here are your bonus points

but only if you include it within []

lovetox, whatever the IP-literal rule of RFC 6874 allows