XSF Discussion - 2021-03-23

I'm kinda confused by this change: https://xmpp.org/extensions/xep-0176.html#revision-history-v1.1 > Make the 'foundation' attribute a string instead of an unsignedByte, in accordance with RFC 8445 [1]. XEP-0176 is meant to implement RFC 5245, XEP-0371 is meant to implement RFC 8445. Why would we adjust XEP-0176 to be in accordance with a related but incompatible RFC?

i’m confused about the existence of 371

couldn’t we have just made incremental changes to 176

The RFCs are not fully compatible, so, no?

I mean apparently a bunch of clients seem to do RFC 8445 over XEP 176 now

Because it was already widely used and looked similar to what they're doing I guess

is there any client doing 371?

Don't think so, but most WebRTC clients probably should

Assuming they use libwebrtc

Given that most (if not all) WebRTC clients are incompatible with what was around before, there really was no reason to stick with the 176...

to be honest for me it was a discovery thing. i didn’t even know about 371

or have a good enough understanding to be able to tell the differences

(i still don’t know the difference)

without putting words into peoples mouths I assume this may have been similiar for the two implementations that predate Conversations

We’ve always struggled for review on Jingle stuff, *especially* Jingle WebRTC stuff. There’s a couple of very knowledgeable people around, but nothing like the volume of knowledge we have relevant to most XEPs.

larma, 371 used to use 5245 as well and was then upgraded to 8445

without any problems

so i'm wondering if the same can not be done for 176

(possibly with some extra disco features for example for "i support 176 + tcp candidates" or something)

I implemented a client of 176 that implements RFC 5245 and it can't be connected from Conversations because libwebrtc implements RFC 8445 (it probably works the other way round because that changes who is the controller). Incompatibility to me sounds like a good reason not to update/change a Draft XEP

OTOH we now have a bunch of clients claiming 176 conpat in disco that are not fully compatible, so I guess the damage is already here ✏

larma, I originally only wanted to extend from unsignedByte to unsignedInt, as that was what libwebrtc (and all clients based on it) was doing.

But RFC 5245 was already requesting a string up to 32 characters (whatever that means, probably bytes).

“ foundation = 1*32ice-char”

“ ice-char = ALPHA / DIGIT / "+" / "/"”

Link Mauve: I guess I was more confused about the reference to RFC 8445 than the change itself

Ah hmm, maybe I was the one who got confused, as 5245 already had the exact same requirement and we imported it as an unsigned byte instead, probably in accordance with implementations of the day.

larma, AIUI the incompatibility doesn’t come from 5245 vs. 8445, but from mandatory DTLS-SRTP vs. not supported.

That's something else

Or maybe you’re aware of some incompatibility between the two RFCs that I’m not?

I'm specifically talking about ICE

8445 allows sending data before candidate nomination, basically making candidate nomination optional (also the reason the got rid of the aggressive nomination mode). 5245 requires nomination before sending. If the controlling side is 8445 they might never nominate a candidate, thus the 5245 side can't send.

I had to discover that through wireshark ;)

How is that negociated in non-XMPP deployments?

They probably all use 8445 nowadays and just updated when Google dictated them to?

All legacy telephony systems?

They're not compatible with WebRTC anyway due to dtls-srtp

So nobody cares I guess

Others would just make their client such that it's compatible with legacy systems (e.g. by nominating a candidate pair even if not required for them)

If care is taken, you can implement ICE clients that are compatible with different versions of ICE. For example, some early drafts of ICE required sending USERNAME attribute in responses. If you aim for compatibility you can just do it, as adding more attributes than required is allowed and they simply get ignored by RFC implementations

Is “candidate pair” the proper name of a c= line in SDP?

No, that's just the candidate

Because I’ve seen implementations include a dummy one.

In the initial offer.

The pair is a combination of local and remote candidate

Safari does not send proper candidates before granting audio/video permission

For privacy reasons

larma, if there is any low hanging fruit like sending transport-info/remote candidate or someting that improves the situation for you let me know

I guess we'll just make sure to be compatible with RFC 8445 as well when using 176...

Daniel: you mentioned TCP, do you actually do RTP over TCP?

larma: I've disabled that because 176 doesn't support it

Until know I was under the impression that this is the only major difference

As this is what the introduction of 371 focuses on

That was also my impression btw, which is why I never pushed for 0371.

Well ICE-UDP is a datagram transport and ICE-TCP is a stream transport but according to 166 each transport should decide to either be datagram or stream and according to 167 RTP is only allowed over datagram

I referred to the RFC to understand some things, but didn’t compare them thoroughly.

I think having ICE-UDP and ICE-TCP not in the same XEP or at least not use the same namespace makes sense. I also wonder what libwebrtc does when setting up RTP over TCP because it will need to encapsulate the RTP datagrams somehow so they can go on a stream

(encapsulate can be as easy as prefixing with a length)

are XMPP addresses case-sensitive? couldn't find such info in https://tools.ietf.org/html/rfc6122

No

The local part is normalized

Aka 'fancy lower cased'

The fancy part means you can’t just lowercase it and do the comparison, you have to use stringprep.

Not doing so is a vulnerability awaiting.

Yes if you are asking from a developers perspective definitely understand how stringprep works. From a user perspective it's simpler to just think of lower case

L29Ah, so you would look at https://tools.ietf.org/html/rfc6122#appendix-A and then look up the referenced tables in https://tools.ietf.org/html/rfc3454

E.g. https://tools.ietf.org/html/rfc3454#appendix-B.2 is the long version of "lower-case"

thanks!

Anyone available for doing React and Stanza.js for a contracting gig?

Don't forget https://xmpp.work/

sonny, ↑

nope pretty busy ATM

L29Ah, check https://tools.ietf.org/html/rfc7622

Check this handy graph ```dot digraph "xmpp-addr" { rfc3920 -> idna2003 [label="ref"] rfc3920 -> stringprep [label="ref"] rfc3920 -> rfc6122 [label="update"] rfc6122 -> idna2003 [label="ref"] rfc6122 -> idna2008 [label="ref"] rfc6122 -> stringprep [label="ref"] rfc6122 -> rfc7622 [label="obsolete"] rfc7622 -> idna2008 [label="ref"] rfc7622 -> precis [label="ref"] idna2003 -> idna2008 [label="obsolete"] idna2003 -> stringprep [label="ref"] } ```

Rendered:

https://matthewwild.co.uk/uploads/xmpp-addr.png

MattJ: what did you use for rendering?

dot -Tpng

Thanks

FTR I pasted the source so that in the far future, after all the http-uploaded files have expired, someone could still see what it was

This URL isn't expiring any time soon :)

i have a string, "／"

is this a valid XMPP address?

no

why?

oh, it is

it gets normalised to `/` which is apparently a valid domain name.

I mean "valid"

it might be a valid domain name but when parsed as a jid after normalisation it would denote an empty domain name with empty resource

Oh no, is this another case of "turns out a HTTP URL is a valid hostname"?

Zash: no, i'm figuring out why i have a failing test in a xmpp library i maintain

L29Ah, oh no.

that is ... interesting

the test suite has come up with such a nice jid

that the library can serialise (with normalisation) but can't parse afterwards

It's invalid because `/.` is not an existing top-level domain.

Zash: it might be /.local.

Oh no

But _that_ isn't valid

is it not?

empty host, `.local` as resource!

no

not if written as ／.local

normalization of the domainpart happens *after* splitting

True

can’t be reserialized as JID anymore though :)

Also true

L29Ah, congrats, that’s a terrible test case

might even warrant an erratum to RFC 6122, would have to check if 7622 has the same issue

"awesome test case", you mean?

L29Ah, yes, in the original sense of the word

well, in both senses that is

L29Ah, where did you find it?

16:01:35]<L29Ah> the test suite has come up with such a nice jid

which test suite?

it generates random strings

quickcheck

oh

awful

I mean amazing

but awful

you know what I mean

ooh, that's a good one. Works, but definitely seems like something we should be making illegal somehow https://play.golang.org/p/F7GRBN3EkDX

(that example is using 7622, you may have to run it twice for it to cache the library and not timeout because of the download time)

Sam, thanks for the 7622 check

“not if written as / .local” Why does that space matter? It’s still split on the / so the latter part is the resource?

Kev: that's not a space :)

Kev, it’s not a space, it’s a strange slash character

(it's not a '/' either :) )

U+FF0F FULLWIDTH SOLIDUS

Ah, if it’s not / then ok :)

Should be fine because it will never be normalized to '/' in 7622 at least, but it *looks* confusing which is possibly bad

Yes, “fine” might be quite subjective here :)

Eg. if I buy "google.com／definitelynotevil.com"

"Hi, I'm an important person from Google which you can verify from my JID!, I want to give you 5 million dollars!"

ok seems like i need to replace my stringprep stuff with 7622 stuff

L29Ah: does stringprep normalize that to '/'? Yah, that seems like a potential place for serious bugs

yes it does

Is this the point where I repeat my usual complaint that precis-based JIDs are broken? :)

Kev: in this case it appears to be the stringpep based JID that would break.

Does PRECIS fix this?

Sam: In the sense that there’s no upgrade path from stringprep to precis, and therefore they can’t be safely used on the same network unless they’re compatible, and if they were compatible (they’re not) there’d have been no reason to go to precis.

It just uses IDNA's toUnicode for domains, so yah I guess so. It seems bad that stringprep does more than that, because then you'd end up with domains that are valid but you can't use them for JIDs

But I haven't really looked, so that's assuming this is all true and not a bug in one of our libraries.

Kev: they're compatible enough in the real world. But yah, the upgrade path is tricky. If this is really true though, I'd say it's a huge reason to upgrade. stringprep not allowing valid domains in JIDs seems really bad

Anyways, back in a bit then I need to write some tests around this.

“They’re compatible enough in the real world” is exactly the point. People make that argument, but if it were actually true there’d be no reason to need to go to precis.

https://tools.ietf.org/html/rfc3491#section-4 yeah nameprep tells it gets normalized

We can’t both have the “stringprep is broken, we must precis” argument and the “but it doesn’t matter because they’re compatible in every way that matters” argument both being true :)

How about we just be very conservative with new usernames and stuff until we're all on PRECIS

Zash: With which unicode version?

ASCII!

screw that, i'll just sanitize stringprep output for @ and /, and tell that i'm done

AFAIK, you can’t safely mix unicode versions in XMPP (nor in JIDs), and you can’t safely mix precis and stringprep-based JIDs. But we pretend that you can, while pasting “This is fine” GIFs :)

Kev: I disagree. They're compatible in every way that matters, but stringprep is broken in a way that while it's not likely to be seen in the real world seems bad, but not because of its incompatibility with precis, because of its incompatibility with DNS.

I mean, don't get me wrong, we should come up with a graceful way to upgrade and handle the differences, I'm just saying that 90% of the time it actually is fine and stringprep is apparently broken in ways that have nothing to do with its incompatibilities with precis.

I don’t disagree that precis might be better, I just don’t see how we can be expecting everything to work nicely without upgrade paths.

Yah, we should come up with some of those.

It's just going to be hard to get anyone to do it because it will work well enough, so check your database, if there are no differences in any JID or JID that people are sending to: go ahead and upgrade. If there are, wait for an upgrade path.

step 0: find all JIDs in a database

that’s a challenge in and of itself

is your database the only one that matters? doesn't it break interop with everyone else too ?

I'm still thinking Be strict when creating local resources (users, rooms, nickname registrations...) Be relaxed with incoming data from other servers.

kinda like if your email server can enforce authenticated TLS only just turn it on, and enjoy your no messages from anyone from that point forward :)

Zash: 🤖 disagrees with that.

🤖️ doesn't exist, it can't hurt you

It’s even wider than JIDs, actually (Unicode, rather than precis/stringprep).

okay, is "foo@bar:666/baz" a valid jid?

Even with message bodies, unicode version changes can break things.

Are those all the characters they look like?

：

L29Ah, maybe but I don't like it

IIRC you're allowed to put port numbers in the domainpart, but I doubt it'll be reliable

there's generally a wide difference between "what the spec allows" and "what you can actually use in practice"

L29Ah: `[db8::123:abc]` is a legal domainpart too, and that has `:` in it

Zash: oh, thanks

Not completely sure about port numbers

I recall a poor lady named something like Mary O'Toole who's email was Mary.O'Toole@domain.org which is a perfectly valid email that most email systems refused to deliver mail to

not mentioned in 6122

Don't mention email :(

I thought that port numbers weren’t allowed, but would have to scour specs that I don’t want to scour at teh moment.

point being, it's the same in XMPP land, you can't use PRECIS for this reason

ok so i just strip / and @ from domainpart, and declare all jids with domainpart that normalizes to a string with those invalid, what do you think?

s/and/or/

Sounds sensible

> Implementation Note: When dividing a JID into its component parts, an > implementation needs to match the separator characters '@' and '/' > before applying any transformation algorithms, which might decompose > certain Unicode code points to the separator characters.

... but then nothing about what to do if that actually happens 🙁

L29Ah, so what's the lib in question? I only know quickcheck from haskell and there I know one XMPP lib ✏

flow: pontarius-xmpp

I figured

so is pontarius alive again?

no

but i took over to keep pontarius-xmpp in shape and hopefully write a decent XMPP client someday

well, you came here and ask the right questions, so I'd say you are on a good way ;)

at least now i have a replacement for the bitrotten sendxmpp: https://github.com/l29ah/hsendxmpp

Everyone does

google failed me :(

It's slowly growing into the hello world of XMPP :D

But everyone who doesn't continues to use the bitrotten version :)

bitrotten version stopped working for me

We need to get some/all of them packaged in distros

It stopped working for me a long time ago, and then I later heard it was active again (and/or there is a fork with the same name)

hsendxmpp is packaged in nixos :]

(as they package everything on hackage automatically)

flow: i do

L29Ah, you may want to consider not passing the password as command line argument ✏

flow: i also do

L29Ah, yet another https://crates.io/crates/sendxmpp

so many sendxmpp's :P

For full functionality of this site it is necessary to enable JavaScript.

ah sorry https://lib.rs/crates/sendxmpp there you go

Wow, there are a lot of these. I think Martin also maintains https://salsa.debian.org/mdosch/go-sendxmpp

And I vaguely remember a Python one a while ago too (though I don't remember if it was maintained)

Yes, I once listed up those sendxmpps known to me there: https://wiki.ubuntuusers.de/Archiv/sendxmpp/#Alternativen

German, but shouldn't be an issue in that case.

Sam, that was https://github.com/moparisthebest/sendxmpp-py and supposedly it still works for some people but it quit working for me so I wrote the rust one and abandoned it

Oh yah, that's the one.

am I to late for the party? https://github.com/Flowdalic/sendxmpp/blob/master/sendxmpp

I've actually got one somewhere too (but it was just an example of using a library, not something I maintain or that's feature complete). More sendxmpps!

Maybe we should list all these :)

I have an `sendxmpp-curl` that talks to a mod_post_msg I wrote for Prosody a million years ago

> I have an `sendxmpp-curl` that talks to a mod_post_msg I wrote for Prosody a million years ago which works great by the way... i receive several dozen messages per day this way :-)

⒐

🕙

If the domainpart includes a final character considered to be a label separator (dot) by [IDNA2003] or [DNS], this character MUST be stripped from the domainpart before the JID of which it is a part is used for the purpose of routing an XML stanza, comparing against another JID, or constructing an [XMPP-URI]. In particular, the character MUST be stripped before any other canonicalization steps are taken, such as application of the [NAMEPREP] profile of [STRINGPREP] or completion of the ToASCII operation as described in [IDNA2003].

so am i compliant if all the dots right before / in any jid are stripped?

at the end of domainpart, that is

L29Ah, only the final dot

user@example.org is the same as user@example.org.

If there are multiple dots in a row I think that would just be an invalid domain label

I think so too

btw, this is not an xmpp specific thing, in fact all DNS names have a final dot, we just don't write it usually

flow: then it would strip an additional dot at each re-serialization

L29Ah, I think the jid should be rejected prior that

so exmaple.org... would become example.org.. which hopefully your DNS library would reject as invalid DNS name

say i received the jid from a remote party

but the rules what constitutes a valid domainpart are blurry in rfc 7622. I submitted an errata for that, basically you want to allow IPv4 and v6 addresses and DNS names consisting of U-labels in the domainpart ✏

so i just can't handle ⒐ as a domain name in nameprep-based xmpp

time to look for PRECIS implementations it seems

yes, I would suggest that. but ultimately the question is if ⒐ is valid in an U-label

still, rfc7622 notes the same about dots, so i'll have to strip them all

Ah yes, the `blah..` edge case. Such fun.

Perhaps ".." should be rejected early, before you strip the last "."

what about ...?

L29Ah, What about what?

matches ..?

L29Ah, (I'm joking). A domain name is an array of labels, written as a single string with the labels separated by dots, and so if you encounter a zero-length label in a domain name anywhere but the end (where there is one implicitly) then it's an error.

L29Ah, But note it's not just '.', but other separators, like (I think) '·'.

What about '·'‽

there a list somewhere™

https://github.com/MiniDNS/minidns/blob/ba0619d5404ee90096aa45c8c45c6a7ae26029b8/minidns-core/src/main/java/org/minidns/dnsname/DnsName.java#L59

Zash, so the answer is "no"

is there a normal form of domainpart?

L29Ah, no, which will probably hurt us in case of IPv6

Isn't that nameprep?

DNS has that, and something something A vs U labels. IPv4 is easy, IPv6 has a canonical form too afaik.

But you don't encounter very many IP literals in the wild. Prosody didn't even support them at all until recently.

I had in my head that they weren’t valid for JIDs for some reason.

Zash, I don't see no nameprep in rfc7622

flow: well if you look at rfc7622 then you get to play rfc recursion from https://www.rfc-editor.org/rfc/rfc7622.html#section-3.2 and on

have fun!

Zash, not sure in which direction you want me to go

can you point me to the next step after rfc7622 § 3.2 ? ✏

No, sorry, don't wanna risk summoning back the headache that's been looming over my head today.

I guess my point is that there is not much to follow, besides probably U-Label

but IIRC there is no normal form for U-labels

Aren't there a bunch of RFC links in the following sections?

I think so

but nothing I connect with domainparts (besides, you guessed it, U-labels)

Convert to A-labels (that's the pure ASCII one, right?), lowercase. Done! 😀

This implies that the string MUST NOT include A-labels as defined in [RFC5890]; each A-label MUST be converted to a U-label during preparation of a string for inclusion in a domainpart slot.

L29Ah, that's a bit misleading

XMPP domainparts should (IMHO) never contain A-labels

domainparts that form a DNS name, consists of U-labels ✏

so that this is trying to say is: if you want to make a domainpart out of something that you know is an A-label, then transform it to an U-label prior creating the domainpart

L29Ah, sorry, I somehow missed the 'NOT' in your statement

. ✏

L29Ah, FYI there is a cropus if valid/invalid JIDs at https://github.com/igniterealtime/jxmpp/tree/master/jxmpp-strings-testframework/src/main/resources/xmpp-strings/jids

but be aware that it is not impossible that some strings are in the wrong category. I am happy about feedback and/or new suggestions