-
larma
I'm kinda confused by this change: https://xmpp.org/extensions/xep-0176.html#revision-history-v1.1 > Make the 'foundation' attribute a string instead of an unsignedByte, in accordance with RFC 8445 [1]. XEP-0176 is meant to implement RFC 5245, XEP-0371 is meant to implement RFC 8445. Why would we adjust XEP-0176 to be in accordance with a related but incompatible RFC?
-
Daniel
i’m confused about the existence of 371
-
Daniel
couldn’t we have just made incremental changes to 176
-
larma
The RFCs are not fully compatible, so, no?
-
larma
I mean apparently a bunch of clients seem to do RFC 8445 over XEP 176 now
-
larma
Because it was already widely used and looked similar to what they're doing I guess
-
Daniel
is there any client doing 371?
-
larma
Don't think so, but most WebRTC clients probably should
-
larma
Assuming they use libwebrtc
-
larma
Given that most (if not all) WebRTC clients are incompatible with what was around before, there really was no reason to stick with the 176...
-
Daniel
to be honest for me it was a discovery thing. i didn’t even know about 371
-
Daniel
or have a good enough understanding to be able to tell the differences
-
Daniel
(i still don’t know the difference)
-
Daniel
without putting words into peoples mouths I assume this may have been similiar for the two implementations that predate Conversations
-
Kev
We’ve always struggled for review on Jingle stuff, *especially* Jingle WebRTC stuff. There’s a couple of very knowledgeable people around, but nothing like the volume of knowledge we have relevant to most XEPs.
-
Daniel
larma, 371 used to use 5245 as well and was then upgraded to 8445
-
Daniel
without any problems
-
Daniel
so i'm wondering if the same can not be done for 176
-
Daniel
(possibly with some extra disco features for example for "i support 176 + tcp candidates" or something)
-
larma
I implemented a client of 176 that implements RFC 5245 and it can't be connected from Conversations because libwebrtc implements RFC 8445 (it probably works the other way round because that changes who is the controller). Incompatibility to me sounds like a good reason not to update/change a Draft XEP
-
larma
OTH we now have a bunch of clients claiming 176 conpat in disco that are not fully compatible, so I guess the damage is already here✎ -
larma
OTOH we now have a bunch of clients claiming 176 conpat in disco that are not fully compatible, so I guess the damage is already here ✏
-
Link Mauve
larma, I originally only wanted to extend from unsignedByte to unsignedInt, as that was what libwebrtc (and all clients based on it) was doing.
-
Link Mauve
But RFC 5245 was already requesting a string up to 32 characters (whatever that means, probably bytes).
-
Link Mauve
“ foundation = 1*32ice-char”
-
Link Mauve
“ ice-char = ALPHA / DIGIT / "+" / "/"”
-
larma
Link Mauve: I guess I was more confused about the reference to RFC 8445 than the change itself
-
Link Mauve
Ah hmm, maybe I was the one who got confused, as 5245 already had the exact same requirement and we imported it as an unsigned byte instead, probably in accordance with implementations of the day.
-
Link Mauve
larma, AIUI the incompatibility doesn’t come from 5245 vs. 8445, but from mandatory DTLS-SRTP vs. not supported.
-
larma
That's something else
-
Link Mauve
Or maybe you’re aware of some incompatibility between the two RFCs that I’m not?
-
larma
I'm specifically talking about ICE
-
larma
8445 allows sending data before candidate nomination, basically making candidate nomination optional (also the reason the got rid of the aggressive nomination mode). 5245 requires nomination before sending. If the controlling side is 8445 they might never nominate a candidate, thus the 5245 side can't send.
-
larma
I had to discover that through wireshark ;)
-
Link Mauve
How is that negociated in non-XMPP deployments?
-
larma
They probably all use 8445 nowadays and just updated when Google dictated them to?
-
Link Mauve
All legacy telephony systems?
-
larma
They're not compatible with WebRTC anyway due to dtls-srtp
-
larma
So nobody cares I guess
-
larma
Others would just make their client such that it's compatible with legacy systems (e.g. by nominating a candidate pair even if not required for them)
-
larma
If care is taken, you can implement ICE clients that are compatible with different versions of ICE. For example, some early drafts of ICE required sending USERNAME attribute in responses. If you aim for compatibility you can just do it, as adding more attributes than required is allowed and they simply get ignored by RFC implementations
-
Link Mauve
Is “candidate pair” the proper name of a c= line in SDP?
-
larma
No, that's just the candidate
-
Link Mauve
Because I’ve seen implementations include a dummy one.
-
Link Mauve
In the initial offer.
-
larma
The pair is a combination of local and remote candidate
-
larma
Safari does not send proper candidates before granting audio/video permission
-
larma
For privacy reasons
-
Daniel
larma, if there is any low hanging fruit like sending transport-info/remote candidate or someting that improves the situation for you let me know
-
larma
I guess we'll just make sure to be compatible with RFC 8445 as well when using 176...
-
larma
Daniel: you mentioned TCP, do you actually do RTP over TCP?
-
Daniel
larma: I've disabled that because 176 doesn't support it
-
Daniel
Until know I was under the impression that this is the only major difference
-
Daniel
As this is what the introduction of 371 focuses on
-
Link Mauve
That was also my impression btw, which is why I never pushed for 0371.
-
larma
Well ICE-UDP is a datagram transport and ICE-TCP is a stream transport but according to 166 each transport should decide to either be datagram or stream and according to 167 RTP is only allowed over datagram
-
Link Mauve
I referred to the RFC to understand some things, but didn’t compare them thoroughly.
-
larma
I think having ICE-UDP and ICE-TCP not in the same XEP or at least not use the same namespace makes sense. I also wonder what libwebrtc does when setting up RTP over TCP because it will need to encapsulate the RTP datagrams somehow so they can go on a stream
-
larma
(encapsulate can be as easy as prefixing with a length)
-
L29Ah
are XMPP addresses case-sensitive? couldn't find such info in https://tools.ietf.org/html/rfc6122
-
Daniel
No
-
Daniel
The local part is normalized
-
Daniel
Aka 'fancy lower cased'
-
Link Mauve
The fancy part means you can’t just lowercase it and do the comparison, you have to use stringprep.
-
Link Mauve
Not doing so is a vulnerability awaiting.
-
Daniel
Yes if you are asking from a developers perspective definitely understand how stringprep works. From a user perspective it's simpler to just think of lower case
-
Zash
L29Ah, so you would look at https://tools.ietf.org/html/rfc6122#appendix-A and then look up the referenced tables in https://tools.ietf.org/html/rfc3454
-
Zash
E.g. https://tools.ietf.org/html/rfc3454#appendix-B.2 is the long version of "lower-case"
-
L29Ah
thanks!
-
dwd
Anyone available for doing React and Stanza.js for a contracting gig?
-
MattJ
Don't forget https://xmpp.work/
-
Link Mauve
sonny, ↑
-
sonny
nope pretty busy ATM
-
flow
L29Ah, check https://tools.ietf.org/html/rfc7622
-
Zash
Check this handy graph ```dot digraph "xmpp-addr" { rfc3920 -> idna2003 [label="ref"] rfc3920 -> stringprep [label="ref"] rfc3920 -> rfc6122 [label="update"] rfc6122 -> idna2003 [label="ref"] rfc6122 -> idna2008 [label="ref"] rfc6122 -> stringprep [label="ref"] rfc6122 -> rfc7622 [label="obsolete"] rfc7622 -> idna2008 [label="ref"] rfc7622 -> precis [label="ref"] idna2003 -> idna2008 [label="obsolete"] idna2003 -> stringprep [label="ref"] } ```
-
MattJ
Rendered:
-
MattJ
https://matthewwild.co.uk/uploads/xmpp-addr.png
-
eric
MattJ: what did you use for rendering?
-
MattJ
dot -Tpng
-
eric
Thanks
-
Zash
FTR I pasted the source so that in the far future, after all the http-uploaded files have expired, someone could still see what it was
-
MattJ
This URL isn't expiring any time soon :)
-
L29Ah
i have a string, "/"
-
L29Ah
is this a valid XMPP address?
-
jonas’
no
-
L29Ah
why?
-
jonas’
oh, it is
-
jonas’
it gets normalised to `/` which is apparently a valid domain name.
-
jonas’
I mean "valid"
-
L29Ah
it might be a valid domain name but when parsed as a jid after normalisation it would denote an empty domain name with empty resource
-
Zash
Oh no, is this another case of "turns out a HTTP URL is a valid hostname"?
-
L29Ah
Zash: no, i'm figuring out why i have a failing test in a xmpp library i maintain
-
jonas’
L29Ah, oh no.
-
jonas’
that is ... interesting
-
L29Ah
the test suite has come up with such a nice jid
-
L29Ah
that the library can serialise (with normalisation) but can't parse afterwards
-
Zash
It's invalid because `/.` is not an existing top-level domain.
-
L29Ah
Zash: it might be /.local.
-
Zash
Oh no
-
Zash
But _that_ isn't valid
-
jonas’
is it not?
-
Zash
empty host, `.local` as resource!
-
jonas’
no
-
jonas’
not if written as /.local
-
jonas’
normalization of the domainpart happens *after* splitting
-
Zash
True
-
jonas’
can’t be reserialized as JID anymore though :)
-
Zash
Also true
-
jonas’
L29Ah, congrats, that’s a terrible test case
-
jonas’
might even warrant an erratum to RFC 6122, would have to check if 7622 has the same issue
-
L29Ah
"awesome test case", you mean?
-
jonas’
L29Ah, yes, in the original sense of the word
-
jonas’
well, in both senses that is
-
jonas’
L29Ah, where did you find it?
-
L29Ah
16:01:35]<L29Ah> the test suite has come up with such a nice jid
-
jonas’
which test suite?
-
L29Ah
it generates random strings
-
L29Ah
quickcheck
-
jonas’
oh
-
jonas’
awful
-
jonas’
I mean amazing
-
jonas’
but awful
-
jonas’
you know what I mean
-
Sam
ooh, that's a good one. Works, but definitely seems like something we should be making illegal somehow https://play.golang.org/p/F7GRBN3EkDX
-
Sam
(that example is using 7622, you may have to run it twice for it to cache the library and not timeout because of the download time)
-
jonas’
Sam, thanks for the 7622 check
-
Kev
“not if written as / .local” Why does that space matter? It’s still split on the / so the latter part is the resource?
-
Sam
Kev: that's not a space :)
-
jonas’
Kev, it’s not a space, it’s a strange slash character
-
Sam
(it's not a '/' either :) )
-
Zash
U+FF0F FULLWIDTH SOLIDUS
-
Kev
Ah, if it’s not / then ok :)
-
Sam
Should be fine because it will never be normalized to '/' in 7622 at least, but it *looks* confusing which is possibly bad
-
Kev
Yes, “fine” might be quite subjective here :)
-
Sam
Eg. if I buy "google.com/definitelynotevil.com"
-
Sam
"Hi, I'm an important person from Google which you can verify from my JID!, I want to give you 5 million dollars!"
-
L29Ah
ok seems like i need to replace my stringprep stuff with 7622 stuff
-
Sam
L29Ah: does stringprep normalize that to '/'? Yah, that seems like a potential place for serious bugs
-
L29Ah
yes it does
-
Kev
Is this the point where I repeat my usual complaint that precis-based JIDs are broken? :)
-
Sam
Kev: in this case it appears to be the stringpep based JID that would break.
-
Zash
Does PRECIS fix this?
-
Kev
Sam: In the sense that there’s no upgrade path from stringprep to precis, and therefore they can’t be safely used on the same network unless they’re compatible, and if they were compatible (they’re not) there’d have been no reason to go to precis.
-
Sam
It just uses IDNA's toUnicode for domains, so yah I guess so. It seems bad that stringprep does more than that, because then you'd end up with domains that are valid but you can't use them for JIDs
-
Sam
But I haven't really looked, so that's assuming this is all true and not a bug in one of our libraries.
-
Sam
Kev: they're compatible enough in the real world. But yah, the upgrade path is tricky. If this is really true though, I'd say it's a huge reason to upgrade. stringprep not allowing valid domains in JIDs seems really bad
-
Sam
Anyways, back in a bit then I need to write some tests around this.
-
Kev
“They’re compatible enough in the real world” is exactly the point. People make that argument, but if it were actually true there’d be no reason to need to go to precis.
-
L29Ah
https://tools.ietf.org/html/rfc3491#section-4 yeah nameprep tells it gets normalized
-
Kev
We can’t both have the “stringprep is broken, we must precis” argument and the “but it doesn’t matter because they’re compatible in every way that matters” argument both being true :)
-
Zash
How about we just be very conservative with new usernames and stuff until we're all on PRECIS
-
Kev
Zash: With which unicode version?
-
Zash
ASCII!
-
L29Ah
screw that, i'll just sanitize stringprep output for @ and /, and tell that i'm done
-
Kev
AFAIK, you can’t safely mix unicode versions in XMPP (nor in JIDs), and you can’t safely mix precis and stringprep-based JIDs. But we pretend that you can, while pasting “This is fine” GIFs :)
-
Sam
Kev: I disagree. They're compatible in every way that matters, but stringprep is broken in a way that while it's not likely to be seen in the real world seems bad, but not because of its incompatibility with precis, because of its incompatibility with DNS.
-
Sam
I mean, don't get me wrong, we should come up with a graceful way to upgrade and handle the differences, I'm just saying that 90% of the time it actually is fine and stringprep is apparently broken in ways that have nothing to do with its incompatibilities with precis.
-
Kev
I don’t disagree that precis might be better, I just don’t see how we can be expecting everything to work nicely without upgrade paths.
-
Sam
Yah, we should come up with some of those.
-
Sam
It's just going to be hard to get anyone to do it because it will work well enough, so check your database, if there are no differences in any JID or JID that people are sending to: go ahead and upgrade. If there are, wait for an upgrade path.
-
jonas’
step 0: find all JIDs in a database
-
jonas’
that’s a challenge in and of itself
-
moparisthebest
is your database the only one that matters? doesn't it break interop with everyone else too ?
-
Zash
I'm still thinking Be strict when creating local resources (users, rooms, nickname registrations...) Be relaxed with incoming data from other servers.
-
moparisthebest
kinda like if your email server can enforce authenticated TLS only just turn it on, and enjoy your no messages from anyone from that point forward :)
-
Ge0rG
Zash: 🤖 disagrees with that.
-
Zash
🤖️ doesn't exist, it can't hurt you
-
Kev
It’s even wider than JIDs, actually (Unicode, rather than precis/stringprep).
-
L29Ah
okay, is "foo@bar:666/baz" a valid jid?
-
Kev
Even with message bodies, unicode version changes can break things.
-
Kev
Are those all the characters they look like?
-
L29Ah
:
-
Zash
L29Ah, maybe but I don't like it
-
Zash
IIRC you're allowed to put port numbers in the domainpart, but I doubt it'll be reliable
-
moparisthebest
there's generally a wide difference between "what the spec allows" and "what you can actually use in practice"
-
Zash
L29Ah: `[db8::123:abc]` is a legal domainpart too, and that has `:` in it
-
L29Ah
Zash: oh, thanks
-
Zash
Not completely sure about port numbers
-
moparisthebest
I recall a poor lady named something like Mary O'Toole who's email was Mary.O'Toole@domain.org which is a perfectly valid email that most email systems refused to deliver mail to
-
Zash
not mentioned in 6122
-
Zash
Don't mention email :(
-
Kev
I thought that port numbers weren’t allowed, but would have to scour specs that I don’t want to scour at teh moment.
-
moparisthebest
point being, it's the same in XMPP land, you can't use PRECIS for this reason
-
L29Ah
ok so i just strip / and @ from domainpart, and declare all jids with domainpart that normalizes to a string with those invalid, what do you think?
-
L29Ah
s/and/or/
-
Zash
Sounds sensible
-
Zash
> Implementation Note: When dividing a JID into its component parts, an > implementation needs to match the separator characters '@' and '/' > before applying any transformation algorithms, which might decompose > certain Unicode code points to the separator characters.
-
Zash
... but then nothing about what to do if that actually happens 🙁
-
flow
L29Ah, so what's the lib in question. I only know quickcheck from haskell and there I know one XMPP lib✎ -
flow
L29Ah, so what's the lib in question? I only know quickcheck from haskell and there I know one XMPP lib ✏
-
L29Ah
flow: pontarius-xmpp
-
flow
I figured
-
flow
so is pontarius alive again?
-
L29Ah
no
-
L29Ah
but i took over to keep pontarius-xmpp in shape and hopefully write a decent XMPP client someday
-
flow
well, you came here and ask the right questions, so I'd say you are on a good way ;)
-
L29Ah
at least now i have a replacement for the bitrotten sendxmpp: https://github.com/l29ah/hsendxmpp
-
MattJ
Everyone does
-
L29Ah
google failed me :(
-
Zash
It's slowly growing into the hello world of XMPP :D
-
MattJ
But everyone who doesn't continues to use the bitrotten version :)
-
L29Ah
bitrotten version stopped working for me
-
MattJ
We need to get some/all of them packaged in distros
-
MattJ
It stopped working for me a long time ago, and then I later heard it was active again (and/or there is a fork with the same name)
-
L29Ah
hsendxmpp is packaged in nixos :]
-
L29Ah
(as they package everything on hackage automatically)
-
flow
L29Ah, you may want to consider passing the password as command line argument✎ -
L29Ah
flow: i do
-
flow
L29Ah, you may want to consider not passing the password as command line argument ✏
-
L29Ah
flow: i also do
-
moparisthebest
L29Ah, yet another https://crates.io/crates/sendxmpp
-
moparisthebest
so many sendxmpp's :P
-
L29Ah
For full functionality of this site it is necessary to enable JavaScript.
-
moparisthebest
ah sorry https://lib.rs/crates/sendxmpp there you go
-
Sam
Wow, there are a lot of these. I think Martin also maintains https://salsa.debian.org/mdosch/go-sendxmpp
-
Sam
And I vaguely remember a Python one a while ago too (though I don't remember if it was maintained)
-
mdosch
Yes, I once listed up those sendxmpps known to me there: https://wiki.ubuntuusers.de/Archiv/sendxmpp/#Alternativen
-
mdosch
German, but shouldn't be an issue in that case.
-
moparisthebest
Sam, that was https://github.com/moparisthebest/sendxmpp-py and supposedly it still works for some people but it quit working for me so I wrote the rust one and abandoned it
-
Sam
Oh yah, that's the one.
-
flow
am I to late for the party? https://github.com/Flowdalic/sendxmpp/blob/master/sendxmpp
-
Sam
I've actually got one somewhere too (but it was just an example of using a library, not something I maintain or that's feature complete). More sendxmpps!
-
MattJ
Maybe we should list all these :)
-
Zash
I have an `sendxmpp-curl` that talks to a mod_post_msg I wrote for Prosody a million years ago
-
sebastian
> I have an `sendxmpp-curl` that talks to a mod_post_msg I wrote for Prosody a million years ago which works great by the way... i receive several dozen messages per day this way :-)
-
L29Ah
⒐
-
mdosch
🕙
-
L29Ah
If the domainpart includes a final character considered to be a label separator (dot) by [IDNA2003] or [DNS], this character MUST be stripped from the domainpart before the JID of which it is a part is used for the purpose of routing an XML stanza, comparing against another JID, or constructing an [XMPP-URI]. In particular, the character MUST be stripped before any other canonicalization steps are taken, such as application of the [NAMEPREP] profile of [STRINGPREP] or completion of the ToASCII operation as described in [IDNA2003].
-
L29Ah
so am i compliant if all the dots right before / in any jid are stripped?
-
L29Ah
at the end of domainpart, that is
-
flow
L29Ah, only the final dot
-
flow
user@example.org is the same as user@example.org.
-
Sam
If there are multiple dots in a row I think that would just be an invalid domain label
-
flow
I think so too
-
flow
btw, this is not an xmpp specific thing, in fact all DNS names have a final dot, we just don't write it usually
-
L29Ah
flow: then it would strip an additional dot at each re-serialization
-
flow
L29Ah, I think the jid should be rejected prior that
-
flow
so exmaple.org... would become example.org.. which hopefully your DNS library would reject as invalid DNS name
-
L29Ah
say i received the jid from a remote party
-
flow
but the rules what constitutes a valid domainpart are blury in rfc 7622. I submitted an errata for that, basically you want to allow IPv4 and v6 addresses and DNS names consisting of U-labels in the domainpart✎ -
flow
but the rules what constitutes a valid domainpart are blurry in rfc 7622. I submitted an errata for that, basically you want to allow IPv4 and v6 addresses and DNS names consisting of U-labels in the domainpart ✏
-
L29Ah
so i just can't handle ⒐ as a domain name in nameprep-based xmpp
-
L29Ah
time to look for PRECIS implementations it seems
-
flow
yes, I would suggest that. but ultimately the question is if ⒐ is valid in an U-label
-
L29Ah
still, rfc7622 notes the same about dots, so i'll have to strip them all
-
Zash
Ah yes, the `blah..` edge case. Such fun.
-
Zash
Perhaps ".." should be rejected early, before you strip the last "."
-
L29Ah
what about ...?
-
dwd
L29Ah, What about what?
-
Zash
matches ..?
-
dwd
L29Ah, (I'm joking). A domain name is an array of labels, written as a single string with the labels separated by dots, and so if you encounter a zero-length label in a domain name anywhere but the end (where there is one implicitly) then it's an error.
-
dwd
L29Ah, But note it's not just '.', but other separators, like (I think) '·'.
-
Zash
What about '·'‽
-
flow
there a list somewhere™
-
flow
https://github.com/MiniDNS/minidns/blob/ba0619d5404ee90096aa45c8c45c6a7ae26029b8/minidns-core/src/main/java/org/minidns/dnsname/DnsName.java#L59
-
flow
Zash, so the answer is "no"
-
L29Ah
is there a normal form of domainpart?
-
flow
L29Ah, no, which will probably hurt us in case of IPv6
-
Zash
Isn't that nameprep?
-
Zash
DNS has that, and something something A vs U labels. IPv4 is easy, IPv6 has a canonical form too afaik.
-
Zash
But you don't encounter very many IP literals in the wild. Prosody didn't even support them at all until recently.
-
Kev
I had in my head that they weren’t valid for JIDs for some reason.
-
flow
Zash, I don't see no nameprep in rfc7622
-
Zash
flow: well if you look at rfc7622 then you get to play rfc recursion from https://www.rfc-editor.org/rfc/rfc7622.html#section-3.2 and on
-
Zash
have fun!
-
flow
Zash, not sure in which direction you want me to go
-
flow
can you point me to the next step after rfc7622 § 3.2✎ -
flow
can you point me to the next step after rfc7622 § 3.2 ? ✏
-
Zash
No, sorry, don't wanna risk summoning back the headache that's been looming over my head today.
-
flow
I guess my point is that there is not much to follow, besides probably U-Label
-
flow
but IIRC there is no normal form for U-labels
-
Zash
Aren't there a bunch of RFC links in the following sections?
-
flow
I think so
-
flow
but nothing I connect with domainparts (besides, you guessed it, U-labels)
-
Zash
Convert to A-labels (that's the pure ASCII one, right?), lowercase. Done! 😀
-
L29Ah
This implies that the string MUST NOT include A-labels as defined in [RFC5890]; each A-label MUST be converted to a U-label during preparation of a string for inclusion in a domainpart slot.
-
flow
L29Ah, that's a bit misleading
-
flow
XMPP domainparts should (IMHO) never contain A-labels
-
flow
they are always U-labels✎ -
flow
domainparts that form a DNS name, consists of U-labels ✏
-
flow
so that this is trying to say is: if you want to make a domainpart out of something that you know is an A-label, then transform it to an U-label prior creating the domainpart
-
flow
L29Ah, sorry, I somehow missed the 'NOT' in your statement
-
emus
Tatsächlich finde ich jedoch einen Fork für eine Monal Android app garnicht so uninteressant 🤔️✎ -
emus
. ✏
-
flow
L29Ah, FYI there is a cropus if valid/invalid JIDs at https://github.com/igniterealtime/jxmpp/tree/master/jxmpp-strings-testframework/src/main/resources/xmpp-strings/jids
-
flow
but be aware that it is not impossible that some strings are in the wrong category. I am happy about feedback and/or new suggestions