larmaI'm kinda confused by this change: https://xmpp.org/extensions/xep-0176.html#revision-history-v1.1
> Make the 'foundation' attribute a string instead of an unsignedByte, in accordance with RFC 8445 .
XEP-0176 is meant to implement RFC 5245, XEP-0371 is meant to implement RFC 8445. Why would we adjust XEP-0176 to be in accordance with a related but incompatible RFC?
Danieli’m confused about the existence of 371
Danielcouldn’t we have just made incremental changes to 176
larmaThe RFCs are not fully compatible, so, no?
larmaI mean apparently a bunch of clients seem to do RFC 8445 over XEP 176 now
larmaBecause it was already widely used and looked similar to what they're doing I guess
Danielis there any client doing 371?
larmaDon't think so, but most WebRTC clients probably should
larmaAssuming they use libwebrtc
larmaGiven that most (if not all) WebRTC clients are incompatible with what was around before, there really was no reason to stick with the 176...
Danielto be honest for me it was a discovery thing. i didn’t even know about 371
Danielor have a good enough understanding to be able to tell the differences
Daniel(i still don’t know the difference)
Danielwithout putting words into peoples mouths I assume this may have been similiar for the two implementations that predate Conversations
KevWe’ve always struggled for review on Jingle stuff, *especially* Jingle WebRTC stuff. There’s a couple of very knowledgeable people around, but nothing like the volume of knowledge we have relevant to most XEPs.
Daniellarma, 371 used to use 5245 as well and was then upgraded to 8445
Danielwithout any problems
Danielso i'm wondering if the same can not be done for 176
Daniel(possibly with some extra disco features for example for "i support 176 + tcp candidates" or something)
larmaI implemented a client of 176 that implements RFC 5245 and it can't be connected from Conversations because libwebrtc implements RFC 8445 (it probably works the other way round because that changes who is the controller). Incompatibility to me sounds like a good reason not to update/change a Draft XEP
larmaOTH we now have a bunch of clients claiming 176 conpat in disco that are not fully compatible, so I guess the damage is already here✎
larmaOTOH we now have a bunch of clients claiming 176 conpat in disco that are not fully compatible, so I guess the damage is already here ✏
Link Mauvelarma, I originally only wanted to extend from unsignedByte to unsignedInt, as that was what libwebrtc (and all clients based on it) was doing.
Link MauveBut RFC 5245 was already requesting a string up to 32 characters (whatever that means, probably bytes).
Link Mauve“ foundation = 1*32ice-char”
Link Mauve“ ice-char = ALPHA / DIGIT / "+" / "/"”
larmaLink Mauve: I guess I was more confused about the reference to RFC 8445 than the change itself
Link MauveAh hmm, maybe I was the one who got confused, as 5245 already had the exact same requirement and we imported it as an unsigned byte instead, probably in accordance with implementations of the day.
Link Mauvelarma, AIUI the incompatibility doesn’t come from 5245 vs. 8445, but from mandatory DTLS-SRTP vs. not supported.
larmaThat's something else
Link MauveOr maybe you’re aware of some incompatibility between the two RFCs that I’m not?
larmaI'm specifically talking about ICE
larma8445 allows sending data before candidate nomination, basically making candidate nomination optional (also the reason the got rid of the aggressive nomination mode). 5245 requires nomination before sending. If the controlling side is 8445 they might never nominate a candidate, thus the 5245 side can't send.
larmaI had to discover that through wireshark ;)
Link MauveHow is that negociated in non-XMPP deployments?
larmaThey probably all use 8445 nowadays and just updated when Google dictated them to?
Link MauveAll legacy telephony systems?
larmaThey're not compatible with WebRTC anyway due to dtls-srtp
larmaSo nobody cares I guess
larmaOthers would just make their client such that it's compatible with legacy systems (e.g. by nominating a candidate pair even if not required for them)
larmaIf care is taken, you can implement ICE clients that are compatible with different versions of ICE. For example, some early drafts of ICE required sending USERNAME attribute in responses. If you aim for compatibility you can just do it, as adding more attributes than required is allowed and they simply get ignored by RFC implementations
Link MauveIs “candidate pair” the proper name of a c= line in SDP?
larmaNo, that's just the candidate
Link MauveBecause I’ve seen implementations include a dummy one.
Link MauveIn the initial offer.
larmaThe pair is a combination of local and remote candidate
larmaSafari does not send proper candidates before granting audio/video permission
larmaFor privacy reasons
Daniellarma, if there is any low hanging fruit like sending transport-info/remote candidate or someting that improves the situation for you let me know
larmaI guess we'll just make sure to be compatible with RFC 8445 as well when using 176...
larmaDaniel: you mentioned TCP, do you actually do RTP over TCP?
Daniellarma: I've disabled that because 176 doesn't support it
DanielUntil know I was under the impression that this is the only major difference
DanielAs this is what the introduction of 371 focuses on
Link MauveThat was also my impression btw, which is why I never pushed for 0371.
larmaWell ICE-UDP is a datagram transport and ICE-TCP is a stream transport but according to 166 each transport should decide to either be datagram or stream and according to 167 RTP is only allowed over datagram
Link MauveI referred to the RFC to understand some things, but didn’t compare them thoroughly.
larmaI think having ICE-UDP and ICE-TCP not in the same XEP or at least not use the same namespace makes sense. I also wonder what libwebrtc does when setting up RTP over TCP because it will need to encapsulate the RTP datagrams somehow so they can go on a stream
larma(encapsulate can be as easy as prefixing with a length)
L29Ahare XMPP addresses case-sensitive? couldn't find such info in https://tools.ietf.org/html/rfc6122
DanielThe local part is normalized
DanielAka 'fancy lower cased'
Link MauveThe fancy part means you can’t just lowercase it and do the comparison, you have to use stringprep.
Link MauveNot doing so is a vulnerability awaiting.
DanielYes if you are asking from a developers perspective definitely understand how stringprep works.
From a user perspective it's simpler to just think of lower case
ZashL29Ah, so you would look at https://tools.ietf.org/html/rfc6122#appendix-A and then look up the referenced tables in https://tools.ietf.org/html/rfc3454
ZashE.g. https://tools.ietf.org/html/rfc3454#appendix-B.2 is the long version of "lower-case"
dwdAnyone available for doing React and Stanza.js for a contracting gig?
ZashFTR I pasted the source so that in the far future, after all the http-uploaded files have expired, someone could still see what it was
MattJThis URL isn't expiring any time soon :)
L29Ahi have a string, "／"
L29Ahis this a valid XMPP address?
jonas’oh, it is
jonas’it gets normalised to `/` which is apparently a valid domain name.
jonas’I mean "valid"
L29Ahit might be a valid domain name but when parsed as a jid after normalisation it would denote an empty domain name with empty resource
ZashOh no, is this another case of "turns out a HTTP URL is a valid hostname"?
L29AhZash: no, i'm figuring out why i have a failing test in a xmpp library i maintain
jonas’L29Ah, oh no.
jonas’that is ... interesting
L29Ahthe test suite has come up with such a nice jid
L29Ahthat the library can serialise (with normalisation) but can't parse afterwards
ZashIt's invalid because `/.` is not an existing top-level domain.
L29AhZash: it might be /.local.
ZashBut _that_ isn't valid
jonas’is it not?
Zashempty host, `.local` as resource!
jonas’not if written as ／.local
jonas’normalization of the domainpart happens *after* splitting
jonas’can’t be reserialized as JID anymore though :)
jonas’L29Ah, congrats, that’s a terrible test case
jonas’might even warrant an erratum to RFC 6122, would have to check if 7622 has the same issue
L29Ah"awesome test case", you mean?
jonas’L29Ah, yes, in the original sense of the word
jonas’well, in both senses that is
jonas’L29Ah, where did you find it?
L29Ah16:01:35]<L29Ah> the test suite has come up with such a nice jid
jonas’which test suite?
L29Ahit generates random strings
jonas’I mean amazing
jonas’you know what I mean
Samooh, that's a good one. Works, but definitely seems like something we should be making illegal somehow https://play.golang.org/p/F7GRBN3EkDX
Sam(that example is using 7622, you may have to run it twice for it to cache the library and not timeout because of the download time)
jonas’Sam, thanks for the 7622 check
Kev“not if written as / .local”
Why does that space matter? It’s still split on the / so the latter part is the resource?
SamKev: that's not a space :)
jonas’Kev, it’s not a space, it’s a strange slash character
Sam(it's not a '/' either :) )
ZashU+FF0F FULLWIDTH SOLIDUS
KevAh, if it’s not / then ok :)
SamShould be fine because it will never be normalized to '/' in 7622 at least, but it *looks* confusing which is possibly bad
KevYes, “fine” might be quite subjective here :)
SamEg. if I buy "google.com／definitelynotevil.com"
Sam"Hi, I'm an important person from Google which you can verify from my JID!, I want to give you 5 million dollars!"
L29Ahok seems like i need to replace my stringprep stuff with 7622 stuff
SamL29Ah: does stringprep normalize that to '/'? Yah, that seems like a potential place for serious bugs
L29Ahyes it does
KevIs this the point where I repeat my usual complaint that precis-based JIDs are broken? :)
SamKev: in this case it appears to be the stringpep based JID that would break.
ZashDoes PRECIS fix this?
KevSam: In the sense that there’s no upgrade path from stringprep to precis, and therefore they can’t be safely used on the same network unless they’re compatible, and if they were compatible (they’re not) there’d have been no reason to go to precis.
SamIt just uses IDNA's toUnicode for domains, so yah I guess so. It seems bad that stringprep does more than that, because then you'd end up with domains that are valid but you can't use them for JIDs
SamBut I haven't really looked, so that's assuming this is all true and not a bug in one of our libraries.
SamKev: they're compatible enough in the real world. But yah, the upgrade path is tricky. If this is really true though, I'd say it's a huge reason to upgrade. stringprep not allowing valid domains in JIDs seems really bad
SamAnyways, back in a bit then I need to write some tests around this.
Kev“They’re compatible enough in the real world” is exactly the point. People make that argument, but if it were actually true there’d be no reason to need to go to precis.
L29Ahhttps://tools.ietf.org/html/rfc3491#section-4 yeah nameprep tells it gets normalized
KevWe can’t both have the “stringprep is broken, we must precis” argument and the “but it doesn’t matter because they’re compatible in every way that matters” argument both being true :)
ZashHow about we just be very conservative with new usernames and stuff until we're all on PRECIS
KevZash: With which unicode version?
L29Ahscrew that, i'll just sanitize stringprep output for @ and /, and tell that i'm done
KevAFAIK, you can’t safely mix unicode versions in XMPP (nor in JIDs), and you can’t safely mix precis and stringprep-based JIDs. But we pretend that you can, while pasting “This is fine” GIFs :)
SamKev: I disagree. They're compatible in every way that matters, but stringprep is broken in a way that while it's not likely to be seen in the real world seems bad, but not because of its incompatibility with precis, because of its incompatibility with DNS.
SamI mean, don't get me wrong, we should come up with a graceful way to upgrade and handle the differences, I'm just saying that 90% of the time it actually is fine and stringprep is apparently broken in ways that have nothing to do with its incompatibilities with precis.
KevI don’t disagree that precis might be better, I just don’t see how we can be expecting everything to work nicely without upgrade paths.
SamYah, we should come up with some of those.
SamIt's just going to be hard to get anyone to do it because it will work well enough, so check your database, if there are no differences in any JID or JID that people are sending to: go ahead and upgrade. If there are, wait for an upgrade path.
jonas’step 0: find all JIDs in a database
jonas’that’s a challenge in and of itself
moparisthebestis your database the only one that matters? doesn't it break interop with everyone else too ?
ZashI'm still thinking
Be strict when creating local resources (users, rooms, nickname registrations...)
Be relaxed with incoming data from other servers.
moparisthebestkinda like if your email server can enforce authenticated TLS only just turn it on, and enjoy your no messages from anyone from that point forward :)
Ge0rGZash: 🤖 disagrees with that.
Zash🤖️ doesn't exist, it can't hurt you
KevIt’s even wider than JIDs, actually (Unicode, rather than precis/stringprep).
L29Ahokay, is "foo@bar:666/baz" a valid jid?
KevEven with message bodies, unicode version changes can break things.
KevAre those all the characters they look like?
ZashL29Ah, maybe but I don't like it
ZashIIRC you're allowed to put port numbers in the domainpart, but I doubt it'll be reliable
moparisthebestthere's generally a wide difference between "what the spec allows" and "what you can actually use in practice"
ZashL29Ah: `[db8::123:abc]` is a legal domainpart too, and that has `:` in it
L29AhZash: oh, thanks
ZashNot completely sure about port numbers
moparisthebestI recall a poor lady named something like Mary O'Toole who's email was Mary.O'Toole@domain.org which is a perfectly valid email that most email systems refused to deliver mail to
Zashnot mentioned in 6122
ZashDon't mention email :(
KevI thought that port numbers weren’t allowed, but would have to scour specs that I don’t want to scour at teh moment.
moparisthebestpoint being, it's the same in XMPP land, you can't use PRECIS for this reason
L29Ahok so i just strip / and @ from domainpart, and declare all jids with domainpart that normalizes to a string with those invalid, what do you think?
Zash> Implementation Note: When dividing a JID into its component parts, an
> implementation needs to match the separator characters '@' and '/'
> before applying any transformation algorithms, which might decompose
> certain Unicode code points to the separator characters.
Zash... but then nothing about what to do if that actually happens 🙁
flowL29Ah, so what's the lib in question. I only know quickcheck from haskell and there I know one XMPP lib✎
flowL29Ah, so what's the lib in question? I only know quickcheck from haskell and there I know one XMPP lib ✏
flowso is pontarius alive again?
L29Ahbut i took over to keep pontarius-xmpp in shape and hopefully write a decent XMPP client someday
flowwell, you came here and ask the right questions, so I'd say you are on a good way ;)
L29Ahat least now i have a replacement for the bitrotten sendxmpp: https://github.com/l29ah/hsendxmpp
L29Ahgoogle failed me :(
ZashIt's slowly growing into the hello world of XMPP :D
MattJBut everyone who doesn't continues to use the bitrotten version :)
L29Ahbitrotten version stopped working for me
MattJWe need to get some/all of them packaged in distros
MattJIt stopped working for me a long time ago, and then I later heard it was active again (and/or there is a fork with the same name)
L29Ahhsendxmpp is packaged in nixos :]
L29Ah(as they package everything on hackage automatically)
flowL29Ah, you may want to consider passing the password as command line argument✎
L29Ahflow: i do
flowL29Ah, you may want to consider not passing the password as command line argument ✏
L29Ahflow: i also do
moparisthebestL29Ah, yet another https://crates.io/crates/sendxmpp
moparisthebestso many sendxmpp's :P
moparisthebestah sorry https://lib.rs/crates/sendxmpp there you go
SamWow, there are a lot of these. I think Martin also maintains https://salsa.debian.org/mdosch/go-sendxmpp
SamAnd I vaguely remember a Python one a while ago too (though I don't remember if it was maintained)
mdoschYes, I once listed up those sendxmpps known to me there: https://wiki.ubuntuusers.de/Archiv/sendxmpp/#Alternativen
mdoschGerman, but shouldn't be an issue in that case.
moparisthebestSam, that was https://github.com/moparisthebest/sendxmpp-py and supposedly it still works for some people but it quit working for me so I wrote the rust one and abandoned it
SamOh yah, that's the one.
flowam I to late for the party? https://github.com/Flowdalic/sendxmpp/blob/master/sendxmpp
SamI've actually got one somewhere too (but it was just an example of using a library, not something I maintain or that's feature complete). More sendxmpps!
MattJMaybe we should list all these :)
ZashI have an `sendxmpp-curl` that talks to a mod_post_msg I wrote for Prosody a million years ago
sebastian> I have an `sendxmpp-curl` that talks to a mod_post_msg I wrote for Prosody a million years ago
which works great by the way... i receive several dozen messages per day this way :-)
L29Ah If the domainpart includes a final character considered to be a label
separator (dot) by [IDNA2003] or [DNS], this character MUST be
stripped from the domainpart before the JID of which it is a part is
used for the purpose of routing an XML stanza, comparing against
another JID, or constructing an [XMPP-URI]. In particular, the
character MUST be stripped before any other canonicalization steps
are taken, such as application of the [NAMEPREP] profile of
[STRINGPREP] or completion of the ToASCII operation as described in
L29Ahso am i compliant if all the dots right before / in any jid are stripped?
L29Ahat the end of domainpart, that is
flowL29Ah, only the final dot
firstname.lastname@example.org is the same as email@example.com.
SamIf there are multiple dots in a row I think that would just be an invalid domain label
flowI think so too
flowbtw, this is not an xmpp specific thing, in fact all DNS names have a final dot, we just don't write it usually
L29Ahflow: then it would strip an additional dot at each re-serialization
flowL29Ah, I think the jid should be rejected prior that
flowso exmaple.org... would become example.org.. which hopefully your DNS library would reject as invalid DNS name
L29Ahsay i received the jid from a remote party
flowbut the rules what constitutes a valid domainpart are blury in rfc 7622. I submitted an errata for that, basically you want to allow IPv4 and v6 addresses and DNS names consisting of U-labels in the domainpart✎
flowbut the rules what constitutes a valid domainpart are blurry in rfc 7622. I submitted an errata for that, basically you want to allow IPv4 and v6 addresses and DNS names consisting of U-labels in the domainpart ✏
L29Ahso i just can't handle ⒐ as a domain name in nameprep-based xmpp
L29Ahtime to look for PRECIS implementations it seems
flowyes, I would suggest that. but ultimately the question is if ⒐ is valid in an U-label
L29Ahstill, rfc7622 notes the same about dots, so i'll have to strip them all
ZashAh yes, the `blah..` edge case. Such fun.
ZashPerhaps ".." should be rejected early, before you strip the last "."
L29Ahwhat about ...?
dwdL29Ah, What about what?
dwdL29Ah, (I'm joking). A domain name is an array of labels, written as a single string with the labels separated by dots, and so if you encounter a zero-length label in a domain name anywhere but the end (where there is one implicitly) then it's an error.
dwdL29Ah, But note it's not just '.', but other separators, like (I think) '·'.