-
flow
Guus, 241277 tests? jid strings?
-
flow
but yes, as moparisthebest said, IIRC jids valid in stringprep could become invalid in precis (not sure about the other way)
-
Guus
> Guus, 241277 tests? jid strings? Usernames, as stored in the database, which we essentially use as node-parts. I ran a script that tried to compare them to eachother after applying PRECIS, to find out if some would end up being duplicates of each other (which I think is warned for by an RFC). I didn't detect duplicates, but a small number could not be parsed at all, using PRECIS (while they do pass nodeprep).
-
Guus
70 out of a quarter of a million doesn't sound to bad, but it's not nil. Also, I fear that my testset might be biased towards Western i18n, so it might not be a good representation.
-
moparisthebest
But there's no negotiation with remote servers as to which to use correct?
-
moparisthebest
So you'll never agree on which jids are valid and everything will break everywhere?
-
Guus
Err... Aren't we all supposed to use PRECIS for a long time already? I was under the impression that Openfire badly needs to catch up.
-
Holger
> but yes, as moparisthebest said, IIRC jids valid in stringprep could become invalid in precis (not sure about the other way) There's also the other way, as stringprep is limited to some Unicode version (3?).
-
Holger
> Err... Aren't we all supposed to use PRECIS for a long time already? I was under the impression that Openfire badly needs to catch up. Same here with regard to ejabberd 🙂
-
flow
Guus, I am always looking for new exhibits for my JID corpus at https://github.com/igniterealtime/jxmpp/tree/master/jxmpp-strings-testframework/src/main/resources/xmpp-strings/jids, care to share those 70? if not the real thing, then maybe obfucstated?
-
flow
Guus, well PRECIS is out for a long time, but that doesn't mean that software implementations immediatly switch to it. There was obviously no strong incentive to upgrade, so most did not
-
Guus
> There's also the other way, as stringprep is limited to some Unicode version (3?). Something like that, yes.
-
Kev
The problem with upgrading to precis is that there's an incentive *not* to, because there's two possible situations as I see it 1) It doesn't matter which you use, because no-one's using JIDs that differ between stringprep/precis - so why bother? 2) It does matter because people are already using JIDs that differ, in which case switching from stringprep where they currently work to precis will break them. At the risk of, as usual, sounding like a broken record (as I've been saying this since the move was first proposed), not having negotiation for a breaking change is a barrier to this.
-
flow
Kev, in some cases there is a third option: upgrade to PRECIS and whitelist the existing non-compatible JIDs
-
flow
not sure if this could be done in e.g. Guus's case✎ -
flow
not sure if this could be done in e.g. Guus' case ✏
-
Kev
That only fully works if you have encyclopedic knowledge of JIDs on the network, though. And as long as they're stable between unicode versions, because you can't safely change unicode versions without negotiation either. It's all a bit of a mess.
-
flow
I also don't see how you could negotiate stringprep/PRECIS in a federated XMPP setup
-
Kev
That might be fair.
-
Guus
... who does PRECIS nowadays?
-
Ge0rG
Well, maybe we can get there one day by accepting either from s2s and enforcing the respective local norm on a domain's local JIDs?
-
flow
Guus, in theory, Smack, because jxmpp always to plug in any XMPP string preperation mechanism you specify
-
Guus
Ge0rG: I don't think they're reversable.
-
Ge0rG
can't we add a unicodeversion attribute into the <?xml?> header of all streams?
-
flow
Guus, I think what Ge0rG suggests is that you be strict when it comes to accepting local(ly created) JIDs and relax when accepting JID strings from remotes
-
Ge0rG
Yeah. To prevent things like terminating the incoming s2s connection when somebody named 🤖 joins a remote room
-
Ge0rG
(or terminating a user's c2s, for that matter)
-
Guus
So.... EJabberd doesn't use PRECIS. From Kev's remarks, I take it M-Link doesn't either. Openfire surely doesn't.... Prosody? Tigase?
-
MattJ
Nope
-
MattJ
Because obviously anyone who does is going to have problems federating
-
Guus
Well, I'm glad to find out that on this, we're not desperately lagging behind. It is somewhat alarming that we have a 7 year old spec that none of the server implementations seem to follow?
-
flow
is it really alarming?
-
flow
Given there appears to be not much gain but, on the other hand, much to lose, by implementing it?
-
flow
That said, I think it would be nice if server implementations allowed for Georg's suggestions
-
Guus
As a standards org, we're muddying the waters a lot by apparently publishing specs that are unusable. That's alarming.
-
flow
Guus, sure, but when it comes to string and unicode stuff, you also muddying the waters by not progressing the standard and fixing issues in the released standard
-
flow
it is a, how's it called, catch 22?
-
flow
I mean the situation is probably similar to IDNA2003 vs IDNA2008
-
flow
but I think, IDNA2008 is now getting more and more depoloyed? so it only took 14 years
-
Guus
Sure, but continuing to use a standard with known defects is different from telling the world that we have standardized on something that we in practice aren't using at all.
-
flow
I am not sure why this is different
-
Guus
The one is accepting known limitations (that can be documented) with using a standard. The other is simply ignoring the standard that you should be using.
-
flow
I guess people get the false impression by a fast moving IT tech world, where new technologies and programming languages are created appearantly on a daily basis, that it takes way longer for existing technologies to be replaced by their envisioned successor
-
Guus
oh, I have no problems in accepting that it can take a long, long time for a new standard to be adopted.
-
flow
we are talking year about decades not years, probably✎ -
flow
we are talking about decades not years, probably ✏
-
Guus
But the move from RFC 6122 to 7622 seems utterly unfeasible, from what I'm hearing here.
-
flow
Guus, well it's not, you could take the first step and prepare Openfire to be strict for local JIDs and relaxed for remote JIDs, for example
-
flow
(as optional feature of course, we do not want to force anybody to follow this scheme)
-
flow
obviously, this feature is probably something to enable for new installations, not for existing ones
-
Zash
Be strict on creation of resource, relaxed with existing ones 🤷
-
flow
so how long does it then take for new installations to replace all existing ones? one decade? two decades? more?
-
Kev
Being strict on creation doesn't mean that it's precis, of course, it means ensuring that it is both valid and consistent in both stringprep and precis.
-
Guus
I have very little interest in introducing a lot of complexity that very likely doesn't go away, especially if the rest of the ecosystem doesn't seem to be moving.
-
flow
yes, being strict could mean multiple different things here, depends on what you want and what's best for your use case
-
Zash
Pick something
-
MattJ
Also you can't just ensure that JIDs are only created that are allowed by PRECIS
-
MattJ
Because future unicode versions may make some of your local JIDs become invalid
-
flow
Guus, ok, so implemenations have little interested in moving. Is your conclusion then that RFC 7622 should never have been published?✎ -
flow
Guus, ok, so implemenations have little interest in moving. Is your conclusion then that RFC 7622 should never have been published? ✏
-
Guus
flow: from all of 12 hours of experience with this, that's what I'm leaning towards now, yes.
-
Zash
MattJ: relaxed with existing things!
-
Guus
Note that I was there when it happened, and I should've said so then.
-
flow
Guus, given that it's just a "request for comments" I think a publication of how an improved version of the standard could look like, seems sensible
-
MattJ
Also it's entirely feasible for closed systems to use PRECIS
-
flow
I mean you can't really forbid anyone to sit down, identify the shortcomings of an existing specification and publish a "better" one
-
Guus
That's what I was wondering, MattJ.
-
Guus
Also, servers that aren't upgrading in decades probably have other S2S issues.
-
Guus
flow: I'm not trying to point fingers to others. I was trying to express that it's alarming that _we_ as a standards org apparently didn't sufficiently catch this.
-
Guus
again: this probably has a lot more to it than that I'm now realizing, being an expert in this for all of 5 minutes.
-
flow
Guus, and I did not take it as pointing fingers at others :)
-
Guus
I'm putting this on stronger than is reasonable, probably.
-
flow
but I was trying to show that I do not see where a standards org could do better
-
Zash
Haven't we been vaguely aware for a long time?
-
flow
Guus, I also think that you may be falling in the trap that a standards org can somehow dictate what implementations implement ;)
-
Guus
Zash: my point is that if we were aware that it's not feasible to implement this in an existing system, the RFC should not have been published at all (again, I'm putting it to strong).
-
Guus
there's a difference between trying to dictate what to implement, and coming up with new standards that cannot be resonably be adopted in a pre-existing ecosystem.
-
Guus
I think the RFCs do mention manual clean-ups as part of the adoption process, but there's no talk about S2S oddities that I've noticed, for example.
-
Guus
While that sounds like a dealbreaker, pretty much.
-
Zash
RFC bis it is then! :)
-
flow
hmm, I guess this is mostly controversial because it affects the representation of XMPP addresses, some of which may become invalid in newer versions of the standard. but then again, a standards org should be allowed to publish newer versions of a spec, even if it's a breaking change
-
Guus
Aren't you shooting yourself in the foot by doing so?
-
flow
who is "you"?
-
flow
the standards org? the server vendor? the server operator?
-
Guus
the standards org.
-
flow
I don't think so, no
-
Kev
> I was trying to express that it's alarming that _we_ as a standards org apparently didn't sufficiently catch this. We knew, and chose to ignore it. These points were raised at the time.
-
Guus
You're creating a scenario that hurts the ecosystem.
-
Guus
Kev: that's the bit that I now find alarming. Do you recall the reasons for ignoring it. Benefits that outweigh the issue?
-
Kev
I didn't understand at the time, and still don't. It felt like I was in a minority of one at the time.
-
Zash
Meanwhile, Prosody is about to start actually enforce stringprep.... (disallow undefined charactere)✎ -
Zash
Meanwhile, Prosody is about to start actually enforce stringprep.... (disallow undefined characters) ✏
-
MattJ
Ecosystem moving!
-
Zash
We can worry about precis in the next 5 years or so
-
Guus
Zash: that's exactly what I thought I was at the end of. :)
-
Guus
but, if Prosody isn't enforcing stringprep... doesn't it have pretty much the same S2S vulnerabilities that moving to PRECIS would introduce?
-
Zash
> Be strict on creation of resource, relaxed with existing ones 🤷
-
Zash
Existing ones includes all of s2s
-
Zash
Otherwise I couln't even be here because of Rixon 👁🗨
-
flow
Guus, I am still be curious about those JIDs :)
-
Guus
flow, trying to get them for you
-
flow
Guus, thanks, much appreciated
-
Guus
flow, in Java, getting the String bytes in UTF_8, then converting to hex, that's a safe way to represent those values as UTF-8 code units, correct?
-
flow
Guus, yes, but I would not expect that you need to fall back to a binary encoding to represent JIDs
-
flow
fwiw, you could also hexify the java-native char[] presentation of the String, no need to go over UTF-8
-
Guus
we're talking about values that may or may not be valid JIDs :)
-
Guus
I'm not sure if I can trust the terminal and clients I will be sending you this through to not cause issues?
-
flow
yes, but unless we are talking null byte or a few other code points, I would not expect that you need to fall back to a binary encoding here. of course, I don't know those JIDs so I could be wrong
-
flow
Guus, then try both maybe, i.e., show print the string as is, and the hexified representation✎ -
Guus
that's exactly what I'm doing.
-
flow
Guus, then try both maybe, i.e., print the string as is, and the hexified representation ✏
-
Wojtek
Is there any case where one would want to store FullJID in roster?
-
Zash
Yes, users want to do illegal things. Don't let them!
-
MattJ
It raises a lot of questions if you do that, I advise to stay away from such craziness :)
-
Wojtek
but is it actually illegal? RFC doesn't mandate BareJID here - right?
-
MattJ
I don't think I've ever encountered text that explicitly forbids it, no
-
MattJ
But you can't really subscribe to a full JID. So while you *could* add it to the roster with no subscription, I don't see what happens after that.
-
Wojtek
yeah, logically it doesn't make much sense though I was pondering if ther could be some weird one
-
MattJ
My opinion: this stuff is hard enough without making it harder for ourselves: just disallow it :)
-
Wojtek
:D
-
emus
Guys - have you seen this? Its the 22/02/2022. .. or 2022-02/22... What so ever - the perfect dat to remind you of the upcoming release of the XMPP Newsletter! 😆🎉🎉 📝 https://yopad.eu/p/xmpp-newsletter-365days
-
Zash
Even better in Finland or other GMT+2 timezone, where they get 2022-02-22T22:20:22+02
-
emus
lol 😆
-
Alex
I have started memberbot for our Q1-2022 membership application period. Feel free to chat with memberbot when you are an XSF member
-
Guus
Impressively long list, this quarter. :)
-
Guus
Thanks Alex.
-
vanitasvitae
emus: its not only 2022/02/22, the day of week is even a "Twosday"!
-
Zash
😀
-
vanitasvitae
For once all ze Germans wiz zeir funny akzent are right!
-
moparisthebest
SSH day
-
moparisthebest
How about we commit to seriously looking at PRECIS as soon an IPv6 and DNSSEC are universally deployed?✎ -
moparisthebest
How about we commit to seriously looking at PRECIS as soon as IPv6 and DNSSEC are universally deployed? ✏
-
Zash
👍️
-
emus
vanitasvitae: ☺