XSF Discussion - 2022-02-22

Guus, 241277 tests? jid strings?

but yes, as moparisthebest said, IIRC jids valid in stringprep could become invalid in precis (not sure about the other way)

> Guus, 241277 tests? jid strings? Usernames, as stored in the database, which we essentially use as node-parts. I ran a script that tried to compare them to eachother after applying PRECIS, to find out if some would end up being duplicates of each other (which I think is warned for by an RFC). I didn't detect duplicates, but a small number could not be parsed at all, using PRECIS (while they do pass nodeprep).

70 out of a quarter of a million doesn't sound to bad, but it's not nil. Also, I fear that my testset might be biased towards Western i18n, so it might not be a good representation.

But there's no negotiation with remote servers as to which to use correct?

So you'll never agree on which jids are valid and everything will break everywhere?

Err... Aren't we all supposed to use PRECIS for a long time already? I was under the impression that Openfire badly needs to catch up.

> but yes, as moparisthebest said, IIRC jids valid in stringprep could become invalid in precis (not sure about the other way) There's also the other way, as stringprep is limited to some Unicode version (3?).

> Err... Aren't we all supposed to use PRECIS for a long time already? I was under the impression that Openfire badly needs to catch up. Same here with regard to ejabberd 🙂

Guus, I am always looking for new exhibits for my JID corpus at https://github.com/igniterealtime/jxmpp/tree/master/jxmpp-strings-testframework/src/main/resources/xmpp-strings/jids, care to share those 70? if not the real thing, then maybe obfucstated?

Guus, well PRECIS is out for a long time, but that doesn't mean that software implementations immediatly switch to it. There was obviously no strong incentive to upgrade, so most did not

> There's also the other way, as stringprep is limited to some Unicode version (3?). Something like that, yes.

The problem with upgrading to precis is that there's an incentive *not* to, because there's two possible situations as I see it 1) It doesn't matter which you use, because no-one's using JIDs that differ between stringprep/precis - so why bother? 2) It does matter because people are already using JIDs that differ, in which case switching from stringprep where they currently work to precis will break them. At the risk of, as usual, sounding like a broken record (as I've been saying this since the move was first proposed), not having negotiation for a breaking change is a barrier to this.

Kev, in some cases there is a third option: upgrade to PRECIS and whitelist the existing non-compatible JIDs

not sure if this could be done in e.g. Guus' case ✏

That only fully works if you have encyclopedic knowledge of JIDs on the network, though. And as long as they're stable between unicode versions, because you can't safely change unicode versions without negotiation either. It's all a bit of a mess.

I also don't see how you could negotiate stringprep/PRECIS in a federated XMPP setup

That might be fair.

... who does PRECIS nowadays?

Well, maybe we can get there one day by accepting either from s2s and enforcing the respective local norm on a domain's local JIDs?

Guus, in theory, Smack, because jxmpp always to plug in any XMPP string preperation mechanism you specify

Ge0rG: I don't think they're reversable.

can't we add a unicodeversion attribute into the <?xml?> header of all streams?

Guus, I think what Ge0rG suggests is that you be strict when it comes to accepting local(ly created) JIDs and relax when accepting JID strings from remotes

Yeah. To prevent things like terminating the incoming s2s connection when somebody named 🤖 joins a remote room

(or terminating a user's c2s, for that matter)

So.... EJabberd doesn't use PRECIS. From Kev's remarks, I take it M-Link doesn't either. Openfire surely doesn't.... Prosody? Tigase?

Nope

Because obviously anyone who does is going to have problems federating

Well, I'm glad to find out that on this, we're not desperately lagging behind. It is somewhat alarming that we have a 7 year old spec that none of the server implementations seem to follow?

is it really alarming?

Given there appears to be not much gain but, on the other hand, much to lose, by implementing it?

That said, I think it would be nice if server implementations allowed for Georg's suggestions

As a standards org, we're muddying the waters a lot by apparently publishing specs that are unusable. That's alarming.

Guus, sure, but when it comes to string and unicode stuff, you also muddying the waters by not progressing the standard and fixing issues in the released standard

it is a, how's it called, catch 22?

I mean the situation is probably similar to IDNA2003 vs IDNA2008

but I think, IDNA2008 is now getting more and more depoloyed? so it only took 14 years

Sure, but continuing to use a standard with known defects is different from telling the world that we have standardized on something that we in practice aren't using at all.

I am not sure why this is different

The one is accepting known limitations (that can be documented) with using a standard. The other is simply ignoring the standard that you should be using.

I guess people get the false impression by a fast moving IT tech world, where new technologies and programming languages are created appearantly on a daily basis, that it takes way longer for existing technologies to be replaced by their envisioned successor

oh, I have no problems in accepting that it can take a long, long time for a new standard to be adopted.

we are talking about decades not years, probably ✏

But the move from RFC 6122 to 7622 seems utterly unfeasible, from what I'm hearing here.

Guus, well it's not, you could take the first step and prepare Openfire to be strict for local JIDs and relaxed for remote JIDs, for example

(as optional feature of course, we do not want to force anybody to follow this scheme)

obviously, this feature is probably something to enable for new installations, not for existing ones

Be strict on creation of resource, relaxed with existing ones 🤷

so how long does it then take for new installations to replace all existing ones? one decade? two decades? more?

Being strict on creation doesn't mean that it's precis, of course, it means ensuring that it is both valid and consistent in both stringprep and precis.

I have very little interest in introducing a lot of complexity that very likely doesn't go away, especially if the rest of the ecosystem doesn't seem to be moving.

yes, being strict could mean multiple different things here, depends on what you want and what's best for your use case

Pick something

Also you can't just ensure that JIDs are only created that are allowed by PRECIS

Because future unicode versions may make some of your local JIDs become invalid

Guus, ok, so implemenations have little interest in moving. Is your conclusion then that RFC 7622 should never have been published? ✏

flow: from all of 12 hours of experience with this, that's what I'm leaning towards now, yes.

MattJ: relaxed with existing things!

Note that I was there when it happened, and I should've said so then.

Guus, given that it's just a "request for comments" I think a publication of how an improved version of the standard could look like, seems sensible

Also it's entirely feasible for closed systems to use PRECIS

I mean you can't really forbid anyone to sit down, identify the shortcomings of an existing specification and publish a "better" one

That's what I was wondering, MattJ.

Also, servers that aren't upgrading in decades probably have other S2S issues.

flow: I'm not trying to point fingers to others. I was trying to express that it's alarming that _we_ as a standards org apparently didn't sufficiently catch this.

again: this probably has a lot more to it than that I'm now realizing, being an expert in this for all of 5 minutes.

Guus, and I did not take it as pointing fingers at others :)

I'm putting this on stronger than is reasonable, probably.

but I was trying to show that I do not see where a standards org could do better

Haven't we been vaguely aware for a long time?

Guus, I also think that you may be falling in the trap that a standards org can somehow dictate what implementations implement ;)

Zash: my point is that if we were aware that it's not feasible to implement this in an existing system, the RFC should not have been published at all (again, I'm putting it to strong).

there's a difference between trying to dictate what to implement, and coming up with new standards that cannot be resonably be adopted in a pre-existing ecosystem.

I think the RFCs do mention manual clean-ups as part of the adoption process, but there's no talk about S2S oddities that I've noticed, for example.

While that sounds like a dealbreaker, pretty much.

RFC bis it is then! :)

hmm, I guess this is mostly controversial because it affects the representation of XMPP addresses, some of which may become invalid in newer versions of the standard. but then again, a standards org should be allowed to publish newer versions of a spec, even if it's a breaking change

Aren't you shooting yourself in the foot by doing so?

who is "you"?

the standards org? the server vendor? the server operator?

the standards org.

I don't think so, no

> I was trying to express that it's alarming that _we_ as a standards org apparently didn't sufficiently catch this. We knew, and chose to ignore it. These points were raised at the time.

You're creating a scenario that hurts the ecosystem.

Kev: that's the bit that I now find alarming. Do you recall the reasons for ignoring it. Benefits that outweigh the issue?

I didn't understand at the time, and still don't. It felt like I was in a minority of one at the time.

Meanwhile, Prosody is about to start actually enforce stringprep.... (disallow undefined characters) ✏

Ecosystem moving!

We can worry about precis in the next 5 years or so

Zash: that's exactly what I thought I was at the end of. :)

but, if Prosody isn't enforcing stringprep... doesn't it have pretty much the same S2S vulnerabilities that moving to PRECIS would introduce?

> Be strict on creation of resource, relaxed with existing ones 🤷

Existing ones includes all of s2s

Otherwise I couln't even be here because of Rixon 👁🗨

Guus, I am still be curious about those JIDs :)

flow, trying to get them for you

Guus, thanks, much appreciated

flow, in Java, getting the String bytes in UTF_8, then converting to hex, that's a safe way to represent those values as UTF-8 code units, correct?

Guus, yes, but I would not expect that you need to fall back to a binary encoding to represent JIDs

fwiw, you could also hexify the java-native char[] presentation of the String, no need to go over UTF-8

we're talking about values that may or may not be valid JIDs :)

I'm not sure if I can trust the terminal and clients I will be sending you this through to not cause issues?

yes, but unless we are talking null byte or a few other code points, I would not expect that you need to fall back to a binary encoding here. of course, I don't know those JIDs so I could be wrong

that's exactly what I'm doing.

Guus, then try both maybe, i.e., print the string as is, and the hexified representation ✏

Is there any case where one would want to store FullJID in roster?

Yes, users want to do illegal things. Don't let them!

It raises a lot of questions if you do that, I advise to stay away from such craziness :)

but is it actually illegal? RFC doesn't mandate BareJID here - right?

I don't think I've ever encountered text that explicitly forbids it, no

But you can't really subscribe to a full JID. So while you *could* add it to the roster with no subscription, I don't see what happens after that.

yeah, logically it doesn't make much sense though I was pondering if ther could be some weird one

My opinion: this stuff is hard enough without making it harder for ourselves: just disallow it :)

:D

Guys - have you seen this? Its the 22/02/2022. .. or 2022-02/22... What so ever - the perfect dat to remind you of the upcoming release of the XMPP Newsletter! 😆🎉🎉 📝 https://yopad.eu/p/xmpp-newsletter-365days

Even better in Finland or other GMT+2 timezone, where they get 2022-02-22T22:20:22+02

lol 😆

I have started memberbot for our Q1-2022 membership application period. Feel free to chat with memberbot when you are an XSF member

Impressively long list, this quarter. :)

Thanks Alex.

emus: its not only 2022/02/22, the day of week is even a "Twosday"!

😀

For once all ze Germans wiz zeir funny akzent are right!

SSH day

How about we commit to seriously looking at PRECIS as soon as IPv6 and DNSSEC are universally deployed? ✏

👍️

vanitasvitae: ☺