jdev - 2023-07-12

any reason for clients to send the "MUC join presence" twice in a row? I've noticed dino does that, is it just a bug related to https://github.com/dino/dino/issues/1457 ?

only twice? those are rookie numbers. poezio used to send 8+ or so

haha ok

Clients should only send the join once.

what should I do? send a "kick" (110, 333) before completing the sequence? just ignoring it?

(also, reporting the issue to dino, yeah ;))

As a server? Pretty sure 45 covers this, and IIRC it says that you should treat the second join as a fresh join.

ok. ignoring sounds tricky, the client may have shutdown or the connection broke and it is trying to join again. so treating it as a new join is okish, except for the part where in large groups this means sending a ton of useless presences

thanks for your replies, it makes my "lazy loading" (delaying presences after join) of participants trickier to implement than I hoped it would be, but that comes with open ecosystems I guess :)

nicoco, note that clients such as poezio have optimisations for presences received before the join.

After the join it will treat them all independently, before the join it will batch update its internal state, not sorting the participants list every presence, etc.

because it sends joins 8 times? :D

No, only once nowadays I think?

hehe I'm kidding

Note that clients may send a separate presence to set their presence in the chat.

anyway, I think it makes sense for super large groups to send the subject before the whole list of participants, especially in the contacts of presences containing vcard-temp hashes for avatars

> Note that clients may send a separate presence to set their presence in the chat. Right, but that won't have the MUC join element in it.

> anyway, I think it makes sense for super large groups to send the subject before the whole list of participants, especially in the contacts of presences containing vcard-temp hashes for avatars Except that the spec requires that you don't?

yes, don't do that

client software relies on that order to know when the join sequence is complete

Isn't it a valid optimization tho, to pretend participants aren't really there until they say something?

maaaybe, but that's differetn from "sending subject first, then presence flood"

jonas', Kev: well, it just looks like I'm the only participant, then a bunch of people join afterwards?

is that really that bad?

I thought it was an OK optimisation

nicoco, note what Link Mauve says: knowing that "this is the initial presence flood" allows for important optimizations

I'd probably keep a cache of the last active participants and send only those, then subject, then inject presence for whoever speaks that weren't in the cache.

nicoco, you can avoid re-sorting lists and stuff, which means reducing complexity from O(n² log² n) to O(n log n) ✏

in fact, participants that have spoken since slidge was started will have their presences sent before the joiner presence and subject

if you do what Zash says though and only send a "relevant" subset of presence, you obviously win

nicoco, I do not understand that sentence

slidge starts, receives messages from Bob in group A. when dino joins group A, Bob presence will be sent unoptimized (ie, before the join sequence).

but also, on the first join by a client of group A, slidge will make a legacy API call: "fetch all members of group A"

and (a) I don't want this to be blocking for the join sequence (b) I never even want to retrieve this member lists for groups that no xmpp will join

isn't that a valid case for such optimisations?

> O(n² log² n) to O(n log n) I hadn't thought of that, and maybe I should just block everything then…

Yes roster sort is a killer

> I'd probably keep a cache of the last active participants and send only those, then subject, then inject presence for whoever speaks that weren't in the cache. Zash: this is more or less what slidge does. except it's *not only* those who speak that are lazy loaded, all members are (in the limit of what the legacy API allows to retrieve, though)

nicoco, so when you say "unoptimized", it's a server-side optimization, you're talking about

yup, server-side

but I understand that it makes it harder to optimize to clients, which I had not thought about

If you spam Gajim with join presences it will simply freeze because roster sort will take all cpu

It's not expected that servers ddos you with presence after successful join

Also xeps depend on roster being complete

lovetox: from what I've experienced, in practice, it makes "joining takes a little time" to "instant join", but my desktop is beefy.

See for example the mic hats xep

nicoco: because you test it with one MUC, but clients join 30

nicoco, do you want some toaster to play with? :)

ok, I had been advised to do this optimisation in here before, but apparently it sucks then :)

Did we?

Link Mauve: I do have my android device as a toaster, just harder to test during dev with it because android ✏

Link Mauve: oh yeah, but I will not point fingers :P

the really annoying part I guess is that I want to include the vcard-temp thing in the initial presences

but I guess I should first send a list of presences without it, then for participants that have avatars send another one after (unless avatar has been retrieved, processed, and hashed before)

this is worse server-side, but if it's better client-side, I'll just ditch my recent "lazy load participant" optim, ok

nicoco, that could very much work, yes

nicoco, also, if this lazy loading of participants is only needed on the first client join and afterwards you can cache the stuff inside slidge, maybe it's bearable.

That's what people want apparently 🤷️

Just make sure that all the messages arrive out of order

tbf it doesn't have to be like that. discord or telegram large groups just don't load the participant list on join, only when you try to display it. ✏

MIX can do that, too.

but nobody likes MIX for whatever reason.

I do o/

I know :) I initially thought I'd support MIX instead of MUC for non-anon groups, but it seems very few clients will implement MIX, so I don't have a lot of incentive to do so…

the chicken-egg-issue

no servers -> no clients. no clients -> no servers.

yep. but we could do more to change that 😕

like implementing MIX in gajim|dino|... instead of NIH-ing? :-)

i would settle for accepting MIX is where we are heading and its better than MUC

for starters

then we can work on implementing everywhere 🙂 I would be interested to

define "we"

I'm not against adding support in slidge, but I just feel too amateurish to implement it "in the dark", ie, without being able to test against a client while developing

> define "we" xmpp community ↺

i know you specifically are not so against it

nicoco, I also think that a gateway component is probably not the right thing to vanguard a rollout---as useful as gateways are, they are pretty nieche compared to common MUC usage.

agreed

but at the same time, I support rich replies, reactions, etc. so…

msavoritias, thing is, agreement is just words. if nobody does it, that won't help. doing it on server and client side is what manufactures actually useful agreement. ✏

There's the matter of ROI tho, the benefits of a big rewrite don't seem worth it and would take you backwards until you catch up to feature parity with MUC.

correct

While slowly applying small fixes to MUC until it becomes close enough to MIX that we can just slap some protocol translations on it, seems more feasible.

(as a server developer who is _not_ getting paid to implement MIX (or at all))

> msavoritias, thing is, agreement is just words. if nobody does it, that won't help. doing it on server and client side is what manufactures actually useful agreement. well words are powerful and helps us change the culture around it. but fair i know i need to do stuff myself 🙂 ↺

lovetox: isn't receiving "a flood of renames" the same as getting "a flood of presences after join"?

A flood of anything after successful join is bad I guess

I have to display renames to the user

Also joins ans lefts if the user wants

But why would you send a flood of renames?

to avoid blocking while fetching participants "display names", but OK, I'll make that blocking the join. it's marginal, only on the first joining a MUC since slidge was started, and also, I'll probably cache that persistently in the future™

thanks for your replies

btw, why do you need to sort participants? isn't that only needed when the "muc roster" is shown?

UI frameworks may still do that automatically unless you tell them to hold their horses for a moment

(and "for a moment" should generally not be too long, so what you'd typically do is buffer the participants in the xmpp client layer and flush them into the list only later)

and "doing things only when showing" is how you get sluggish UIs :|

jonas: What's your real name?

Re-add me to the XMPP group and apologize

I'm not going to speak the way you want me to

And if you outright label correct things as wrong, you're gonna be sorry when you see the truth

i think this is the time to bring up the CoC :)

https://xmpp.org/extensions/xep-0458.html

plus the specific rules for operators@: https://xmpp.org/community/channels/operators/

sagaracharya, as per the operators@ rules, your ban will be lifted after a day.

we expect you follow the rules then, or stay out of the place.

and with that, I'd like to close the discussion here, because moderation of operators@ is not on-topic here.

you may discuss moderation, while following the rules, once you're allowed back in.

jonas': I see, thanks for the explanation. but then it's a matter of sluggish UI vs sluggish joining :)

I take sluggish one-time-action with a clear "something is happening here" indication over sluggish overall UI any day.

jonas: So will you apologize after a day?

After a day, we'll discuss if your action was biased or not and you will apologize

Every small deviation from your expectations is not even remotely a 'violation of rules'

I will clearly quote your provocation following which you can apologize

sagaracharya: This channel is for technical discussions, please stop.

> There's the matter of ROI tho, the benefits of a big rewrite don't seem worth it and would take you backwards until you catch up to feature parity with MUC. See: nearly 15 year old Wayland, not yet at feature parity with X ✏ ↺

Thankfully!

^

Also see https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/ re: rewrites

I was wondering whether I should search my history for that :)

This was the article I had in mind when I wrote that about ROI :)

Mix would not be a rewrite it would be a additional to muc and the trying to find good abstractions over both in your code and horribly fail

Hm can servers not do a mix - muc bridge kind of thing where I have to implement only mix

Sure, if we decided MIX was a good idea we would want that in both directions

As is mix has no backwards compatibility with muc so all channels would just be completely seperate, that 100% needs fixed before implementing it would even be feasible

Mix was spec'd for closed deployments and is only useful there at the moment

Other than being 100% not true, I agree with what you say.

I remember heated details about making details of MIX work in a way that would allow to join a MIX room from a MUC client

but then again I'm getting old, and I probably dreamed that up

Well if it was spec'd for the open federated network I would have expected some thoughts about an upgrade path for muc besides "flag day!"

The problem with MUC is, that it's not bad enough for people wanting to switch

unless you have a MUC with thousands of users

or want to work around stuff like presence above

also yes MIX can be implemented side by side.

so we dont have to do all at once

this is the kind of words I was talking about 🙂

lovetox, is https://xmpp.org/extensions/xep-0408.html what you were asking for earlier?

There's no flag day, you just have the rooms support both MUC and MIX.

yep

Side by side is the split-network-flag-day approach, single rooms that support mix and muc is the only sane upgrade path, has any of that been specified or implemented ?

thats moving the goalposts now. but I am out of this discussion anyway...' ✏

moparisthebest, XEP-0408?!

What goalposts moved? I've always said mix must have backwards compatibility or an upgrade path that isn't "ok everyone move to the mix room now" (flag day) to be usable on the public federated network (for the record, the only network I care about)

I think that's always been a part of the MIX design

Personally I find mix over complex and don't really like it on its own terms myself, but if your main issues is back compat it's not too hard

> moparisthebest, XEP-0408?! Very cool, I guess https://xmpp.org/extensions/xep-0408.html#mix-muc-invite-choice should be changed so an invite is valid for both because disco is useless and it seems likely 1 account will have multiple clients some of which only do muc and others mix for sometime ↺

What's wrong with disco?

Oh, you mean disco against the client

I'm using multiple clients only half of which are online at any time

Yes, sorry