XSF Discussion - 2021-11-10

ah ok. when I want to organise I shouls subscriba aswell?

emus: No, Mentors/Admins get added automatically.

ah ok

hi jonas’ I see that https://github.com/xsf/xeps/pull/1124 is not in the council agenda, but I think it should be discussed there, shouldn't it?

goffi, the mailing list thread is rather new and we have the policy (or try to...) to first let the list discussion settle

hence I added ~"Needs List Discussion"

I also replied to your thread on list while I was at it :)

jonas’: I've replied already to your email ;)

ok, I didn't see that yet :)

jonas’: OK make sense, so I guess I'll have to wait, thanks!

goffi: I have not received your message on list duo

I haven't received the second one either, maybe there is some greylisting? It's on the archive though: https://mail.jabber.org/pipermail/standards/2021-November/038640.html

Got it now

me too

great, it's not just me who gets delayed by the list server for some reason ;)

I too received it just now

goffi, I think the issue clearly points at why we cannot retrofit that requirement into XEP-0060 and should instead quite explicitly state that no ordering can be assumed and that '413 should be used when ordering is required.

if '413 currently mandates keeping modification times and stuff, I suggest that it should be modified to make those optional and negotiated, to cover the minimum use case of having items in most-recent-first / least-recent-first order.

but I'll put that into an email, later, too

jonas’: yes please, put into an email. I'm very unconfortable with unordered pubsub items, but if we have a consensus on that, I'll update 0413. Let see how the discussion evolves

jonas’: also I'm worrying to see little adoption of 0413 before long, resulting in issues in Movim and Libervia use cases (and I'm pretty sure that most if not all pubsub implementation are already ordered, just not all at the same order). I'll put that in an email too.

goffi, yup, let's move this to the list

goffi: Is there value in supporting this things for retrieving PubSub items both with and without RSM?

I.e. can't we just say "if you want sane behavior, use RSM" (and thereby kinda discourage the PubSub-specific mechanism, and kick it should we ever come up with a PubSub 2.0)?

Holger, RSM is orthogonal to that

RSM would only order relatively to the internal order of the pubsub storage, which is undefinde as per '60

nothing says that it needs to be chronological or whatever

jonas’: I'm talking about '413. It currently extends both the' 60 mechanism and RSM.

oh, sorry :)

nvm me then, hopefully

I'm unhappy with having a generic mechanism plus specific mechanisms in at least '60 and now also '313.

For no good reason, AFAICS (and '313 is even Stable now).

why do 90% of people not understand how RSM works? :(

I'm tired of explaining this.

(and why whatever RSM does (except maybe reversal of ordering) is completely orthogonal to any selection features the underlying protocol may offer)

It's okay

Flipping pages is "selecting"?

Hack!

How's it different from the ordering thing in 0413, which is supposed to be an RSM extension, no?

Ah "except reversal of ordering".

Flipping pages is 1) not in RSM 2) not likely to be added to RSM 3) one of the most requested things "missing" from 313 (despite my disagreement)

MattJ, I'd never deny any of those 🙂

MattJ, the thing I don't get is how that's specific to MAM.

MAM is (was?) the only thing that delivers results the way it does

RSM was designed with things like disco in mind where results are a simple list embedded into a single stanza, and flipping makes zero sense

The problem with MAM is that developers wanted to render/store results as they are received, instead of after the last result set is fully received

'413 seems to manage to extend RSM despite those original use cases, no?

and then they additionally wanted the last one first

I can't comment on 413, I've neither fully reviewed it or implemented it

It didn't exist at the time, and nobody proposed it at the time, and those are the root facts :)

Whatever, it's too late for MAM. But maybe not too late for '413 🙂

To be fair, RSM explicitly has text in it saying that people don't need to request the entire result set and may just want some pages of it based on UI actions.

So I don't think the assertion that people doing that was only imagined for MAM is true.

Maybe I missed it, but I didn't see anyone make such assertions

Was that not what you meant by "The problem with MAM is that developers wanted to render/store results as they are received, instead of after the last result set is fully received"?

I mean simply that MAM breaks the results (of a single page) into multiple stanzas

and this leads to the apparent desire to control the order of results within a single page

(instead of within the full result set)

Oh, you mean people want to render as they're received *within* a result page? ✏

Yes

It does simplify the client logic quite a lot if you only have to append/prepend to your chat log, rather than needing to inject in the middle.

I dunno maybe jonas’ is right and I'm just too stupid to understand the great picture. But in my book, (1) '313 extends RSM, but makes that extension specific to MAM for no reason obvious to me. And that (2) '413 extends RSM in the same way, just as a generic, reusable solution (yay). And that (3) '413 _also_ extends the PubSub-specific way for no reason obvious to me.

I didn't look at '413 beyond its title recently, so maybe (2) and (3) are right ;)

So we end up with at least three different solutions to the exact same problem, which doesn't seem like optimal protocol design to me.

(And apart from redundant code, you also end up having to think about how to handle conflicting requests and whatnot.)

I understand it's too late to avoid the duplication in '313, so my question simply was whether (2) vs. (3) could be avoided.

Hot take: if you're "tired of explaining this" and "90% of people [do not] understand how RSM works" then that's a good indication that the current state of the world is bad and should be changed.

Also +1 to everything Holger said, I get lost and confused all over again every time I look at any of this.

Sam, it is so obvious and sensible to me though that I have absolutely no clue what to do about it.

It needs someone who both understands why the current design is sensible *and* why people fail to understand it to make something better, and I don't know if anyone is in that place.

I think we need someone who understands *if* the current design is sensible *and* why people fail to understand it, but otherwise yah

But I dispute the assumption that because one person understands it it's sensible.

it is very similar to how SQL works, fwiw, which is what I'd base the sensibility on. SQL is good at returning slices of data.

well, not exactly *very good*, but RSM improves in exactly those areas where SQL lacks (by making "LIMIT" not based on indices but on opaque stable identifiers of the records), but yeah.

tired, as I said

JFTR I think the question whether 0059 is sensible and generally well-understood is somewhat unrelated to my point. I was talking about duplication of the '413 functionality (Order-By) and jonas’ response was like "you're wrong (except maybe for the order-by thing you're talking about)" 😛

> why people fail to understand it probably due the lack of good documentation

I have a branch somewhere to improve it in '59, but I failed to find words which weren't already in there.

every improvement is good, but I was more thinking about examples and rationale provided in e.g. modernxmpp

ISTR not a misunderstanding of '59, but rather an ideological disagreement on the nature of the result set being managed.

jonas’, MattJ, just BTW, before-id and after-id might be filter criteria rather than paging tools, but I don't get how those are specific to MAM either. RSM is always operating on IDs, and other RSM users might be just as interested in those selection features, no?

it's an implementation detail of MAM that the things used by RSM are the same as the things used by MAM

Yes. Seems unrelated to my point 🙂

except for the fun wording in MAM (which, since before/after id makes no sense anymore), RSM could AES-encrypt the IDs and things should continue to work just as if it hadn't

Different topic: > servers that understand the 'include-groupchat' field and store groupchat messages in the user's archive must advertise the 'urn:xmpp:mam:2#groupchat-available' feature I'd assume a common configuration might be that servers store MIX but not MUC messages in user archives. In that case they'd probably just advertise #groupchat-available because clients expecting MUC messages won't fall apart if they receive an empty response, I guess?

Or to put it differently, what's the point of the #groupchat-available feature? Allowing the client to decide whether it's necessary to query the room archive directly?

Holger, no, for MIX that's covered by a different feature indicating MIX-PAM support I think

I see no such feature.

sadface

Holger: Roughly speaking, it's to resolve a heisenstate so clients can know whether they *might* receive gc messages or not.

So a client can say "I'm happy to receive gc" or "I must never receive gc", and the server feature needs advertising because unless the server supports it the client including such a clause is illegal.

Kev, but there's the separate #groupchat-field feature for that.

Let me try to remind myself what I did, then.

Oh, yes, you're right, this one is the one the server uses to announce that it does store gc messages in the archive.

jonas’, ah sorry there's 'urn:xmpp:mix:pam:2#archive', I failed at searching.

So for that MIX-only setup, the server would announce 'urn:xmpp:mix:pam:2#archive' but not #groupchat-available? I.e. the latter is really #muc-available?

It's possibly of limited use, but the change did remove Council objections.

#muc-or-whatever-else-you-might-do-with-type=groupchat-the-sky-is-the-limit

And for MIX there's no need for the client to explicitly ask for groupchat messages?

As-written #groupchat-available is any groupchat.

Holger, good question

We could update the text for pam#archive so that something like "If MIX messages are the only groupchat messages stored in the archive, #archive may be advertised in stead of #groupchat-available" or somesuch.

Ok from reading the specs these things aren't clear to me. I think I mentioned on the list that "MUC vs. MIX" would be good to clarify during LC 😛

Yeah that would make sense IMO.

People still think MIX in a user's MAM is a good thing?

Not me!

I do.

But from reading MIX specs I understand that's the recommended setup.

Me neither.

I think at the very least, we need good dedup mechanisms, first.

Else we'll end up like Matrix, just MUCH worse.

Agreed

maybe gap filling before dedup.

otherwise, where would the dups come from ;)

And that, yes. Again "like Matrix but worse".

it shouldn't be too hard in theory, given that we have a single source of truth regarding message order for each room :)

You say "Matrix but worse", but Matrix *does* have some nice properties.

We don't disagree :)

So, any time we solve things in a different way to Matrix, it's likely to be worse in some ways, and hopefully better in others.

But it's necessarily very complex to get those nice properties, while MIX (+ MIX messages in user MAM) is a half-baked incomplete solution to obtaining those properties in XMPP

Kev: Absolutely. We end up with their downsides (worse) while gaining little of their advantages.

Should we drop MIX and do MOX (Matrix over XMPP) instead, then? :)

It doesn't remove the centralization of channels on a particular service, and it introduces inconsistent history upon any federation hiccups

The latter of those is fairly solvable, I believe. Solving the former would be so nice (and I think you end up with a Matrix model, more or less, when you do). FMUC solves it in the cheapest possible way (or at least close) for when bandwidth is heavily limited and intermittent, but there's a significant tradeoff because of that.

Well I guess we don't necessarily have to agree / solve these things immediately as long as we have proper MAM/PAM/MIX feature announcements (and no normative text towards user archives in MIX). With the downside that MIX client devs will have to implement different solutions if they want to support both models.

quick question about https://xmpp.org/extensions/xep-0280.html#mandatory-support - should we advertise "urn:xmpp:carbons:rules:0" only if the exact set of rules is implemented (i.e. forwarding anything else on top of that set would mean no feature being advertised)?

It's intended to be exact (the following bit says the client can assume it's the exact set).

How does it help the client to be aware of the rules?

ok, thanks :-)

Holger: I'm not convinced that it does, but this was a compromise reached so we didn't hold up Carbons forever while waiting for the perfect immutable set of rules to be defined.

> A Pubsub entity supporting Result Set Management (XEP-0059) [16] SHOULD include a feature of "http://jabber.org/protocol/pubsub#rsm" in its disco#info response, to make it clear that the RSM feature is for PubSub, and not for e.g., Message Archive Management (XEP-0313) [17]. [ https://xmpp.org/extensions/xep-0060.html#subscriber-retrieve-returnsome ] This doesn't work for PEP, right ...

goffi, if a server supports e.g. '413 Order-By for PubSub, the server is supposed to announce *two* features, right? I.e.: ``` <feature var='urn:xmpp:order-by:1'/> <feature var='urn:xmpp:order-by:1@http://jabber.org/protocol/pubsub'/> ``` What's the point of announcing the first one?

Holger: yes. The first one makes it easy to check who supports order-by in any way.

Hmmm.

Who would want to check that, and why? 🙂

statistics I guess. But feel free to send your concern on standard@, I may remove it from a future version if people doesn't see any point in that.

people don't*

The idea behind my question was that this *could* be reserved for some generic '59 extension, such as omitting the `by` attribute and just reversing the default order of the result set by stating `<order-by desc='true'/>` or so. But of course this could always be done by bumping the namespace.

So, not important.

> SQL is good at returning slices of data.

GSoC has been announced, and it's quite different. Rough summary: Varying project lengths, people don't have to be students, but do have to be new (can't have existing contributors).

But it is not yet on their website, right? https://summerofcode.withgoogle.com/archive/

ah got it

https://opensource.googleblog.com/2021/11/expanding-google-summer-of-code-in-2022.html

:)

These changes all look positive to me.

😚️

TL;DR - newcomers of open source that are 18 years and older (can be student, but no need) - support both medium sized projects (~175 hours) and large projects (~350 hours) - mentors and their GSoC Contributors can decide together if they want to extend the deadline for the project up to 22 weeks (I think it has to start from June)

Kev - I understand I have to work into two directions now: - Apply as organization - Prepare the community CC: flow

I've not checked when applications open, I'm afraid, but certainly I think we're now in a position we can prepare the community.

Did not saw this yet

Mentoring organizations can begin submitting applications to Google (~2 weeks)

https://developers.google.com/open-source/gsoc/timeline

Then I would like to put this to the board agenda to finally decide if we want to attent at GSoC '22 ralphm, dwd, arc, mattj