XSF Discussion - 2022-06-09

Does MAM allow all MUC rooms (of a particular service) to be searched with one request?

No

Well, it doesn't explicitly not

E.g. you could have a (virtual?) MAM archive on the MUC domain and query that

But that's not something called out in the XEP, only querying.individual MUCs

One of the projects that I'm on has a requirement to be able to search for particular message text across rooms. I'm thinking that the approach with MAM on a service (rather than a room) could be used for that, but I'm wondering if me holding a hammer makes everything look like a nail.

I'm not sure what would be more suitable... any alternative solution I can think of would look pretty much the same

Thanks. Good to know that I'm not overlooking something obvious. :)

Checking the scope of the search regarding user permissions will be an interesting challenge

Indeed

Also those permissions may change over time, which changes the results, which isn't generally permitted for MAM queries

Not even for searches?

MAM doesn't know/care about searches - it's just another filter field

(the internet delivers: https://opensourceconnections.com/blog/2019/05/29/falsehoods-programmers-believe-about-search/ )

Persistent storage of words was a mistake

this is why I'm only storing 1's and 0's.

If you only issue "fresh" mam queries it should be fine. But if you have an after key it may do weird things depending on what you want

encrypt all relevant state into the after key.

tombstones?

trombones!

Zash: tombstones doesn't help if a new room gets added

But yeah, having the list of rooms in the key could work

Maybe, just maybe, generic MAM might not be suitable for "search all the rooms"

For search you probably just don't need an after key though? I guess if there are many results one might use it as a pagination hack

Yeah. Back to the archive protocol it is ;)

Technology evolves in cycles?

hash all the room names into a single hash, reject searches if the hash changed?

It's like the pubsub -> PEP -> add all pubsub features to PEP

hash all room state into some sort of chain of blocks?

OR we could rule all those problems to be "implementation details"

most of XMPP is built on top of "implementation details"

implementation details all the way down

On the topic of MAM, I can see how one would fetch the next 40 messages after a certain ID or time. But how would you get the previous 40 messages before a certain ID or time? Using the flipped pages feature?

`<before>a certain ID</before>`

Yeah that's just '59.

Tobias, flipped pages doesn't change the results that are returned

It just changes the order that the server sends you the results (which is either the most critical or useless feature, depending on whom you ask)

Right. But before and limiting results to 40 could also return the oldest 40 entries, not?

Flipped pages does not do that, it just flips the page that gets returned

Ok

Tobias, no, can't return the oldest 50.

Tobias, it returns the "page" immediately preceeding the specified ID.

To get the oldest 50 messages, you just make a simple request with no filters except max=50

Ok. Thanks

and `<before/>`

otherwise you get the _first_ 50 messages

That's the same as the oldest 50 messages, which is what I thought the question was

argh

my brain

I should update https://matthewwild.co.uk/uploads/mam-explorer/ for the latest version of the XEP

Verily

Right. That also works if I use times via the end field and limiting the results with max?

Yes

How do you `<before/>` in that?

"Page before (id)"

No I mean literally `<before/>`

I basically want page before date because server does not do before yet

Seems you can't set it to the magic empty element thing meaning "get me the last page"

Maybe it's time to introduce ASCII schemas in XEPs

Zash, ah, no

In fact, this thing doesn't behave correctly wrt before

Indeed, I was hoping nobody would notice

It needs some love

Don't we all ?

Yes please

and chocolate.

and coffee?

So to limit my end query, i have to use start?

darn, that's a better one.

Tobias, limit in what way?

What are you trying to query for?

the 30 messages before some time

But the server doesn't support RSM?

it supports RSM, but just no before/after

So it doesn't support RSM?

Heh

Is there much more to RSM? Just the max/count?

You need https://xmpp.org/extensions/xep-0313.html#sect-idm45497150480480

ah..ta...maybe that <before> will work

Otherwise you'll get the first (oldest) matching results, but if you add <before/> you get the most recent

You don't need the form fields

ta

will try that :)

So the server does support <before>id</before>?

If so, you should be paging using that instead, otherwise you may get inaccurate results

e.g. if you had >50 messages with the same timestamp

https://cerdale.zash.se/s/uM73sc1wTK4eDxCB8TgpWEk4/mam-explorer.patch

MattJ, suggestion ^

Applied locally, thanks

Was just looking to see if there was any low-hanging fruit

elseif query["before"] and (have_seen[query["before"]] or msg.id == query["before"]) then

Looks like I (mistakenly?) did that on purpose

Oh wait, selected = false, what

Hmmm... ```html <input id="mam-page-param-id" list="mam-ids"> <datalist id="mam-ids"> <option>a</option> <option>b</option> etc... </datalist> ```

inb4 accidentally implements MAM querying in CSS selectors

Does '59 spell out that only one of `<after>` and `<before>` is allowed at the same time?

No, unfortunately

And mam-explorer allows it! Unacceptable!

:)

So is using the after form field the same as using the after field in the <set> node of RSM in the query?

Generally, yes. My advice is to ignore the form fields (which are only in the latest MAM version). Most of the time you don't need them, and you should just use RSM for paging.

The form fields are meant for when you need to select a range of messages between two ids

Which some clients need for particular sync strategies

Meaning the before-id and after-id fields?

Yes

(as opposed to the whole form)

In case anyone cares, https://matthewwild.co.uk/uploads/mam-explorer/ is now more correct in its results (thanks to some help from Zash)

Question is, will we have a board meeting today?

So many possible witty responses, but I'll stick to: "Hopefully"

I just saw your email, and saying we haven't had one for a couple of weeks is an understatement I think. Not sure when the last meeting was, but it's been a while I think.

Yea.

Well hopefully we can at least get quorum this morning because right now..

And yet, it fails to move.

who else is board? ralphm ?

We haven't seen him in weeks

I though we agreed we'll communicate via email

what't the best way to add work to the Board's agenda?

Email the board list

No, that's not a good way, mail from non-members gets stuck in a queue

Ge0rG: what's the work?

MattJ: I'd like to get going an analysis of the EU's new Digital Markets Act and Digital Services Act, and what they mean for XMPP service operators

Isn't that the "XMPP servers are illegal" law?

Only if they're big ;)

So we're all required by law to run Snikket ?

Only people trapped in the EU I think

jcbrand: wasn't an agreement, just a proposal. It's never been discussed, and I think this is a fairly clear reason why we shouldn't

What is a fairly clear reason?

> No, that's not a good way, mail from non-members gets stuck in a queue Ge0rG is a member and we have received other email from a member recently that was discussed. I think it's better than this chat.

Sorry, I mean a member of the mailing list, not a member of the XSF

But we received an email from someone not on the board, we're they a board member before?

For historical reasons, board@ is a private list (if we could go back in time, it would probably be more open, and we would have a board-private@ for private discussions)

It's current board members + council chair (+ Peter?)

It's also possible that the mail you're thinking of was manually approved

But that queue is not routinely checked, I think Peter just does it now and again

The official contact address for the XSF is info@

Ok, can we direct people to that email address in order to contact the board?

Sure, it goes to Ralph, me and Peter, and we can bounce it to board@ if needed (a lot of noise goes there)

I'm not sure the EU law only bans "big" XMPP servers, it's "users" right? and if you have federation enabled, technically every XMPP account with federation could be counted as a "user" ?

"technically" / "could"

so I think you have to assume it bans XMPP, unless you are willing to go to court over it

Non-lawyers speculating much?

The law isn't technical, and nobody is going to consider XMPP as a single entity with N(all users across all servers) users

moparisthebest, why do you think it "bans XMPP"?

this is the one that requires any service provider to scan all messages and report anything bad to the govt right? the "mandatory scanning+reporting" law?

No, that's something different

and that definitely doesn't ban XMPP

It doesn't ban anything, and nobody really has to do anything by default. To get things started, they have to issue a specific provider with a "detection order", and that's when you have to comply with their scanning requirements.

seems to me like it does but who knows, only someone willing to take it to court I assume

> so I think you have to assume it bans XMPP, unless you are willing to go to court over it Usually laws work the opposite of that :) assume you're fine until a judge rules on a similar enough situation

only if you are willing to risk being the one being ruled on

I'm just going to assume that the EU makes things difficult for US megacorporations and carry on with my life.

You risk being rules on for some law you didn't know about or didn't understand just by existing

Being afraid of hypothetical legal situations is far too paralyzing to be practical

That's what software patents is for

Not laws 😛

They are indeed targeting the US megacorps, no doubt about that (especially trying to block Facebook Messenger E2EE)

Zash: well, EU is also well known to making laws to kill community projects, like GDPR, but those are known situations not hypotheticals like this

Doesn't mean they'll never extend it beyond that (once they have the tool, they'll use it), but I don't think we need to make drama like "XMPP is banned" when it's nowhere near true

Community projects? Thought you had to be a company of a certain size for it to really apply.

Zash: not for the deletion requests stuff for example

Which has killed many small projects

I've heard of zero such events.

Small search crawlers, the entire OpenPGP keyserver presence in Europe, etc

I agree they think they are narrowly targetting the US megacorps but in practice are just dragnet affecting everything

I mean to them XMPP as a concept doesn't even exist

its meeting time, lets start in 2 minutes

That time of the quarter again

here is our Agenda for today: https://wiki.xmpp.org/web/Meeting-Minutes-2022-06-09

1) Call for Quorum

as you can see 35 members voted via proxy, so we have a quorum

2) Items Subject to a Vote

New and Returning members., you can see the appliction page here: https://wiki.xmpp.org/web/Membership_Applications_Q2_2022

3) Opportunity for XSF Members to Vote in the Meeting

anyone here who was not voted yet and wants to do so now? Memberbot is still online for accepting your votes

looks like nobody wants to vote in the meeting. Then I will shutdown the bot and start working on the results

4) Announcement of Voting Results

🥁️

when you reload the page you can see the results here: https://wiki.xmpp.org/web/Meeting-Minutes-2022-06-09#Announcement_of_Voting_Results

all reappliers are accepted. congrats to everyone

Congrats to all

5) Any Other Business?

looks like there is none

6) Formal Adjournment

I motion that we adjourn

2nd

thanks everone

As always, thank you Alex!

I will work on sending out the minute sand updating websites and lists on the minutes over the weekend. Because I am travelling tomorrow, so be patient ;-)

Thanks Alex!

thanks Alex !

Danke Alex

Thanks Alex, and congratulations all :)

> but in practice are just dragnet affecting everything FWIW this is completely the opposite of what they are actually doing. As I said, they have to serve a specific organization with a specific order, that's not what a dragnet or blanket ban is.

An XMPP operator could totally be subject to such an order, but XMPP as a protocol cannot be

and this is all unrelated to the Digital Markets Act, which was the original question from Ge0rG. Which I'm also pretty sure has no impact on any current community XMPP operator (so I'm not sure if there was a specific concern that I might have missed?)

we might be talking about different bills

this one https://european-pirateparty.eu/parliament-approves-chatcontrol/

That's the one

Understand that certain people who oppose the regulations also put a certain spin on it. Practically nothing you read on the topic is neutral. Even the legislation is hand-wavy in parts (e.g. to avoid everyone cooking up their own tech, and/or using that as an excuse, they are going to provide the scanning tech to anyone who is subject to a detection order - but this tech hasn't been put together yet)

Yep same as the myth of GDPR killing small businesses for some reason

Last time I heard about those regulations it was mentioned that there is no small operator exemption

Yeah, PGP keyservers were dead long before GDPR, that much I'm certain of :)

Ge0rG, DMA or "Chat Control"?

MattJ: not sure if DMA or DSA

Chat Control is another horror

I haven't done as much research into the DSA yet, I guess that's next on my list then

It would be good to have some analysis and maybe at least a non authoritative statement on the blog

Normally I'd volunteer for this kind of boring legal work, but I'm way too much overloaded

Can you volunteer someone else who's qualified?

> Yeah, PGP keyservers were dead long before GDPR, that much I'm certain of :) They're still not dead yet, but most of the small ones in EU are. Plenty out here I still use regularly

I thought the system got flooded by garbage signatures and collapsed on itself

. ✏

Reject key servers, embrace DANE! https://datatracker.ietf.org/doc/html/rfc7929

Yeah, gpg has not pulled signatures from keyservers by default for a long time due to flooding issues

WKD is actually usable for most people, unlike DANE :)

but WEB 😭️

also, defaults

DANE-over-HTTPS

only acceptable if it's HTTPS-over-XMPP

Pretty sure DANE works fine for basically everyone? Browsers don't support it, but that's not relevant in this context

Tell that to anyone with a .im domain

Oh, you mean from that side. Sure. Don't buy those ;)

Most TLDs are fine

snikket.chat in dnskey when? 😉

MattJ tells me soon ish ;)