XSF Discussion - 2023-03-09

what does an extend MAM metadata response look like for empty archives?

and is it different from disabled archived?

It's on my specs to-do to clarify this: https://logs.xmpp.org/jdev/2023-01-16?p=h#2023-01-16-3eedf318e3ba95f6

MUST? Prosody violates this then.

In which cases?

Empty archive

Right

It would return just <metadata/>

I think we just need to clarify "if the archive is not empty"

empty metadata sounds good to me though

I'll write that email now

or... just get back into XEP mode and write a PR

I think it should also spec that specific case too. instead of returning an error or so

one could possibly argue for item-not-found as well

not merly allow empty metadata but say thats how it's supposed to be i mean

As I noted in the above chat, I prefer empty to error, partly because <metadata> is also used in bind2

yes. and we should make it distinct from archive is broken or not enabled

Right, what does it mean for an archive to be "disabled"?

Wouldn't you just have service-unavailable then?

and no disco feature

and no <metadata> in bind2

mhhh I don’t know. server supports it but user has it disabled

broken? internal-server-error?

that would probably still have the feature?

disabled in user preferences?

internal-server-error is wrong if it's intentional (internal-server-error could imply that trying again might work)

anyway. i haven’t full thought through all the edge cases yet. but I think we already concluded that empty element is the right way to go; independently of how corner cases are handled

> archive is broken or not enabled clarify what those terms mean here?

but I was just thinking that on empty metadata will start my MAM-id tracking on the client side. and any sort of error would lead me to try again on the next login

yes i mean disabled by user preference

I think it should just be missing from disco features then

Those are per-user already

What would be missing?

https://xmpp.org/extensions/xep-0313.html#example-29

Does that mean disco changes on preferences?

Otherwise I'm not sure what the difference is (or how to signal) between "not supported so it doesn't work" or "disabled so it doesn't work"

or why that distinction is useful to have

Buuut doesn't disabled just mean no _new_ messages are added to the archive, but there still could be some that you could query for?

I don't think we defined that disabling archiving would wipe it (yet)

I don't think we want to go down that road

We could, but I don't see a use-case for it

Removing disco features when MAM is disabled is reasonable and backwards-compatible

Because claiming to support MAM when actually no MAM operations will succeed is silly

So disabled := default==never AND always:empty() AND never:empty() ?

Maybe

I don't like that options protocol though, and I'd like to retire it

in XEP-0441 terms

if disco is removed from my account, how do I know I can reenable it? Because it's enabled on the server? ✏

You re-enable it through whatever protocol you disabled it

who is "you"

the account owner

The same "my" from your message

I mean which client. How does the client know

The same way it knew how to disable it

Ok I'm missing something that seems obvious to you. The disco feature is different?

But if that way goes away if you disable it, how does it know it's still there?

What goes away?

The configuration protocol?

The disco feature

Only the MAM disco feature would go away

https://xmpp.org/extensions/xep-0441.html there doesn't seem to be a disco feature here

Oh, XEP-0441

Two things:

1) > I don't like that options protocol though, and I'd like to retire it

2) It's not clear that setting it to 'Never' and having nobody in your 'always' list is the same as actually disabling MAM entirely

Nothing has changed in that regard

But that doesn't conflict with what I'm asking :p

I don't think you can fully "disable MAM" via XEP-0441, I don't think it can/should remove the disco feature (because then yes, you lose the ability to configure MAM)

Ok

I don't think XEP-0441 should continue to exist, and I intend to remove it from Prosody in the future

So what is the "configuration protocol" you're talking about

It doesn't yet exist

Ok

Prosody doesn't allow you to disable MAM on a per-account basis, or remove the disco feature

No wonder I was configured

As far as I'm concerned, all this is hypothetical

Rather than the options XEP-0441 gives you, I'd rather switch to something that allows someone to control the retention settings, and remove support for the per-JID stuff (which has the ability to break things in unexpected ways)

We also need to add an in-band way for someone to purge their archive

Also might let you do funky things like include MUC PMs in MAM :D

MattJ, just for my understanding, mam’s before-id and after-id are inclusive (the stanza referenced as after-id is included in the results) buf rsm after/before are exclusive, no?

What?

I thought before-id and after-id was a trick to let you have both after and before in RSM

Thus same semantics

Except for the ordering implications of before

the XEP says they are (in some scenarios) interchangable

but what i'm trying to get at is that i'm suspecting they maybe aren’t

after-id: A stanza ID indicating the first message in the query results.

this means inclusive, no?

Umm, I think that is wrong

i mean they kinda have to be inclusive. because when I retrieve a metadata and then do a query for after-id=metadata.start and before-id=metadata.end I want start and end in the result

obviously

How did you come to that kind of reasoning?

but rsm because it is a paging thing needs to be exclusive or otherwise page A and page b would have overlapping entries

To fetch everything you don't specify any before/after

so the intent is that they are exclusive?

Yes

ok; suppose I’m catching up. I know my last local MAM id is `23`. I retrieve the metadata (or get it via band2) and learn the the current end is `42`. then I’m doing a query for anything between 23-42

i want the 42 to be included, no?

You do. You run a query for after-id=23

You don't include before-id

what does that metadata thing do then?

But then you may run into live messages?

The metadata thing is informational, for example it can be used to determine when there are gaps

Can it?

Since start/end are implicit in any query, you don't need to actually insert those things into filters

Due to being away for a long time, so the server expired previous overlap?

Yes

ok... I thought it could be used to put an upper bound to my catch-up query

You would use your last local id for that, no?

that’s the lower bound

So, If the <start> is *not* in your local database, then you have a gap. Probably.

Daniel is ordering the bounds by timestamp in ascending order ;)

That is *wrong*. Don't do that.

Timestamps are bad, mkay!

I think it was a comment on Daniel's use of "upper" and "lower" in this conversation (which is logical), hopefully not on database schema

ok forget upper and lower. but I thought i could limit it on both sides

yes, like MattJ said :)

You could query with after-id=your last local message, <{rsm}before/> and page backwards until you don't want to anymore

That would get you the latest new messages first

Except thanks to OMEMO, that's totally useless, innit?

my question isn’t about the order of messages

Daniel, upper bound on number of messages to retrieve? that's what I answered. query backwards from most recent. Then you can bound the result at any point by stopping.

No, that's also not what Daniel is asking for

Then I have no idea what anyone is talking about

And I probably won't understand without a whiteboard

It's just that there is no way to form a MAM query that says "get me the archive contents as they were when you sent me the <metadata>"

No inclusive id fields no

except ids

You have to have an open-ended query, and as you put it earlier, deal with the possibility that you might "run into live messages"

But you can just stop querying after you find the message with the id

The end id, I mean

or do a query based on the meta, then ids={start, end}

Yes, that's one way :)

Pity we didn't define 'ids' to be additive

That would solve this relatively cleanly

Sortable uuids!!

and it doesn't generally make sense to combine it with other filters

singpolyma, orthogonal, but nice

Yes, if designing from scratch I probably would have mandated sortable ids

https://xmpp.org/schemas/ does not appear to hold a schema that defines the http://jabber.org/protocol/compress/exi namespace, although https://xmpp.org/extensions/xep-0322.html links to http://xmpp.org/schemas/exi.xsd (and even has a byte size and hash for that file). Does anyone know where that file disappeared to?

I have a project that contains a XEP-0322-01.xsd, but that's clearly outdated.

something something xmpp protocol registry

wut?

https://github.com/joachimlindborg/XMPP-EXI/blob/master/exi.xsd seems more up to date - but I'm not sure if it is the most recent version

It's not of a size that's equal to what's listed in the XEP. :/

The contents of the file would be what is available in section 7 of the XEP.

Perhaps white-space might be different however

Thanks Peter. Sucky for hashing, but at least i've got the namespace and element names right this time ... :)

Peter Waher: am I right to assume that https://xmpp.org/extensions/xep-0322.html#defaultSchema should not be referencing 'xep-0322-01.xsd' but 'exi.xsd' instead (as that's the name used in the XSD listing)?

sorry guys I was stuck in traffic, will start the member meeting shortly

just need to update the agenda quickly

okay, anyone ready?

ack

Here I am

okay

here is our agenda for today: https://wiki.xmpp.org/web/Meeting-Minutes-2023-03-09

1) Call for Quorum

as you can see 41 members voted via memberbot. So we have a quorum

Guus: It should reference a snapshot directory with agreed-upon default versions of schemas. In the current version, the -01.xsd corresponds to the exi.xsd file. The extension takes into consideration the gradual evolution of extensions and corresponding schemas. But you’re correct: We can remove the -01 from the file name in the schemaLocation specification.

2) Items Subject to a Vote

new and returning members, you can see all the applications here: https://wiki.xmpp.org/web/Membership_Applications_Q1_2023

3) Opportunity for XSF Members to Vote in the Meeting

any XSF member here who has not voted yet and wants to do so now?

Memberbot is still online

looks like there is none

then I will shutdown memberbot and start counting

still counting ...

🥁️

👀

getting there, final check ;-)

yea🎉

4) Announcement of Voting Results

when you reload the page at: https://wiki.xmpp.org/web/Meeting-Minutes-2023-03-09 you can see the results

except of 1 candidate everyone got accepted

congrats to everyone, big list of candidates this quarter

Congratulations, I count on you when my reapplication time comes! Thank you Alex!

Great, thanks Alex!

Thank you very much @Alex and all. I am honored

Congrats to every newcomers and reappliers 😊

welcome nicola

nicola: welcome !

5) Any Other Business?

looks like thre is none

6) Formal Adjournment

I motion that we adjourn

someone needs to second

Alex: I second '-'

thanks everyone

Thank you Alex !

Thank you @Alex !!

Thanks everyone!

Thanks all