XSF Discussion - 2019-08-29

Ge0rG: how so?

ralphm: it allows the server operator to derive the user's screen time in the xmpp client

That has almost nothing to do with the GDPR.

And if it has, there's legitimate interest for processing this information, namely for ensuring a particular feature. Actively using it to track screen time, rather than temporarily stalling traffic, however, would require permission.

Finally, servers don't have control over why/when clients send CSI hints, and the specification leaves that entirely up to clients.

ralphm: so "using it to track screen time" is the actual issue, yes. Also it's got potential implications in enterprise setups, where unions are very keen on not allowing anybody to measure the employees' activity times

I understand completely, but that's not what CSI's purpose is. If you continue in that direction, there are many activities one could theoretically track to do that.

Now, if there was a dedicated protocol to explicitly relay user activity stats, that'd be different.

Yeah, but data collection is not solely about the original purpose of the data

There is difference in processing for legitimate purposes, processing it for analytics, combining it with other data, etc.

And then if you store it, there are also concerns around retention and deletability.

I personally think that the issue as you describe it is not a GDPR issue, unless the server is processing the information for more than the purpose of minimizing battery use.

This is getting similar to the discussion we had about the clarification of stored message content

ralphm: I agree with that

Well yeah, there, if you agreed that the server keeps an archive for you, then that's fine.

And public channels have a clear way of finding this out before joining.

ralphm: the question was whether the content is PII or even special category PII

PII is an old term not valid for GDPR discussions. There, we have Personal Information, and that's broader than the US term.

and article 9.1 stuff

Personal Data, I mean

ralphm: oh, I wasn't aware those are different things

Ge0rG: oh yeah

Even if you, for example, have a unique identifier for a person, linked to some data, but you don't know who that is, that's still considered Personal Data.

So often, just anonimising might not be sufficient.

ralphm: the question was whether the content is Personal Information or even Special Categories of Personal Information

Well, I think in this case, since the purpose of the protocol is not at all about knowing if somebody's screen is on, you don't get to the special categories. Especially since the server cannot know in which cases the client is going to send it.

In principle CSI just says: hold that thought until I say when.

ralphm: but you know the client implementation from disco#info, and you can derive those things

yes, but that is being actively evil

purposes and such

Right

you *can* do it, but as long as you don’t, you’re fine

E.g. not storing / logging it would be a good start

In any case of doubt, by the way, talk to lawyers.

-ENOLAWYER

lawyers are expensive

ralphm: lawyers are there to _increase_ your doubt

jonas’, Ge0rG seems concerned about corporations, they (should) have lawyers as well as data protection officers.

Usually it is about three things: purpose, consent, and retention.

I've learned that engineers, including myself, generally don't have sufficient legal knowledge to make proper assertions in this area.

But I learned quite a bit over the last few years.

ralphm, if that was true, the result would effectively be that we have to shut down all operations (free jabber servers and possibly even software development) right now.

except maybe daniel, because he might be able to afford a lawyer

While being an engineer as well, I've been doing a large chunk of GDPR consulting in the last two years or so

jonas’: there are totally things an engineer can do: log as little as possible, or at least not continuously of you're not debugging an issue, only process information for the stated purpose, give control over retention, inform about what is stored/processed.

Ge0rG: I'm not saying you have to be a lawyer. I said 'generally' and 'when in doubt'.

ralphm: right

hooking into the silence: I was thinking that we might want to set up a mentoring process for first-time XEP authors

and I’d volunteer right away to be a mentor to those

my main issue is with how to present this

and where

i.e. how to make people make use of that?

How that would be? would it be a document with guidelines and so? (It is a very good idea by the way)

Well, getting in contact with an Editor before/during submission seems like a good first step in general.

jonas’: I'd suggest adding a paragraph for prospective authors at https://xmpp.org/extensions/

maybe linking to a wiki page that you maintain, listing best practices and mentors

XEPs 0143 and 0134 should be good starting points to highlight, as is XEP-0001.

none of those is a good starting point

sorry, but no

Why?

they describe processes from the inside, not from the outside

I think?

something like that

there is a lot of information in there, but a lot of that also does not matter for when you’re first writing a XEP

That doesn't make sense to me.

How does it not matter?

I’m trying to formulate what I mean

I mean, those documents are nice and all to get your document in a reasonable state, but I don’t think they deal well with the experience a newcomer has or might have when submitting a XEP while not having had a lot to do with the community before

Aixo es el que vull veure

Vaulor: English, please.

ralphm, the mentorship things would be about the non-technical parts of the stuff, really. What interactions to expect on the mailinglist? How to properly process feedback? What to do with a rejection by council?

none of that is, to my knowledge, clear from the XEPs you mentioned

Vaulor, hah, wrong tab, right? :D

jonas’: right, I think that the suggestion by Ge0rG is a great one, except a proper page on the website itself would be nice. Noting some initial tips, suggestion to get mentorship with Editors, and then reference those XEPs as formal procedures and detailed guidance.

Yup, sorry wrong tab indeed

a "proper" page is fine with me

Vaulor: 🙃

jonas’: in any case: yay, go for it

context: the Message Reactions XEP, I think the silence from the authors speaks for itself. I tried to reach out to them yesterday, no reply yet though.

jonas’, what part about the process is unclear wrt to the reactions xep?

Daniel: it is entirely unclear to me why it hasn't been accepted (yet?)

Is your question still open after reading this, Daniel? > 07:33:30 jonas’> ralphm, the mentorship things would be about the non-technical parts of the stuff, really. What interactions to expect on the mailinglist? How to properly process feedback? What to do with a rejection by council?

right, I don’t think the rejection was made clear and public in the thread even.

that’s something council should probably work on

(or maybe the editor)

> Daniel: it is entirely unclear to me why it hasn't been accepted (yet?) yes. i kinda understand that. but that what raises the question on how 'mentorship of authors' can help with that

jonas’, i'm personally not really under the impression that the authors (of that particular xep) need help with any of these points

Daniel: so was it indeed rejected? Where can I see this? I tried looking at the recent vote tallies and yesterday's meeting logs

the council on reactions was one month ago, I think?

Why was it on the agenda (item 5) for yesterday's meeting then?

wasn’t that retractions?

Oh wait that agenda was an older one. Sorry

actually that was exactly one month ago

I see now that Kev vetoed it

don't confuse reactions, retractions and references with each other.

I found the minutes of that meeting: https://mail.jabber.org/pipermail/standards/2019-August/036345.html

this is where "post-factum mail tagging" could come in useful

ralphm: I followed that up with a long mail to the list with an offer to do the needful myself to remove my veto. Although that time has probably now passed as I had the cycles then and don't now.

Well, maybe having explicit, separate, rejection notifications would help here. I thought we had that done before, or suggested, but I missed this.

Which is about as good as Council can do, I think.

re XEP-0353: 15:17:20 dwd> I'm +1 for this. Seems widely deployed and sensible. Daniel claims otherwise on-list. Fight!

Right, I am looking for that mail

Oh, crap, it was buried in the thread of the initial submission. Sorry, Kev

A pitty that we only require two implementations before a xep can advance only for 'final'

flow: because we have that problem a lot?

so reactions passed?

Daniel: no

ralphm: the problem that xeps advance without much implementation experience nor feedback? I think so

flow: we were talking about accepting XEPs in the the first place.

ralphm, I was responding to jonas' comment regarding xep353

Which moved to Draft, right?

Daniel, vetoed on list by Kev

and regarding accepting XEPs: I really think we are missing a IETF I-D equivalent. Where people can just create a stable home for their ProtoXEP to work on

I don't see why that's a problem. I have concerns about XEP-0353, but it is not set in stone.

flow: we do, it is called proto xep

And also

ralphm, I'd argue we do not

ralphm, you can’t really edit a protoxep tho?

or can I? maybe i should try

Daniel, it's not really specified or at least promoted that you can

Generally council should accept those. I understand that Kev tried to prevent chaos by not accepting Reactions at this stage, but I am not sure I agree.

I guess you could PR against XEPs in inbox

But if this is encouraged then we should promote it more

Otherwhise establish something like xmpp.org/extensions/protoxeps/my-kewl-new-xep.html

where authors could PR against without much review

because I actually wanted to edit the aesgcm uri scheme one

so let me try that and see what happens

go for it :)

If a XEP is not accepted, an author is encouraged to address the feedback. I don't see how you can have it reevaluted without submitting a new version to the inbox.

Re-evaluated

But again, I think in general most of this should happen in experimental.

ralphm: I note that Kev didn't really 'not accept' in practical terms, given that he was ready to make the changes right away so it could be resubmitted.

I believe the concern with reactions specifically is that seems undesirable to have this end up in production clients in its current state

the expected quality for an experimental xep varies between councils and moods

What if I want to develop a protoxep without submitting it right away? Could I simply PR it into inbox?

Kev: I appreciate that of course, but the end result now is limbo?

Yes, I wasn't anticipating the authors not taking me up on the offer.

> I believe the concern with reactions specifically is that seems undesirable to have this end up in production clients in its current state I'm not sure that blocking it from becoming experimental will help that

flow: eh, that's a good question

i'd implement reactions now if it weren’t for other problems in Conversations

Daniel, it probably does not prevent it, but then again, what's the gain for having it accepted if it does not prevent it?

Kev: I don't notice that offer, so maybe they haven't either? Maybe we should make non-acceptance notifications explicitly a new thread.

Didn't

Daniel: even given the concerns?

And why not raise the bar a little bit for experimental to not accept XEPs with issues *and* a solution?

flow, that is an interesting question. probably stable home + discoveribility

Daniel, well you could argue that stable home is already extensions/inbox/reactions.xml

err .html

And I'd say that it is a feature that it is, in its current state, not easily discoverable

We've had tried discussion before, I think Experimental is that state, and adding another one just moves names around.

ralphm, what if i don’t share the concerns? (to be clear we are talking very hypothetical here because i'm not going to implement it anyway any time soon)

but i can see the point of 'my users want this now and it is good enough for me and i don’t want to wait for the very complex problem of archive message collapsing to be solved beforehand'

I agree with Kev that there are issues, I don't like that we now have a proposal with issues that is not Expirimental and now people can't, you know, experiment. I also think it would be bad to have in production clients with the expectance of large changes, but YOLO.

but if experimental is that state than reactions shouldn’t have been blocked?

^

> I also think it would be bad to have in production clients with the expectance of large changes I agree but i don’t think the xsf should try to prevent that

because it won’t be able to anyway

and the attempt has other problems

also 'experimental' comes with the warning of 'do not use this in production'

My concerns: 1) Unclear what ids should be used, and if that works properly in all cases, particularly in MUCs. Document doesn't state it, and the single example seems to use the original stanza id. 2) Interaction with MAM. There are various situations where you can retrieve message from the archive without having the referred message. 3) There should likely be a summary/counts feature. 4) I'm not sure if I like the way updates are done.

I still wonder why nobody hasn't simply wrote a quick "Reference Other Stanza" XEP, doesn't appear to hard. And would probably kickoff the discussion and steer it hopefully into the right direction

Daniel: I agree that the XSF shouldn't block stuff implemented in clients when in experimental. I just think it is not a great idea. And I also think that we may need to move a bunch of things from Experimental to Draft.

flow, that’s kind of 367 though?

(except 367 is not enough for reactions)

mathieui, possibly, but I think it is broken

mathieui, why is it not enough for reactions?

ralphm, https://dino.im/xeps/reactions.html#business-id?

indeed, XEP-0367 would need some work, and maybe a rename, but this was discussed here before as a possible basis.

regarding (1)

Daniel: is this a progression on the not-accepted specification

?

ralphm, oh tbh i didn’t check

Well, I don't think it is wise to discuss alternative documents that aren't somehow covered by our IPR

policy

i wasn’t aware that the documents didn’t match

Another argument to establish an I-D equivalent for XEPs

that link was just the first one to shop up in my browser history

flow, no support for updating or removing an element

Daniel, the documents are strictly identical as far as I can tell

See, I was really thinking that what I was doing was the pragmatic thing to do that a) Got the XEP published ASARP b) Got us to a state where further XEPs could do consistent things c) It wouldn't be obviously harmful to have in production clients.

Daniel: they don't seem to be different. And what I meant is that it is a bit vague even with that text. More examples would help.

I wasn't expecting it to end up with the protoXEP being abandoned instead.

I thought the outcome would be that the XEP got accepted a fortnight later, in a much better state.

ralphm, sure. there are things that can and should be improved.

Without insight into the author's intentions, I'm not sure if it has already been abandoned. Maybe vacation got in the way.

on the other hand we also have experimental XEPs that are literally 60% TODOs

Kev: in that case you should have just moved on with your offer to make a PR

Having at least a response on Kev's extensive notes would help.

Kev waiting for a response seems reasonable.

Ge0rG: Hindsight is a harsh mistress.

flow: IIRC, sending a PR against inbox will mean that the editors trigger a resubmission.

Kev: indeed.

Kev > Which is why I offered to the authors to fix this now so the XEP could be accepted, as I had some spare cycles at the time, but that offer wasn’t taken up. I very much want a reactions XEP published, and was willing to put the time in to make it happen. How would that fix have looked like?

using attachments?

(or just link me to the 'offer')

Kev: do you foresee a clearing in your time? I'd happily collaborate on this.

If I got the authors right, they talked to Sam about 0367 and got told that it's not acceptable for Reactions.

Maybe they should instead come here or on list.

Including Sam

ralphm: maybe this is one of the non-obvious things for new authors that jonas’ was talking about earlier

yeah

and I don’t think all of those non-obvious things can be written down

but someone who has more knowledge of the internal workings of the XSF and the process will be quickly able to answer it

and it’d be good to have a person assigned to each newbie-proto-xep who’ll deal with that type of questions

I like Sam a lot, but I don't think he's been aware of all the discussions that were had on this topic (and all other things concerning things 'attached' to messages).

Also, Message Attaching, References, etc. all need work.

I think the easiest thing if Sam doesn't want Attaching to be used for this might be to either ask Sam to let someone else take over Attaching, or to submit a ~= identical XEP intending to be used more generally, and retire Attaching.

ralphm: what do you see missing from Message Attaching?

Kev: let's hope nobody rejects that ~= identical XEP on the grounds of it being duplication of Attaching, then.

Ge0rG: particularly around how we all store and retrieve these things from archives

Kev: I think that asking Sam would be the polite route, while Council could easily take over Attaching, assign it to anybody standing up for it and let it be the right vehicle.

Maybe gather everyone at a Summit and reinvent all these message relation things from first principles? 🙂

I deliberately said 'ask', not 'reassign', yes.

ralphm: this is a hugely complex topic that's absolutely deserving its own XEP

Ge0rG: yes, but if we don't start to address it early on, this might be a lot of pain

ralphm: we are already half-in into that pain.

Ge0rG: I don't think we've barely scratched the surface.

ralphm: we have north of four different ways to reference one message from another.

Zash: I've been toying with the idea of having a spec-writing-sprint on this indeed. The Summit is too far out.

maybe the Reactions authors are actually right. We won't get rid of those legacy ways, and having four or five different legacy ID lookups in the server isn't such a big piece of work, compared to the other challenges.

(I mean: the difference between having four vs five)

well, isn't the text about ids in reactions not a copy-paste from XEP-0353?

This alone is reason to think that one should use the other

ITYM 0367

well if we had (or can create in a timely manner) a general id referencing mechanism that we know for sure to be a one size fits all solution we can use that

I meant 0367 of course

I talked with several people about Reactions XEP as well as general purpose referencing. I personally think we are running in completely wrong direction with that right now (including the blocking of ProtoXEPs, but also with general purpose referencing in general), but as announced on the list, I am happy to do any changes to Reactions XEP or other XEPs needed to get it accepted if there was council advice what is the best/acceptable way to do it. Last status I am aware of is that there are at least two ways to do it and there is no consensus which is the right or better way

Speaking of 0367, I think its biggest conceptual problem is that the payload to be attached is not inside of the <attach-to> element

That makes it impossible to use <attach-to> as a general vehicle for a LMC to a message that is attached-to another message.

(do I need to draw ASCII art?)

Ge0rG, but then having it inside will break the LMC fallback for clients not implementing it

Ge0rG: I'd like to see that art, indeed. I think I understand what you mean, but not sure if it really is a problem.

mathieui: right. bummer.

to be fair the introduction of 367 even mentions emojis. so it seems like a somewhat good fit?

and creating Message Attaching 2.0 that is 95% identical might only be more confusinig

<message from=romeo id=1>cat-pic</> <message from=juliet id=2><attach-to 1/>awww, this is ctue!</> <message from=juliet id=2><attach-to 1/><attach-to 2><lmc/></>awww, this is cure!</>

ralphm: ^

larma: I didn't see your response to the comments by Kev on list, in the thread, or on the council minutes. Great that you want to move forward!

Ge0rG: oh, you mean attaching with a body?

Ge0rG: well, this is what we have threads for. I don't think I'd see what you have there as a valid case.

i think the issue occurs when lmc uses attach-to

then you don’t know which attach-to is which

ah

ralphm: attach-to is an element that's not embedding what you are attaching but going side by side with it. So if you reference two different messages in one, the recipient doesn't know the meaning of the individual attach-to elements

I'm not sure if I'd like to see multiple attach-to elements in one stanza ever.

So maybe, LMC (or hopefully a more general message editing proposal) shouldn't use attach-to. Storage wise it also seems different from attaching.

LMC (etc) are basically versioning on an original message

attach-to is *not* suitable for general purpose referencing in its current versions as it can not be used when two messages are being referenced for different reasons. It would work for the purpose of Reactions though, at least I don't see any conflict with existing XEPs at that point if its done (but one needs to be careful in the future about it)

well if the content of the attach-to were to be a child of the attach-to element you could simply treat it as to indivudal attaches. but yeah i agree that lmc is different to attaching and therfor should not use attach-to

Daniel: but if you stuff the payload into the attach-to element, you lose backward compatibility.

well for some things that's good

because for reactions we don’t want a fallback anyway

ralphm: what about delivery receipts or read markers? Are those suitable for <attach-to>? How many different reference mechanism are actually needed?

XEP-0367 currently also says you can't have multiple.

Daniel: who is "we"?

Ge0rG, in that case the majority of people on the mailinglist

If we can't / don't want use attach-to for LMC, we need something else than attach-to for the general purpose referencing for thin client MAM

Ge0rG, most people who can envision how hellish a reactions fallback would be

Because both LMC and Message Attaching should be attached in the thin client MAM usecase

> Ge0rG, in that case the majority of people on the mailinglist 🤷

putting the stuff inside the attach-to element would also limit what a MAM archive has to store

Ge0rG: I am not sure if receipts or markers are a good fit either, indeed. When I think of attaching I think of use cases currently covered by References, SIMS, reactions.

ralphm: yeah, so what's the cut-off point for use-case specific syntax vs. using a general attach-to vehicle?

The current Attaching XEP is _obviously_ good for everything that appends visible content to another message.

And MAM wise, you probably want entirely different handling between: 1) LMC/editing, 2) reactions, 3) References/SIMS, 4) receipts, 5) markers.

but then again, a receipt checkmark icon is also visible content attached to the message.

Ge0rG: I see the latter more as meta data than an attachment

ralphm: let's see... 1) store all edits together, deliver them all together 2) store all reactions together with the message, deliver them all together 3) same as 1 and 2 4) _maybe_ send receipts on behalf of the archive; don't forward receipts from each client of the user because once is enough(?) 5) same as 4?

“ralphm: I also think it would be bad to have in production clients with the expectance of large changes” < it's probably been said already, but what goes into "production", whatever that means, is up to the dev and not the xsf.

Btw, I'd also like to bring up the issue of SCE because we don't have enough complexity on this issue yet: When we have SCE and want to use thin client MAM, the reference must be outside of the encryption so that the server can attach. However the encryption usually also provides authenticity, thus the reference must also be inside the encryption. The thing outside the encryption should also carry minimal information, so should only be <ref id=A/> or similar as any information what kind of reference it is leaks information outside ot SCE (which kind of is its sole purpose). Additionally the server can't do any merging on content in SCE cases.

Daniel: is conversations making any use of per-contact-device markers/receipts?

pep.: I said I agreed with that indeed.

larma: agreed. but there most probably will be a set of elements that a sending client must put both into and outside of SCE

“flow: I still wonder why nobody hasn't simply wrote a quick "Reference Other Stanza" XEP, doesn't appear to hard” < if you read the reactions thread, Marvin volunteered

the interesting thing is when you put content inside of the <attach-to>

...and if you read today's discussion in here, it IS hard.

And no it's not exactly 367. From what I understand 367 wasn't meant to be used how council is pushing it to be (quote from the author, Sam) ✏

pep.: a ref to that quote please?

Ge0rG: 1) Not sure you want to deliver all edits. Maybe just the last one. 2) Store together (probably a separate dimension), retieve a summary, when needing more info (like when hovering), retrieve them all. 3) I'm not sure actually 4/5) markers are interesting in that you may have a bunch in groups, and you may want summaries, like with reactions

ralphm: so we need four different ways to reference messages, for the purposes of Future MAM?

pep.: The goal of Council is to give guidance on protocol development. This might indeed conflict with an author's original intentions, and I am not sure if this is a problem.

pep.: i.e. once a document is in the process, we rely on general consensus with guidance by council.

Ge0rG: well, I'd say that the id handling is the same for all of them, but how such things are processed is different. I'm not sure if it should all be forced into a single element.

Ge0rG: and yes, MAM handling itself is going to be much more involved than the currently naive single dimension.

ralphm: from a MAM service implementers point of view, I don't just want to know the relationship graph (IDs), but also the type of edge invovled, so that I can do useful processing

yes

ralphm: so maybe "this message is some kind of reference of that message" is only useful if you want a naive two-dimensional store where the original messages are stored as a relationship group

but not if you plan the things you outlined above

“Ge0rG: pep.: a ref to that quote please?” < either I got it from the reactions thread from Marvin, or I heard it directly from Marvin who asked Sam when writing reactions

(Still catching up with messages on this channel)

pep.: the last mails from Sam were purely sarcastic, it seems

Ge0rG, what I wanted to say: the general purpose referencing XEP would have to have some kind of usage attribute on it, so we can do something like `<ref id=A usage=attaching /><ref id=B usage=lmc />` (not necessarily this combination, but I am certain it will happen that we want one message to reference two others). Now if we put these <ref />s with usage outside of encryption we are leaking information. I am arguing for the ref outside of the encryption being somehow different than the ref inside. As the ref outside may be manipulated by a MITM, it shouldn't be trusted by the recipient at all and be completely ignored. It should become a server annotation only, which is not relevant for the message as displayed in a client, but can be used for thin client message attaching (in which case a MITM attacker can only manipulate the message such that it is not displayed in thin clients at all, which they can anyway). I'd prefer if we somehow write up what will be the usecases of the general purpose referencing so that we can actually find its requirements, before we force XEPs like Reactions to use something that makes little sense in practice later on.

Ge0rG: well, I suppose the storage itself could be multiple tables or a DAG, or whatever. I'm not sure t.b.h., but the retrieval patterns differ.

larma: this is an interesting point in the current discussion ralphm and I have

Ge0rG: mails from larma

larma: what is SCE again?

stanza content encryption

ralphm: Stanza Content Encryption

420

pep.: those don't count.

pep.: I'm looking for a statement from Sam, directly.

Ge0rG: same

Larma: I'm not sure about the various cases in-or-outside encryption. I'd expected you'd want to encrypt all the things.

ralphm: you can't encrypt for the MAM store

Ge0rG: yes of course

This is conflicting with "encrypt all the things"

Ge0rG: I'm not sure if having some bits encrypted and some not is a useful thing.

ralphm: nobody said it would be easy.

I.e. I don't think having server side archiving mixes with e2ee

ralphm: it has to mix

Why?

you could argue that you can't have a "thin e2ee client", but I think there are people doing OMEMO in Converse.js

People will want to a) have e2ee and b) use MAM and thin clients. We must make sure that it works.

personally, I think that XMPP and E2EE don't mix well, and that XMPP should be used in scenarios where E2EE is not wanted. But my opinion is very unpopular.

I mentioned this before, I am a bit skeptical about OMEMO and e2ee in XMPP context in general. You can't have all the things, I think.

ralphm: we have to accept what the users want and try to make it work out

larma: how, bar the protocols itself, do you think this is achievable?

we already are the blackberry of instant messengers.

Ge0rG: I haven't got any offers for narcotics yet.

ralphm: didn't you complain about spam just some days ago?

yes, no narcotics :/

I'm sure there were also offers for illegal drug markets.

maybe you couldn't read them because they were encrypted with Russian.

hm

ralphm, e2ee works with thin clients if minimal thin client annotations are attached outside the encryption. You are not leaking a lot of information by that, but still get the advantage of thin clients

Ge0rG, in my very personal experience a not insignificant amount of users are fine with the limited scope of omemo/otr/27 only encrypting the body

i rarly get asked for sce

Daniel: do they even know?

larma: you mean that the act of referring is outside the encryption, but not the payload of said referencing

What kind of user tells you that

ralphm, exactly and also the reason for referencing should not be mentioned outside

larma: that might theoretically work, but it is probably a pain for proper handling of all the use cases I mentioned above.

> ralphm, exactly and also the reason for referencing should not be mentioned outside this conflicts with Smart MAM

SMAM :-P

ralphm: maybe a useful degradation would be "store all the grouped message in bulk, deliver them in bulk"

pep., good question. i don’t know. the ones that are just using e2ee for the good feels probably not. but we don’t need to 'fix' it for them

Ge0rG: I guess.

Given that meta-data kills, XMPP E2EE is a huge scam.

other than that i reckon it is not 'unknown' that the existing e2ee methods don’t do sce

Ge0rG, Not sure how smart your MAM should be, but if your server-side information is limited, obviously you cannot be super-smart

Ge0rG, i'd argue that the meta data that kills is mostly the who with whom

and not 'has read a message'

Also IMHO this is mostly hypothetical, the smart MAM will probably be adopted as fast as MIX

Daniel: speaking of which, somebody just complained over in prosody@ that their Conversations is sending a file encrypted via socks5 bytestream to gajim, but gajim doesn't decrypt it

larma: normally I'm the one with the pessimistic adoption rate estimations

larma: standards development is a) not easy, b) not fast. MIX has a lot going for it, but these concepts are hard and people's time is limited. If somebody would be hire me to work on this full time, I would.

yeah, can't we get our own cryptocurrency billionnaire sponsor?

I'd be glad to work on XMPP full time as well.

A "MAM delivering grouped messages in bulk" would work and is rather easy to implement both server and client side. It's not a perfect optimization, but it solves the problem of thin clients.

And just to be sure, the reference itself should also be inside the encypted payload, to prevent spoofing, right?

right

Anyway, I am still curious: in which cases would one have to reference two different messages from one stanza?

a type-less reference outside, and a typed reference (LMC / reaction / ...) inside

ralphm: the current go-to example is an LMC of a message with a reference

Yeah, I don't see that use case.

why not?

because you want a dedicated protocol for LMC?

No

I think that you'd either have loosely coupled messages in threads (using thread ids), or actual attachments. If you want to LMC the former, you can. I am not sure the latter would need to support edits.

Other than, maybe retraction, which is still not LMC

technically, LMC and retraction are very close

except for the moderator use case

But for retraction you can just refer to the thing you are retracting. You don't need the whole thing.

so you mean a retraction lacks a payload?

I think so

Like in PubSub

I'd actually like to see retraction re-worded as a payload-less LMC, but I see there are reasons against that

I'd like edits instead of LMC

OTOH, we could extend LMC with a @by attribute denoting who did the edit

and retractions separate

I'm using LMC as a synonym for arbitrary edits. The "last message" limitation is the worst thing about that XEP

let's kill it then

If I had unlimited time, I'd rewrite LMC into "Message Edits"

ralphm: what would you do differently with "Message Edits", in comparison to LMC?

Make sure it has the same id handling as the other things we've been discussing, lift the limitation on the last message, change the title?

I /don't/ think we should support editing messages by others in any form. This is why I think we should have a separate thing for deletion.

eh. retraction

ralphm: speaking of ID handling, we haven't figured out yet all cases of how to edit a message that we didn't receive back from a MAM yet.

Moderators should surely be able to retract others' messages

Ge0rG: I think all of this discussion hinges on figuring that out

ralphm: I think it's useful to have editing of messages by moderators.

Ge0rG: why?

You could remove expletives for example. mod_pastebin could be implemented in terms of editing.

I think that opens quite a can of worms

Either a message is acceptable, or it isn't.

Because we have a good server side mechanism to enforce what's acceptable, that can be used by moderators?

I'm not thrilled about a way that allows moderators to edit random messages. It makes it hard to see whose message it was. I suppose you also need to be able to trust the MUC server in any case.

I'd much rather have a situation where people are in control of their own words, and if that's not acceptable, the message is retracted in its entirety.

That also makes it much easier to have edits override the original. If others can edit, you need to also be able to retrieve the original.

Is having the XSF not wanting to deploy specific features reason to prevent a XEP that implements exactly that feature from being standardized?

Reading back, it still isn't clear to me how Sam thinks XEP-0367 should be used, and how Council wants something else.

I'm not liking content editing by third parties either - but should we therefor not allow such a XEP to be standardized? ✏

Guus: well, I don't think it is wrong for Standards JID to have an opiniated process.

Guus: we deal with privacy and security concerns, so maybe yes that could mean we won't standardize a particular feature.

Guus: and that in turn might mean that you'd have to have a private extension, or write an IETF draft

> larma: "However the encryption usually also provides authenticity, thus the reference must also be inside the encryption." I am not sure about the "must" here. And I also think this is a good example for something we should not bother with at this stage. I do believe can't solve the metadata issues with e2ee in XMPP right now, and probably ever.

flow: I have the same feeling about metadata

\o/

Hi! By who's lack of activity https://xmpp.org/extensions/xep-0396.html is Deferred?

matkor: usually this means the Author, but in principle 'all of us'

matkor, everyone. As in, someone just needs to work on the XEP

matkor, but why do you bother about it being deferred?

if you implemented it, then please provide feedback

ideally that leads to an update of the XEP, un-deferring it

Re attach-to/reference: Why not simply have a <reference-ohter-stanza/> without childs which references at most one stanza regarding one concern?

s/at most/exaclty/

flow, I am just newbie XMPP user trying to transfer files encrypted between gajim and conversations. Not sure what is "right way" to do it

matkor, that's the question we ask yourselfs here daily: What is the right way :)

Seems Conversation uses XEP-0396 with success I wondered why gajim is lacking it.

flow: "references exactly one stanza regarding one concern" so what do I do if I want to reference two stanzas regarding different concerns?

matkor, the answer is usually because nobody implemented it

flow: yes, I was just discussing with Ge0rG whether there are cases you'd need to refer to multiple messages, and I stated that I don't think there is one, really.

Ge0rG, send two stanzas

Ge0rG: still like to see that use case

or maybe it is not clear to me what "want to reference two stanzas regarding different concerns" means

flow: what, if the same content relates to two stanzas in different ways?

Ge0rG, send two stanzas?

also, please provide a more-or-less real-world example

flow, sure, but there are usually reasons why sth is not implemented like a) there is better way, b) sth is obsolete

otherwise it is hard to reason about it

matkor: c) nobody has found the time, yet

matkor, the most common reason is because nobody did it

ralphm, flow: I don't have a convincing example at hand, but I'm pretty sure we'll end up wanting to edit a message that's attached-to another message.

Ge0rG, edit as in? LMC?

flow: yes

flow: see my discussion with ralphm above regarding "edit" vs LMC

Ge0rG, the LMC that message that is attached-to another message with the yet-to-be-invented generic attach-to mechanism

*then LMC that

flow: but I want LMC to use the generic attach-to mechanism!

it's not generic if it doesn't cover LMC

then bump LMC to do so

flow: but then I have double references.

`<message><body>this and that message</body><ref id=A start=0 end=4 /><ref id=B start=8 end=12 /></message>` = message referencing two other messages, although these references are probably not relevant at all for thin client MAM

Ge0rG, double references?

flow: two references in one message

Where?

larma: that is a good use case indeed

flow: 1) this is an edit of my message X; 2) this is a reaction/attachment/whatever to message Y

ralphm: I'm sure Message References does cover that one ;)

Ge0rG: except if you want to use XEP-0367 for doing referring to another messages.

larma, how often do you reply in one mail to two mails in a mailing list? That is one of the seldom uses features of mailing lists and MUAs

the mail protocol allows it, but is very seldom used

flow: in the context of Message References, you can also reference users, URLs, etc.

flow: so it would be perfectly fine to have a message with 64 references, containing the nickname of each other occupant of this MUC

Ge0rG, probably, but that is not the context discussed here

flow: if only we had thread references.

We are strictly taking about referencing stanzas, and the question is: Do we want to introduce additional complexity by allowing one stanzas to reference more then one other stanza

flow, so you are proposing to change from "seldom used with XMPP" to "I use email instead because XMPP doesn't support it"

ralphm: but only because 0372 lacks support to reference messages ;)

There's a difference between pointing to a message with some id (which can be documented seperately), and use cases thereof: creating a secondary message that attaches to, or edits, a particular message, v.s. a separate message that wants to 'refer' to what was said before. FWIW, I'm not sure how to render larma's example.

flow: I've given an example above.

larma, no, I am saying that I never once in my life found myself in the situation that I wanted to reply two more than one "post" with a single "post"

be it xmpp or mail

and I believe this to be true for the majority of users

I was multiple times in a situation where I wanted to reply to multiple mails from subthreads of the same thread. On standards@.

Arguably, I'm not the majority of users.

Ge0rG, then why didn't you do?

I think larma is more like including hyperlinks

larma's example

flow: because I had no idea how.

flow, But what does that imply? That we shouldn't be able to handle it on a technical level?

larma, I don't believe the increased complexity is justified. Please note that I am strictly talking about referencing stanzas

XEP-0372 has a way to include hyperlink references to URIs and we may want to construct a way to express a pointer to a particular message, either in private context, or in MUCs, but I don't think this is the same as the other use cases.

Ge0rG, let's assume you didn't know how is because your MUA either does not provide the feature or because it is very well hidden. Then you can ask yourself why that's the case…

`<message><body>look at that previous message</body><reply-to id=A /><reference-back-to id=B /></message>`

I.e. you can still have a URI that points to an XMPP message, or a tweet, or a something else.

that that URI potentially would not be processed by the message archive in any way, which is probably fine

larma: we don't have reply-to, really, we have threads

flow: indeed

but nobody uses threads, so we should replace them with generic references.

god no

ralphm, we don't have X so we should never ever support it

but if we do that, we have a convincing example of why it makes sense to have two references in one message.

so we can have references with a exactly-one limit, which is processed by the message archive, and references with a one-or-more semantic, which are then not processed by the archive. That sounds like a good comprosmise

references and References.

although I am not sure if we need the one-or-more semantic references ever

larma: not at all. We do have 20 years of prior work, and Ge0rG's assumption that nobody uses threads is probably not true.

It's probably not true, but it's pretty close to it.

In the end this discussion is about the tightness of coupling

larma, it's not "never support it", but be obviously should not design protocols with things that do not exists (and are not going to exists in the near future) in mind

flow: I disagree on this one. We should take into account the long-term delelopment of the protocol when adding new XEPs

Otherwise, we end up with MAM and Carbons.

I can't catch up with all the backlog, but re: multiple references, I think you have one "This message follows on from this other message" reference (attaching, if you will), but potentially have a message reference many other things (some of which may be messages). e.g.: "re: Ralph's mesage<AttachmentReference> Hey @<Reference>Ralph, I agree, but did you see @<Reference>Ge0rG's <reference>message earlier?"

Ge0rG, I did not say that we should not take the long-term development into account. But I have never heard of anyone talking about introducing reply-to

Which is why I'm quite happy with the idea of keeping attaching and references distinct.

Be it the two XEPs, or two different annotation types in a single XEP.

Kev, does that matter much?

Kev: I'm not quite sure what you mean by "follows on"

Yes, and I realise there's essentially three reference types, as you say that.

Kev: but I assume 0372 is about showing a kind of hyperlinks inside of the body, whereas 0367 is about structural order of messages in the archive, delivery and display?

Kev: three?

There's In-Reply-To, which was my first reference in that message, there's "Here are some other data relevant to the message", which were the others, but there's also "I am hanging this message off the other and it doesn't make sense in isolation", like Reactions and Editing/Correction.

If a message is strictly about another message (this message is marking up that other message with hyperlinks, and I'm referring to characters 4-6 in this other message, References), Reactions, Edits, Markers, then I'd say we can come up with a generic mechanism for including such a singular reference. E.g. by using attach-id. If you want to point to other things loosely (again References, but the *target* of the link is elsewhere) you can use URIs or somesuch.

In-Reply-To is what threads are for. But isn't a Reaction also In-Reply-To?

I am happy to limit 372 to the second type. I originally intended to cover everything, but I'm not dogmatic about that.

Ge0rG: you can model it all kinds of ways. In e.g. Slack, a textual reaction is a separate thing, that itself can have emoji reactions, marking up (references), etc.

ralphm: I'm trying to understand all the implication, and maybe to find a good way forward.

I think that the trinity proposed by Kev is a good approximation

Kev: I'm saying that in 372 you have two types of things to refer to: 1) the plain text you are marking up, with char refs, 2) hyperlinks to tie to those hot texts.

(Incidentally, I don't think threads necessarily are fit for purpose, but I'm happy with just saying it's a distinct requirement and excluding it from discussion)

ralphm: There are two halves of that type of reference, yes.

for 1) you need something like attachments, 2) URIs are great

ralphm: for 1 you need what's written in 0372

unless you want to make the marking-up a separate message from the original text

Ge0rG: Which I think has value.

in which case, you attach-to the original message a set of reference-markups

e.g. a MUC service that annotates a message once it's fetched an async resource.

Ge0rG: this is about 'anchor' v.s. 'uri'

Ge0rG: we can replace 'anchor' with 0367.

ralphm: now I'm totally confused.

And then make it easier to mark up multiple parts of the text without having to repeat the anchor.

ralphm: to quote from 0372, which is anchor and which is uri? 1 <reference xmlns='urn:xmpp:reference:0' 2 begin='72' 3 end='78' 4 type='mention'

Ge0rG: a link has two parts: the text you want underlined, the URI you want to jump to. Clear?

Let's see if I can avoid distraction long enough to gist some example thoughts.

5 uri='xmpp:juliet@capulet.lit'/>

ralphm: so 2+3 is the anchor, and 5 is the uri

ralphm: and if you want to make the <reference> a stand-alone message, you need to attach-to the list of <references> to the original message.

Ge0rG: XEP-0372 has two examples: one with a plain text included, there's no anchor here, because you are referring to the included body. The other has no body element, so you want to reason over some other, earlier message. So that example has an anchor attribute, and I'm saying that we can replace the latter with 0367.

Ge0rG: yes

The former is section 3.2, the latter is 3.4.

<message id=1><body>Hi Juliet</body></message> <message id=2><attach-to id=1> <reference xmlns='urn:xmpp:reference:0' begin=3 end=9 uri='xmpp:juliet@capulet.lit'></reference> </attach-to></message>

Something like this maybe?

One flaw with XEP-0372 as it is now is that you can have a single message that has references with two different anchors. I think that's confusing and stupid, and having that replaced with XEP-0367 would be a lot cleaner.

ralphm: wwhen do you need two different anchors?

Noever

https://gist.github.com/Kev/675475f30933b6d4f63bc9fce35bd29a

Never

ralphm: it's not long ago that you argued this would never be needed.

Ah.

Ok, I'm suggesting we could do something like this, which is an advancement of 372/367

I think it's what Ralph's describing.

But if not, I've not understood and would like to :)

attach-to is the thing that could be 367, or something like it.

yes

Kev: the metatext inside your gist is confusing.

reference is the thing that's 372-ish.

yes

yes

yes to what, actually?

Ge0rG: Which bit of metadata? Happy to update the list, I'm trying to balance brevity and clarity.

a confirmation of Kev's interpretation of me

I guess I should add a reaction to that too, hang on :)

Kev: do you have an opinion on nesting stuff inside attach-to v.s. sibling?

Kev: I'd like to see an approximation of the XML for @target; I think I get the start/end things, even though they are distracting my OCD

Kev: what do you think of the XML I posted above?

it would also solve ralphm's issue with multi-anchoring

and it would solve the issue with "dumb" MAM grouping.

I don't have an issue with multi-anchoring, we should not have it.

ralphm: I do, I think it's nested.

Ge0rG: Let me have a look at your XML, 2s (Gist's edit mode isn't working for me so I can't update my proposal for reactions).

and you could do SCE with low effort:

<message id=2> <attach-to id=1/> <ENCRYPTED><attach-to id=1><reference xmlns='urn:xmpp:reference:0' begin=3 end=9 uri='xmpp:juliet@capulet.lit'></reference></attach-to></ENCRYPTED> </message>

Yes, I think your 12:04 XML is roughly equivalent to what I pasted in intent.

Kev: but at most one, right?

I'd also like to consider that there is semantically two types of attachments: Those that only the original sender and maybe a MUC (admin) can do (LMC, Retraction, Attaching?) and those that everyone can do (Reactions, replies)

okay, "low effort" actually means you need to extract attach-to skeletons

Ah, hi larma.

Yes, I was thinking about ACL on this stuff earlier. I suspect this doesn't affect the wire protocol, but I'm aware a MUC might need to e.g. whitelist attach-to payloads.

larma: that is a good point, clients would need to check for this

https://gist.github.com/Kev/675475f30933b6d4f63bc9fce35bd29a updated

<message id=1><ENCRYPTED><body>Helo tere</body></ENCRYPTED></message> <message id=2> <attach-to id=1/> <ENCRYPTED><attach-to id=1><lmc-marker/></attach-to><body>Hello there!</body></ENCRYPTED> </message>

larma, good point

https://gist.github.com/Kev/675475f30933b6d4f63bc9fce35bd29a updated again

Ge0rG, why is the body outside the attach-to? looks wrong to me

larma: because of backward compatibility to pre-LMC clients

But semantically what you wrote is "attach the lmc marker to the message and have a message with this body"

the lmc-marker is meant to signify that.

is backward compatiblity really something we should be concerned in this case?

flow: yes.

This is also why I wondered if it wouldn't be just as fine to have the attach-to be empty and a sibling.

I think, in as far as we do fallbacks at all, we might have to consider having duplicate content here.

Kev: you last example, message 6, would have backwards compat issues

ralphm: it's not elegant, and only workable as long as you have at-most-one attach-to. Also it makes parsing in "Smart" MAM way harder.

Ge0rG: still want that more than one example.

ralphm: just because we two can't come up with one, it doesn't mean there is none

I think having attach-to a sibbling makes sense for those messages that do work as standalone, i.e. when you don't have the previous message (LMC, replies), but not for those that don't (reactions, retraction)

larma: good point

Updated.

Kev: what's wrong with my <lmc-marker/>?

larma: for reactions, retraction, you wouldn't point to two different messages in one stanza.

Ge0rG: Maybe just that I missed it in the flow.

Kev, I totally dislike the duplication of the body for backwards compat

larma: Good, me too :)

Kev: 12:12:49

ralphm, that's for those that only point to one (attach-to, not references)

Ge0rG: That might work.

ralphm: what about reaction retraction?

larma: 0372-style references are a different topic

Ge0rG: https://gist.github.com/Kev/675475f30933b6d4f63bc9fce35bd29a ?

Ge0rG: you can just retract the stanza of the initial reaction, like message 4 in Kev's example

Kev: yeah

Kev, that's implying that you can only change the body with LMC

Yes, that's what it is for.

ralphm: no, it's for any kind of payload.

> Correction MUST only be used to change the logical content details of a stanza (e.g. the message body) and not to change the nature of the stanza

In this example, the attachment payload is still within the attachment element, which is good, it's just that it's then a pointer (careful not to say reference) elsewhere in the stanza.

You're both right.

Or all three.

You can replace any payloads, but not rewrite the logical-type of the stanza.

Kev: rename the element to <message-edit/>

You mean s/body-is-edit/message-edit/?

Kev: yes. or maybe message-is-edit

edited-message

Bike shedding!

These are details I feel don't need to be in a pseudo-exmaple.

What's really relevant, I think, is that the LMC instruction lives within the attach-to.

And that references can also live within an attach-to, as can reactions.

And that we can have multiple attach-to elements with different payloads in one message!

Why is that better than as a sibling (trying to get my head straight)

I think having the message a sibbling of the attach-to but still put something inside the attach-to is semantically broken. Either the content of the attach-to is attached, or the sibblings, but this combination is just awkward

Ge0rG: or we stop thinking we need multiple.

ralphm: it makes processing for Smart MAM easier, because you don't need to analyze all the siblings and derive conclusions from their presence

ralphm: Clear hierachy. Sometimes you have 'extra noise' in stanzas, and it's much more convenient to be clear which bits apply where.

Ge0rG: I don't yet understand the need for multiple attach-to in a single stanza.

And yes, what Ge0rG says about nesting, too.

Kev: nobody does, but in a year we've codified that There Can Only Be One, and somebody comes up with a compelling use case that can't be implemented any more

Kev: 0184 would have been so much better with multiple payloads

Kev: ok. I'm asking because back in the day, Blaine thought we should have originally gone with allowing Atom payloads next to the pubsub event wrapper, so that you could interpret the notification even if you didn't understand pubsub.

Ge0rG, I am not sure that this means that it can't be implemented any more

but nobody came up with the use case of offline message mass-acking

So SMAM would see an attach-to, and process its children (this is also a motivation for having the LMC body within attach-to, not external)

I sure would require a ns bump

flow: an ns bump and server-side conversion modules.

point is, it is easy to go from a exactly-once to one-or-more semantic, but not the other way around

or "easier"

flow: or clients adding multiple versions of attach-to elements with different semantics into one message.

I'm not sure I'd say it was 'easy', but 'easier', yes.

with the proposed semantics, is there a reason NOT to have multiple attach-to in one message?

flow: well, since it moves into attach-to, you don't actually need to change the namespace for backwards compat. Of course older clients now see multiple messages, because they don't know attach-to

Ge0rG: If everything is properly nested, no. But if we allow leakage outside the attach-to, kinda yes.

Ge0rG: and if we ever go from a esactly-once to a one-or-more semantic, you can say "told you so"

flow: and then file bug reports for clients that only hanlde one

> point is, it is easy to go from a exactly-once to one-or-more semantic, but not the other way around I think the opposite is true: if you have implemented parsing for N elements, and the message only may contain one, your code just works™

"feature requests"

I do find flow's argument about ease of switching fairly compelling, though. We could publish with one, and see if anyone has a counter-argument while it' sexperimental.

Kev: "A receiving entity SHOULD be able to process multiple attach-to payloads"

Ge0rG, I don't see how parsing is the heavweight thing yo'd have to change here

Ge0rG: That's the worst of all worlds, isn't it? :)

Hey, I could edit all my messages to the same text with one message!

flow: not parsing but interpreting.

:headdesk:

That's exactly what I want to prevent.

flow: `AttachTo message.getAttachToPayload()` vs. `List<AttachTo> message.getAttachToPayloads()`

implementing interpreation of exactly-once is probably way easier than one-or-more

We've removed batch stuff in several protocols, like pubsub.

If we have Unencrypted: `<message to=room from=ralph id=4><attach-to id=3><reaction>👍🏻</reaction></attach-to></message>` Encrypted: `<message to=room from=ralph id=4><attach-to id=3 /><ENCRYPTED><attach-to id=3><reaction>👍🏻</reaction></attach-to></ENCRYPTED></message>` This confuses me a bit: Is attach-to now to be semantically understood to a) attach sibbilings b) the full message c) the content of the attach-to element

larma: The content of the attach-to. Although in the case of LMC there's this open question about whether it's ok to referer out to the body or not.

larma: what about <attach-to id=3><ENCRYPTED/></>

Ge0rG, that would be the logical consequence, but then this is going to be terrible with legacy fallbacks

Kev: btw, nit (but not really), you don't need the "@" if you have References type mention.

or rather: <attach-to id=3><external-sibling name="ENCRYPTED" ns="urn:xmpp:sce:0"/></>

(More garbage in body is never really welcome)

and <external-sibling/> is a reference to the element name and xml:ns of the external sibling. MAXIMUM FLEXIBILITY!

pep: but it might be what the user typed

`<message><attach-to id=A><ENCRYPTED>actual</ENCRYPTED></attach-to><ENCRYPTED>fallback</ENCRYPTED></message>` definitely sucks

ralphm: the user's client should replace `@user` with `user` and a reference, then! :P

Ge0rG, how would you do if you attach multiple subblings?

Ge0rG: if could

larma: have multiple <external-sibling/> elements. Or just define the ns if they all share one

Ge0rG: I meant 'it could'

ralphm: I'm clearly thinking of the Slack UX here

Ge0rG: Slack doesn't actually remove the @

Showing the @ is fairly usual in modern clients, I think.

But I don't think this is the most important of the discussed details :)

nah

I like how this is shaping up.

Kev, ralphm: am I overengineering things with <external-sibling>?

Ge0rG: yes

Much?

yes

A more pragmatic alternative?

I don't know, honestly.

I don't entirely understand why it needs to be alongside rather than nested, yet.

I mean, the reference-tracking implications of that are really bad.

Kev: the default would be nested

I think having an empty attach-id is fine. If the encrypted payload is unwrapped, it can override it.

When would you external it?

the only real use-case I see is message edits with legacy.

ralphm: I think an empty attach-id is pointless. It doesn't allow the archive to do anything useful, and other recipients are able to decrypt.

Kev: it allows the archive to group related messages

Kev: this was discussed arelier

Kev: so that you could still have a limited two-dimensional grouping in MAM, even though you don't know about the encrypted bits.

Kev: a thin E2EE client still benefits from that grouping

Ge0rG: I am seriously considering whether the best thing for legacy fallback might be to have <fallback/> inside the attach-to, and then clients will need to understand <attach-to/fallback> in order to show a fallback.

I.e. retrieve all the encrypted attachments in one go

ralphm: But you can't have any grouping if there's no id, can you?

Kev: what would <fallback/> mean?

Kev: I mean no payload

Ge0rG: It'd be a plaintext that can be shown to the user if your client doesn't support that type of attachment.

So you need your client to understand attach-to/fallback, but once it does it can render a fallback sensibly for any other attachment type.

Kev: wait, what?

Kev: <message to=room from=ralph id=4><attach-to id=3 /><ENCRYPTED><attach-to id=3><reaction>👍🏻</reaction></attach-to></ENCRYPTED></message>

Kev: MAM can know that this thing belongs to 3, but the real thing is inside

ralphm: Yes, that seems arguably ok. That's not an empty id though :)

I didn't say empty id

"I think having an empty attach-id is fine" :)

oh, I did

whoops

If you didn't mean an empty attach id there, I don't know what you meant :)

See, now I need edit

But yes, empty attach-to other than the id seems reasonable.

Suboptimal, but the best you can get if you want encrypted payloads.

Is `<message id=A><body>First</body></message>`+`<message from=B><attach-to id=A><any-attachment /></attach-to></message>` then considered to be compacted to something like `<message id=A><body>First</body><attached from=B><any-attachment /></attached></message>` server-side? (the not so smart version of thin client MAM, the smart version would probably look into `<any-attachment />` and make some special handling for it)

larma: could you please reformat your messages for the XML to begin at new lines, and maybe have some more structure?

I don't think it's compacted, no.

What do you mean compacted?

ralphm, for the usecase of thin client MAM

(which seems to be the main use-case of all of this after all)

The attach-to lives in its own right, linked to the first message, rather than replacing the message itself.

So you'd be able to query the attachments of a message through SMAM.

larma: the only optimization a MAM server should do in this dumber case, is to ensure that when you retrieve a message, you get the attachments, too.

But, by default, when requesting through MAM you'd not get them.

Wait, what? :)

ralphm: also vice-versa? When you query for an attached message, to receive the original first?

larma: for the slightly smarter interaction, you probably don't want to retrieve all the things. Like with reactions, I'd say we'd have an element with a summary of reactions that's returned when you receive a message from MAM, but you need to query for all the individual ones to show who sent them.

ralphm, OK makes sense to me, just want to ensure we are not missing the actual goal of all of this, as we already talked about over-engineering

(like in Slack, you only see the counts, until you hover over them)

but that means you need to deliver a map(reaction -> count)

yes

If we don't have any special semantic meaning of the content of <attach-to> I'd say it is better to not actually put stuff in there and make it always a sibbling

I'm happyish to punt solving how you do the archive queries to later, as long as we're confident we're encoding enough in the protocol that we /can/ solve it later. ✏

Kev: oh, MAM is a going to be quite another topic, indeed

larma: I think we do have semantic meaning of the children of attach-to.

larma: but we actually _want_ to have semantic meaning for the children

just not generally.

the two exceptions being E2EE and legacy Edits

“ralphm: Slack doesn't actually remove the @”, then the other side can put it back, but this has nothing to do on the wire.

pep.: And thus isn't worth discussing at the moment :)

> but that means you need to deliver a map(reaction -> count) Ge0rG, yes this implies that the SMAM server understands reactions, has special handling of it and doesn't make use of the generic attach-to feature (which is kind of funny as we are mostly talking about the generic feature for exactly that usecase)

larma: oh, of course it would rely on the generic attach-to feature

Sorry, why isn't SMAM using attach-to?

it would just make better use of it

pep.: what I believe Slack does is always send the original thing (plaintext) and then updates to it to mark it up. On slow connections you can see this happening sometimes.

Kev, Ge0rG, If the server needs to be able to understand what reactions means in `<attach-to id=A><reactions /></attach-to>` it could also just understand what reactions means in `<reactions id=A />`

But it doesn't /need/ to understand what reactions mean, does it?

larma: the idea is definitely that a MAM server with knowledge about attachments, will have a more efficient storage retrieval.

There may be value in it doing so, but it doesn't fall apart if it doesn't.

RIght

Kev, If it wants to build a map(reaction -> count) as suggested by Ge0rG it needs to understand

Yes. If it wants to be very smart, it can (and that is a good thing).

Yes, it would be a discoverable feature.

But a somewhat-less-smart option is also available.

larma: yes; but when we introduce another kind of attached-to payload, SMAM will automatically provide the grouping for it, just without an improved summary

And it's already smart enough that it can know that it's a {ns}reaction child in the attach-to, so a client that doesn't support those won't bother retrieving them, even if the server doesn't do the mapping and summary.

So there is GMAM (grouped MAM) and SMAM (smart MAM). GMAM would rely on attach-to and SMAM would actually look into the reactions?

There's a hierachy of smarts, yes, if you want to call them GMAM and SMAM.

So SMAM is an improvement on GMAM is an improvement on MAM.

You could have a way to say: give me all the messages (but without attachments), and then if it knows about reactions might include summaries. Then later you might retrieve the attachments.

Do I need to make another gist once I've thrown some lunch in the oven?

Yes

Doing the same

Hmm bapao

This is very much not what I needed to get done today!

Kev: yes you did. Your boss may complain to me.

Also thank you

I'd like to see an example of a message attachment being edited

speaking of gists https://gist.github.com/ge0rg/a255cb521e45bbcb89573aa3218f0cd5

larma: I don't really

is that a valid summary of today's discussion? ^

"Can be sent by anyone" - unclear to me, more thought needed.

larma: I'd rather see retraction and then a new one

"wide consensus, except for Ge0rG" - I think I'd like to hear if anyone has an example where it's beneficial; I'm not dead-set against it.

"i.e." - you mean "e.g." ;)

"an without a payload" - missing some words, I think.

But yes, I think that's pretty close to a summary, and is useful to capture, thanks Ge0rG.

> You could have a way to say: give me all the messages (but without attachments) ralphm, but you'd actually want to also put some attachments directly, like message retraction, no?

I don't believe retraction should be an attachment.

Because it has different semantics from attachments, for exactly this reason.

Indeed. And retractions should be consolidated by the server

Ah, are we talking about retracting an attach-to, or retracting a source message?

both, I suppose

I think they're very different.

Kev, so you'd want always special handling for retraction, even in GMAM. Obviously makes sense in SMAM

In case of SCE, do we encrypt the retraction element?

Kev: they are, but the protocol doesn't have to be

ralphm: Yes, it does :)

Or, rather, makes a lot of sense to be.

Kev: why? You can retract the stanza. If the stanza is an source message, then everything goes. If the stanza is an attach-to, then its payload goes?

Because I don't think that retracting attach-to by the id of the carrying stanza is entirely sensible.

why?

(is there a way for Gajim to show typing notifications in MUCs?)

So I think it makes more sense to have <message><attach-to><retract-message/></attach-to><message> for retracting a source message, and <message><attach-to type='remove'>describepayload</attach-to></message>

I agree that one shouldn't reference attach-to messages at all

I'm just thinking of the pain of implementing things in the server archive at the moment. This is all going to be moderately painful (desirable features, so pain we have to accept, but still painful), and I'd rather keep the pain as low as we can. I believe that different paths for removing attachments and for redacting messages is simpler.

I was thinking more about: `<message><retract id='...'/></message>` for both.

Kev: on these channels, "the moment" is whenever somebody talks about it, so here it is. It's not relevant for this discussion but it's relevant nonetheless.

And I also believe that storing attachments as attachments is probably easier than storing them as messages.

Kev: yeah, we talked about dimensions before, and I'm not sure yet what the MAM protocol should look like.

Kev, but often the message containing the attachment has relevant information, like the person that attached the message

Also re reactions being "compressed", "I have a use-case" for wanting individual payloads that have been sent (that is, even for just one user, have all the intermediate payloads they've sent)

But semantically, calling a retraction an attachment is a bit odd.

Matrix does the same IIRC 😀

ralphm: Yes, I would be happiest with just <message><retract...></>

pep.: I don't think we're suggesting not storing them, just that it's most useful to present to a client as a summary usually.

pep.: didn't I say we would have the ability to retrieve all the things?

larma: Yes, there's some metadata involved.

We did talk about all this at the summit, I wonder if any of it was captured in the minutes.

http://logs.xmpp.org/xsf/2019-08-29#2019-08-29-24812f82e1f8a8be

Kev: if you'd be happier with my example, what is the problem with it then?

Kev: the lookup?

Your example was to retract attach-to message wrappers in the same way as to retract messages, wasn't it?

I'm certainly not happier with that.

I'm happier when messages are retracted in one way, and attach-to retracted in another.

So like we have retract item and delete node in pubsub?

Somewhat, yes.

I thought it might be easier to just say to undo a stanza being sent, but not a hill for me to die on

ralphm: OK i right have misread

No worries. Things were going FAST

Sorry, I had to vanish into a call. Now that I'm back, the last hour of log doesn't make much sense to me.

You had to be there

You want message retraction to explicitly use a different mechanism than Attach-To? What about Message Edits, should that stay Attach-To?

Which direction does the relation go?

Zash: the new message references the old message

That doesn't specify the relation between the two though. Is there a case of a new message becoming the parent of an older one

But does the new message belong to the old message or the reverse?

can we leave the philosophical questions for later, please, and focus on the protocol?

test

a+

.

Test passed

we meet?

ouch

Oh wow, it's Thursday again :/

Welcome to my thoughts, 10 minutes ago.

(only because I got a calendar reminder 😉 )

0. Welcome

Hope everybody is reconnected, there was a strange disconnect with xmpp.org from here.

I've seen everyone say something.

Hi

wfm

yay

1. Minute taker

assuming we can't find someone from the floor, I'll do 'm this time

Thanks!

2. Badges

thx

I have reached out to mray. He responded and asked the following:

“I'd be interested in the kind of combinations that the system needs to cover. Maybe you can break it down for me without the entire context of XMPP compliance. Eventually even a complete set of possible combinations?”

Who can create such a list?

other submissions did that, no?

combinations of what, exactly?

each requirements

Levels of compliance I guess

All combinations of thingies to appear in the badges

So basically, he did 3 examples, and we now come up with all the combinations we'd like him to construct as actual badges

from the XEP: > This document defines Categories based on typical use cases (Core, Web, IM, Mobile) and Levels (Core, Advanced) based on functionality in the respective category.

so, we need each combination of Category with Level, and add a year / edition to it?

Although Core/Core 2019 seems weird.

2019 would be cool

He doesn't want to read the document, he just wants: (XMPP LOGO) | CLIENT | Advanced ) IM | 2019 (XMPP LOGO) | SERVER | Advanced ) IM | 2019 etc.

or somesuch

I'll wrap up a list

Thanks!

Also thanks nyco fo sending the results out to the list.

Thank you guys

in a spreadsheet?

maybe

3. Roadmap

Apart from this huge discussion this morning, afternoon, I haven't gotten to this.

Good progress on the former, though.

4. AOB

?

None here

I've got two

None here

Guus: shoot

isn't it about time we organize new elections?

you mean Boris Jonhson?

Usually Alex picks this up in September.

ack.

missing id

Next Guus?

2) I've been approached by https://openvoipalliance.org/ - they're interested in involving XMPP in this.

yes, elections are on my radar, have a reminder in my cal for Sept. 1st. Will open the application process after our current voting period

I'd like to see if there's benefit for us in actively becoming a partner in that effort.

(Thanks Alex)

Guus: interesting. I see they already use XMPP and converse.js

I didn't know about it

But yes, I'd be interested to know how they would like us to get involved.

Guus: can you send me the e-mail? I'd be happy to follow-up.

what email? we're talking over XMPP. 🙂

oh hehe

Niiice :)

I'll see if we can arrange for a groupchat of sorts

Thanks

Anything else?

FIN

5. Date of Next

+1W

6. Close

wfm

Thanks all!

I really like this " You can chat with us through our XMPP server to see if your project fits." but would need to check exactly what it is and what can we do

Thank you for the meeting, glad to see all of you today

thx all

more glitches from the xmpp.org server?

Glitches?

ralphm I've updated https://trello.com/c/J97CukTU/358-contact-mray-to-confirm-chosen-badge-design with the requested list of badge element combinations.

Zash: ip6 issues apparently? Says Ge0rG

The connection hung. Looks like a kick against the tires helped.

But then it dropped out again.

https://igniterealtime.org:443/httpfileupload/41c708d3-4197-408d-98a6-3e2d2b5fbfea/image.png

harhg

that was _not_ the copy/paste that I intended - sorry

Features

PEP mandates filtered-notification support

are all PEP services therefor always filtered, or can the feature go unused?

Instant dick-pic upload!

Guus: if you don't have the filtered notification, isn't it just regular pubsub on an account?

ralphm: pep adds other restrictions

My point is that if you don't do one one these, it is no longer PEP but some other subset of PubSub on an account.

Right

Curious about the use case, though

Filtering is mandatory then, for pep

No use case. Bug hunting / code reviewing

Is it PEP if there's no filtered notifications, but instead dynamic presence based subscriptions?

Can clients even apply such configuration?

Well the whole raison-d'être of PEP is not having to explicitly subscribe, so you have auto-subscription + filtering instructions via CAPS.

Zash: there's a difference?

Not in practice

Also not in theory?

Implementation detail I suppose

Although I suppose you can think up another magic to get the notifications you want.

I'd like to have negotiation for the thing ejabberd did, where it'd just ignore your caps and send all notifications

expecting the receiving end to filter

Really? That seems terrible.

One of the use cases when PEP was created was User Tunes.

Not in an environment where you want everyone to get all notifications

Initially people would just include that in presence directly.

So you got presence updates, once per 3 min. for each user.

If you filter and fork notifications on the receiving server it works, and has some benefits like MAM potential

Which presumably is less IM, more IOTish

But this isn't what the PEP XEP says, so it leads to madness.

Hence why I'd like some opt-in between servers for that

Note that I was not looking for a XEP change, I was just trying to get clarification.

Zash: if you are taking about subscriptions on account level instead of resource level, along with s2s dedupe, that seems like a worthy cause.

Especially also for MIX.

ralphm, yeah

Not having to keep (a subset of) disco#info for remote entities seems like a nice thing to me as a server dev.

As long as information is public that works. And identical between subscribers.

let's start our member meeting in 3 minutes

here is our Agenda for today: https://wiki.xmpp.org/web/Meeting-Minutes-2019-08-29

1) Call for Quorum

as you can see 36 members voted via memberbot. So we have a quorum

2) Items Subject to a Vote

new and returning members, you can see the application pages here: https://wiki.xmpp.org/web/Membership_Applications_Q3_2019

3) Opportunity for XSF Members to Vote in the Meeting

anyone here who has not voted yet?

Erm, did I?

Memberbot is still online and will accept votes

Do it! Do it now!

MattJ: don't think so

Ok, I'm on it

go for it

oh!

I did vote, didn’t I?

I'm done, probably made a massive difference to the outcome :)

I wonder if Santa is making notes for his naughty list.

MattJ: have your vote

Yay

anyone else?

looks like we have all votes

Will start counting the votes then

4) Announcement_of_Voting_Results

*drumroll*

if you reload the page at: https://wiki.xmpp.org/web/Meeting-Minutes-2019-08-29#Announcement_of_Voting_Results you can see the voting results

all new and returning members are accepted

congrats to everyone

very high yes votes,, stands for high quality members

5) Any Other Business?