XSF Discussion - 2017-09-28

Right to be Forgotten!

😂

just asking, but there’s no way for a client to delete a message from a mam archive?

No

Not in MAM

Separate specification may define that

That’s what I was thinking as well

That Prosody fork, I forget the name, had something like that IIRC

I was checking if there was something like that in urn:mam:tmp which was removed later, but it doesn’t appear to be so

a coworker came to me because a client asked him to remove a message from the archive on an ejabberd server with mam:tmp

so I had to check

the database schema in ejabberd doesn't depend on mam version, iirc

Yeah, we have to support that in M-Link, so things in the archive can be removed afterwards.

I imagine it'd be a fairly common enterprise etc. requirement.

I'd imagine the opposite

Maybe there is a separate archive that the data gets moved into?

Altho archiving and eg compliance logging doesn't have to be the same thing

(I note it's not the user who's able to do this, it's a sysadmin)

(And it leaves a tombstone)

any IM client (whatsapp, viber) has a function of deleting messages

Kev: Does it have custom protocol or some ad-hoc command?

you can even delete a message on remote end as well ;)

I forget how we do it, TBH. Adhoc, I expect.

You can send a message correction that changes it to an empty message or tombstone

true

I think yaxim will ignore a body-less LMC.

and body-less in that context also includes messages with an empty body

can I using LMC "erase" arbitrary message, or only the last one?

https://xmpp.org/extensions/inbox/message-retraction.html

zinid: the XEP says you may only edit the last message, but I would consider it more practical to allow editing of the last N with N maybe around 20.

Hm. The first time I proposed rolling back Carbons and Resource Locking was in 2015.

I think LMC is better (if it supports arbitrary deletion)

in this case you can design database easily (e.g. append only)

zinid: send a mail to standards@ and propose a change.

Kev: you suggested I write a thought-a-day series about the brokenness of current XMPP. If that was a serious proposal, what do you think would be the best venue to place that series?

It was semi-serious. I'm not 100% sure it's the best thing, but it seems like it might work. I was pondering a mail to standards@.

Kev: I'm very interested in following through on that, in a form that's most acceptable to XSF members. I'm flexible regarding the form and the place to post, and personally I'd probably write a (very long) blog post instead.

I'm concerned that a very long blog post is likely to get TL;DRd.

(including by me)

Because it allows to provide coherent arguments, has better markup and is not XSF-branded.

But give it a go :)

Kev: what about a slide deck?

Oh, that has potential, including for if we do a high-bandwidth meeting to deal withit.

See what people-other-than-Kev think.

people-other-then-Kev: I'd like to prepare a list of things that are currently broken in XMPP. What would be the best place for you to read it? standards@? my blog? XSF blog? a slide deck?

I don’t care :)

jonasw: as in "won't read it anyway" or as in "will red it either way"?

the latter

Ge0rG, zinid, re LMC/deleting messages, that would only be true in a more-or-less controller context though, most client don't support LMC in semi-anon MUCs, and even when they do it would still be duplicated in the history

pep.: but you still "remove" it in your archive, which is the major goal

I'm keen it's not the XSF blog, FWIW.

Kev: I just mentioned it for completeness sake. It was not really serious.

zinid, technically you don't remove it right? You just put a new message in front

Ge0rG: who's the intended audience?

Guus: XSF members and people who implement MAM, Carbons, mobile clients etc.

Guus: essentially everyone contributing to "modern" XMPP

then the mailinglist seems appropriate

The jdev@ list?

members -> members, impelmentors -> jdev

Guus: see above, a coherent presentation will lack formatting and be tl;dr for most.

pick your audience :)

Ge0rG: the more reason to pick your audience carefully.

pep.: yes, but this is kinda normal, your "delete command" appears as "delete marker" in the database, so the original messages becomes a tombstone

pep.: such approach is used in some distributed databases for example

in blockchains also

dwd: (moving over from jdev@) I'd like to make a talk, but can't guarantee to be present.

I think this is standards@, not jdev.

Kev, MUC, not list.

Ge0rG, If you're not present, the talk will be less effective indeed.

zinid, so it wouldn't just be a simple "empty message", it would be more than that and the db (and MAM?) would react to this as well?

dwd: right. The problem I'm trying to solve now is that the challenge is very complex, and most people lack the time.

dwd: Guus suggested mailing to the jdev list. I don't think that's the right venue.

dwd: Kev suggested to make a series of one-thought-a-day posts to standards@, but I'm not sure I can split up the problem appropriately.

Ge0rG, I don't know... I think if you chop up each issue, explain What it is, Why it matters, Who needs to do the next step, and What they need to do, you'd find which issues you can get traction on quite quickly.

^

Ge0rG, If you can't chop it up into smaller issues, then I genuinely think it's a lost cause. It becomes "EVERYTHING IS BORKEN, FIX IT PLS".

Kev: I suggested that he'd pick an audience, and suggested that if that was primarily (client/server) developers, he'd pick jdev.

pep.: db will not react, this goes to client: when it sees the "delete marker", it remembers it and when it matches a message it ignores the message (and/or deletes in local history)

Guus: I think that standards@ would be the most appropriate place.

then go for that :)

at least if the form is "mail(s) to a ML"

zinid, that doesn't work in semi-MUC for most clients

pep.: I know

but current message correction doesn't work in my Tkabber for exmaple :)

so?

Ge0rG: if it's really, really long, but structured, perhaps do this on the wiki?

If you are going to have mutable history, then MAM might not be what you want

pep.: you see, if you want to synchronize deletion with all resources (which can be offline at the moment), how will you do that?

No idea :/

Okay folks, thanks for the feedback. I'll try to structure my thoughts first, then see how I can split them up.

I tend not to worry about deletion because I think it's just not possible

right, that's not an easy way, people inventing tombstones and dotted vectors and stuff like that

deleting a message via LMC in MAM sounds like a terrible idea to me. this leads to inconsistent state between clients, because they might not re-sync old entries.

jonasw: disagree

hm, you’ll have that anyways, no matter how you delete

but something feels off about that approach.

if you keep a delete marker you can synchronize easily

damn, this is done in almost every key-value store

but this is harder for clients to implement...

Prosody MAM / archive code is mostly modeled to assume an append-only ordered list of messages

Zash: just like twitter btw :)

it's normal practice in high-load

Altho it's actually an ordered key-value thing, but the key-value-ness doesn't do very much

ejabberd uses SQL, so I don't care in fact, I'm just thinking that it would be great to have append-only for some usages

LMC of messages before your last join are forbidden (via SHOULD) in the LMC XEP (0308)

jonasw, even if non-anonymous MUC?

the XEP does not distinguish

MUC doesn't tell you if historic messages are from the same senders as those currently present.

MUC-MAM might tho, I think

What's the current status of offline messages when a MAM-enabled client connects?

Will they be dumped to the client after initial presence, either way?

Add that to the list of things we should figure out how they should interact

Ge0rG: you can disable them using flexible offline message retrieval

zinid: xep 13?

yes

You implement that?

yes

we can probably recommend bare minimum implementation for servers

zinid: are there any clients making use of it?

Ge0rG: only ProcessOne customer's clients

Yay.

yes, we recommend doing this :D

because there is no other standard way

but of course XSF can invent something :)

The <purge/> command is prone to race conditions. Yay.

Ge0rG: And bind2 disables them.

Kev: I hope we can make bind2 part of the solution to the problems I'm outlining.

Ge0rG: yes, races are possible, but not fatal, the server will not send you new messages anyway and you receive them via MAM later (so you don't lose them)

Ge0rG: Yes, or something like it (e.g. Dave's 'wrap bind2 up in sasl2' proposal).

xmpp2.0 is a better choice than bind2, sasl2 and others :)

Is xmpp2 a concrete thing yet?

Or is it just the concept of "lets re-do all the things from scratch"?

not concrete of course ;)

there’s https://wiki.xmpp.org/web/XMPP_2.0

Ge0rG is making it concrete :D

Zash: I think I'm explicitly trying *not* to redo everything from scratch.

Kev: that'd make for a good disclaimer on that page

on top

bold

in <blink>

lol

drama

Is there anything on that page that's actually breaking compat enough to warrant a 2.0?

I mean, I imagine things like getting rid of 'jabber:client' and 'jabber:server' etc, and having a single unified namespace.

Kev: I'm trying to redo from scratch the minimum amount of things so we can fix up our shit already.

Zash: maybe it's only XMPP-IM 2.0

my goal is not to break everything but to fix everything :P

bind and sasl are defined in rfc6120, so this cannot be solved inside xmpp-im solely (unless we accept bind2 and sasl2)

zinid: bind2 can be a new XEP, without breaking anything actually.

Redoing the message routing is much more tricky.

Zash, I chose the name for that wiki page, and initially it wasn’t clear to me if we might have to override some ruling from the RFCs, basically warranting a 2.0

If you do things in the context of a negotiable stream feature, then I'm not sure I think that should be called XMPP 2.0

Zash, that’s just a name anyways

let’s call it a working title :)

The name is the most important thing about a project!

What about Session 2.0?

Ge0rG: I think you mean the logo, closely followed by the fancy website on a cool domain

The color is.

There's that old <session/> stream feature that's basically a noop everywhere I know

Ge0rG: there are other problems beyond message routing

Ge0rG: rosters, presences, all this shit is not modern anymore

Whut

Ge0rG, Session 2.0 may indeed be a reasonable name

Zash: popular IMs don't even have presences, there is no point in them when everybody is online almost always

I don't think any of that should be removed, but I'd like message routing to be detached from presence.

Just because "popular IMs" don't show something front and center anymore doesn't mean it doesn't have its uses

zinid: there is even a business use case for presence: synchronized DND

I didn't say it's useless, I said "not modern", meaning not used in popular IMs

I think lots of them have presence.

Slack, Discord, Whatsapp, Facebook all have presence.

And I'm not going to listen to arguments that boil down to "because popularity", if I cared about that I wouldn't be here.

There is real value in presence, and that's actually something that works pretty well on mobile, outside of initial presence flood from your roster when you connect.

you have two kind of presences usages in the clients, manual (set by the user) or automatic (set by the client itself)

yeah, but it seems like they transmit only "chat" status

zinid: do you happen to know who - if any - from the beam project is going to the mentor summit?

i.e. when I open an app, I become available, when I close the app I become unavailable

daniel: /me and Holger

zinid: I'm not going to the summit :-)

this is regarding ejabberd projects, I cannot say about other BEAM projects

ah

shit

I think Mickaël asked on the list and got no response.

I read it wrong way

So maybe nobody's going.

zinid: but you are coming, right?

no, sorry for confusion

I cannot leave russia, lol

That's too bad. Otherwise we would have had 3 xmpp related projects at the summit

maybe Mickael can go?

or Jerome? Chris?

but they are not mentors...

Trying to find his email.

If you haven't decided yet (and registered) then it's probably too late

are we talking about the FOSDEM summit here ?

But they don't have to be mentors

edhelas: gsoc

zinid: Ah got it. No response indeed.

oh ok

Holger: I don't even remember his mail

Have you never sent people to the summit?

It's a free trip to the US

zinid: It was on one of the 1,000 GSoC-related lists, maybe he forgot to add you to that one :-)

And each time you can roll the gitmo dice

no idea, I'm really not into this summit stuff, Mickael is usually managing this

daniel: Last year someone from some other BEAM project went.

zinid: gsoc-mentors@process-one.net on Tue, 19 Sep 2017 09:25:38 +0200.

Holger: I'm not subscribed ;)

Hah.

yeah, I'm a terrible mentor, I said that :)

I have a note: XMPP 2.0 Session Initiation The client and server must coordinate the required information for a sync as soon as possible: Authenticate Client sends a bind2 request containing: stream resumption id (optional, to faciliate Stream Resumption) last-known MAM message-id (optional, to achieve full synchronization) MAM request with a time-delta (optional, alternative to message-id if only the history of the last N time units is needed) initial presence (maybe?) MUCs to join (maybe?) The server automagically figures out whether to do a stream resumption or a MAM sync and provides according data to the client, generates presence broadcast accordingly

I think we're reinventing Matrix approach here in fact

not that it's bad

but what I see here is actually database synchronization between a server and a client

zinid: I think it's good to treat session setup as a way to tell the client everything that changed since the last connect.

zinid: exactly, xmpp based database synchronization.

Ge0rG: yes, but it looks overcomplicated, in normal situation you just need to send a point from where to download

and I also think MUCs and one-to-one chats should not be treated differently

like in matrix, hehe

not sure why we need stream resumption, the messages are in MAM already

seems redundant as well

zinid, don't forget the Pubsub/PEP synchronisation as well

that's wht we were looking into https://xmpp.org/extensions/xep-0312.html

zinid: because session setup is much more expensive than resumption, and you will lose outgoing messages without the latter

when I'm loggin in I'm spammed by hundreds of PEP/pubsub notifications for things that I already had before

edhelas: well my idea is to have a "routing block", it can be a room or one-to-one chat or a pubsub-eventer where you can subscribe and unsubscribe, that's it

edhelas: you're spammed because the desing is shit :P

the design, or the implementation :p ?

zinid: I'm not a fan of MUC MAM, but I try not to reinvent everything.

Maybe it's a good idea in the grand scheme to have all MUCs synchronized to all devices by default.

Announcement : Board & Council elections 2017 https://wiki.xmpp.org/web/Board_and_Council_Elections_2017

Alex, nice work :)

Alex: s/2014/2017/

fixed :-)

:)

now you can add your names :-)

:D

or recruit the people who you think will do a great job in those roles

Can we simply add those? 😈

Alex: yay!

It would be interesting to see "nominations"

Likely: Boardy McBoardface and possibly all guys from Matrix.

looking forward to see who runs for board and council.

I'd elect Boardy McBoardface. For Council.

A vote for Boardy McBoardface is actually a vote for David Attenborough, though.

how that?

Maybe because of https://www.change.org/p/jo-johnson-sir-david-attenborough-to-change-his-name-to-boaty-mcboatface

David McDavidface?

Darth Zash.

dwd, Kev: so I'm trying out a new presentation form: https://op-co.de/tmp/whats-wrong-with-xmpp-2017.pdf

when is this conference planned ?

edhelas: this is the slide deck for a fictive conference.

Ge0rG: nice list ;)

zinid: I knew you'd like it.

Ge0rG, "MAM + MUC = madness" why?

mathieui: because you have to query individual MUCs for their archives, there are different incompatible versions etc.

Ge0rG: but isn't this a good idea? to query only what you need, e.g. when you login you receive a list of mucs with unread messages number, you don't need to query them all, only when a user opens the chat window/tab

which reminds me I have to patch mod_mam_muc at some point too

?

Zash, muc archives are forever

which is bad

and also open to anyone, even someone not in the room

so you can scrape remotely any public MUC for its whole lifetime

zinid: maybe. Depends on whether you want a full-sync client or on-demand sync

mathieui: Pretty sure it respects members-onlyness

Zash, yes, it does

for members-only rooms, for which it indeed works like that

mathieui: I've considered making it optionally restrict access to eg members only, even in public rooms.

the best way would be a muc configuration toggle

See, madness!

That ejabberd MSN bug is really bugging me.

Oh, it's different now.

I see what you did there.

I didn't do anything!

can we re-trigger a website build to update https://xmpp.org/extensions/ ?

(two new XEPs)

Kev, can you give me that privilege in hub.docker.com please?

jonasw: Do you mean on Docker Hub or on EOS? It will happen automatically after a while I think; pretty sure Kev put it on a cron job.

I can trigger a rebuild on Docker Hub if that's all you need (I think?)

I put it in a cronjob

Sam, yes please

the cronjob will only check for a new docker image

oh, sorry, I lied, I don't have my credentials on this machine, sorry

jonasw: find a typo, submit a PR, I'll merge :D

Guus, add or remove a space? :P