XSF Discussion - 2019-03-11

jonas’, hey, I have sent you the protoXEP, I know you're busy, but maybe you will find some time to accept it for the upcoming agenda

zinid, thanks for the hint

lnj, Is qxmpp your project?

dwd: not its not, but I'm contributing to it

lnj, Any idea if the example GUI does MIX at all? I saw the library does.

dwd, are you working at any XMPP projects nowadays?

zinid, Trying to. :-)

zinid, Both Threads Styling stuff (lots of gatewaying into other IM systems) and Metre. I should be doing Openfire too (although I'm not really sure they need my help right now).

zinid, Current Threads thing is a standalone MIX implementation that I'm then building on to do our gatewaying interface. Hoping to get the MIX bits out as OSS for people to play with.

dwd: okay

good to know someone is doing MIX

dwd we just upgraded MINA, and you're wondering if we need your help? 🙂

dwd: I'm working on the MIX implementation in QXmpp, but it's not entirely finished and not all of my pull requests have been merged yet.

But all parsing / serialization of the main xep and xep 405 is working (in the unit tests).

Guus, That's what I mean, you clearly don't!

lnj, Ah, OK. For my purposes, just a real client I can use for exploratory testing would be really helpful.

dwd Imply what you want - we're still assigning all issues to you. 🙂

dwd We'd like to welcome you home. Kindly re-auto-join open_chat 🙂

Would you kindly

That'd be as if I'm giving him a choice. Nope. 🙂

We need our glorious leader back!

The fashion industry has stolen him from us!

So - if a XEP-0060 node supports XEP-0313, the archived items are the events sent out? What about retracts when notify is set to false - do these events which don't get sent get archived nonetheless?

dwd: a retract is not an item, it is the removal of one, so I'm not sure if it is appropriate to store in MAM.

ralphm, What's in MAM then, if not events?

I.e. if you request items from a pubsub node, you only get items, and if a previously existing item was retracted, it will not be included in the result.

ralphm, Oh, certainly. But if you ask for the Things from MAM, that wouldn't be items, but events? Or what?

dwd: this is a good question. Either the MAM archive is a record of the notifications (even if they weren't sent?) or it is an archive of items.

I kinda expected the latter, to be consistent with the pubsub items iq results.

ralphm, I thought it was a good questioin when I asked it.

It is

because we also have other notifications

like node deletion

ralphm, Yes, this is true. Are those also events (as in {http://jabber.org/protocol/pubsub}event)?

``` <xs:element name='event'> <xs:complexType> <xs:choice> <xs:element ref='collection'/> <xs:element ref='configuration'/> <xs:element ref='delete'/> <xs:element ref='items'/> <xs:element ref='purge'/> <xs:element ref='subscription'/> </xs:choice> </xs:complexType> </xs:element> ```

Oh. Gosh. Lots of types I hadn't thought about.

dwd, FWIW the pubsub stuff in MAM has never delighted me, for reasons like this. I've had a lot of feedback that it should at least be split out, but I haven't had time for that yet.

And items can include 'item' or 'retract'.

I don't really understand what it would be expected to return, but I suspect it would have to leave a tombstone for retracted items

MattJ, I would advocate splitting it out if it weren't for the minor point that there seems to be only one sentence of it...

No, there is more: https://xmpp.org/extensions/xep-0313.html#business-storeret-pubsub-archives

Oh right, that paragraph is a sentence

No, two

Since MIX critically depends on MAM and PubSub, this is something that requires a clear answer.

MattJ, SO that looks as if it's saying that it's the events that are stored, but the only mandatory event is the 'item' event (for publicaiton).

MattJ, One assumes, therefore, that other events might be permissible. If one squints enough.

ralphm, This, incidentally, is why I'm asking the question.

So really, events with <items><item/></items>

but with retracted items excluded

ralphm, Yes. Sorry, it seems today everyone has to use very loose parsing on what I'm trying to say...

ralphm, No, it mentions no limitations, just a requirement to include the item publishes.

MAM in turn also seems to kinda support the notion of messages being removed from an archive, without providing protocol for it.

(section 3.2)

It definitely doesn't support the notion of messages being *removed*

ralphm, Seems reasonable. I suspect there are a number of use-cases for siilently removing a message from the archive.

MattJ: second paragraph

MattJ, Blame Kev?

The second paragraph essentially explains that the entry is still there, but without a payload

But, reading XEP-313's 5.1.3, I definitely read that as grafting the MAM protocol on top of a pubsub item store and faking it

I.e. not actually requiring storing events, but constructing them as if sent to a subscriber, wrapped by the envelope.

Also note that it doesn't allow multiple items in one message, which /is/ ok for actual event notifications.

For what it is worth, I think this is fine.

The only gap I see is that a client may not ever become aware of the retraction of an item.

because if it happened to be offline when the notification got sent, and the item is 'emptied' (as MattJ suggests), there will not be a thing representing that deletion in the archive.

However, on the other hand, the retract event *should* be in the user's own archive, assuming the model where the server manages the users subscription (PAM).

And given that, it is sufficiently consistent to me.

I.e. if you were not yet subscribed, retrieving the archive just lacks the retracted items, and is the consolidated state of things.

(retrieving it from the pubsub node)

dwd?

And where did the text about having to return item-not-found for unknown ids in RSM come from? ✏

ralphm, No, I think that sucks a bit. Asking a remote source for MAM items should yield the same result as asking your local source filtering for the remote sender. I think.

Was there a DST switchover last weekend?

US changed, yes

\o/ first time in ... ever ... that I learned about this _before_ being late for a meeting.

dwd: why? The first in this case is the archive of the pubsub node, and the second is the archive of messages sent to you.

Guus: make sure you tell Arc

arc ^^

Lol

It's a DST miracle

(In other news, I think I can make MIX without MIX-PAM work reasonably well, which might make it less of a forklift upgrade)

And EU chickened out of abolishing DST :(

Zash, It may be the only positive from Brexit.

dwd: having MAM remember all the possible events sent to users for pubsub nodes is a bit terrible itself. E.g. it would need to record config changes.

Not even sure about subscription approvals

ralphm, Well sure. But for items, at least, recording old versions of items and retractions seems reasonable.

Zash wait what?

ralphm, Otherwise what's the point of having the archive?

dwd: why? The semantics of a publish request with the same item id, is that the previous item is obliterated and replaced with the new one.

ralphm, Besides which, it'd mean querying the MAM archive of the messages node of a MIX channel always gave you nothing. Boring.

If you do an items request, you don't get that either.

Guus: The EU considered abolishing DST changes, but chickened out and didn't give any requirements and "every country could do whatever they want", so it seems nothing changes and we get the stupid DST headache twice a year.

dwd: why would querying the mam archive of the messages node of a mix channel give you no items?

Zash fwiw, I thought that most countries would still go ahead and change it - although there might not be a uniform choice across Europe (and for those that do stop switching over, it's undecided if they'd go for summer, or winter time.

dwd: even though the notifications are sent as regular <message/> stanzas with bodies, there's no reason the message archive for the node to not return 'proper' constructed events as with every node.

ralphm, Because it's explicitly defined to not store pubsub items.

dwd: so that node is transient?

ralphm, See XEP-0369#4.7.2

Guus: So without coordination we might get a worse mess than now.

Zash perhaps

Zash, Same time for any switches that do happen, mind.

Don't most of the EU switch at the same time already?

dwd: wait, so in that section, which archive is being referred to? The archive of the messages node, or of the channel itself?

ralphm, It doesn't mention an archive there. But your proposal means that querying the archive for the MIX channel gives messages, whereas querying the archive of the messages node of the MIX channel doesn't.

Zash, They do, all, switch at the same time. The UK moved its changeover time, IIRC, by an hour, to match the rest of the EU.

And is mostly in the same timezone (except the uk and finland?)

And what's this about MIX and MAM?

Zash, And Portugal. Not sure about some of Eastern Europe either.

Zash, MIX, MAM, and '60.

Zash, But mostly, what's in a '60 MAM archive.

Don't you get all messages regardless of whether you have online resources?

.. to your account? So you could query them from there?

And without getting a copy per joined resource?

dwd: well, I'm not sure if it a proposal, but it is one or the other. Either the messages node is persistent, but sends out notifications as regular messages instead of pubsub events, or it is not a real pubsub node, and subscribing to it just indicates the desire to get the non-event-but-bare-stanzas-with-payload messages.

Zash no we get it four or more times a year because different parts of the world switch at different times

I meant within the EU tho

In the latter case, maybe having the archive be at the channel level, not the nodes it contains. I'm not sure how that interacts with hypothetical 'root node' subscriptions and their MAM archive.

Yea, that kind of thinking is problematic because it spreads. At a tech conferences here in the US, I keep hearing things like "xmpp? Don't they just use that in the EU?"

No, they also use it in the UK

Zash: I think you want to scroll back to http://logs.xmpp.org/xsf/2019-03-11#2019-03-11-fb946fc83908ade3

arc: you could reply with 'Fortnite'.

MattJ, Another 18 days until that works.

Adults don't understand fortnight

arc, Eve Online. Origin. NATO.

arc, Actually, your president doesn't understand NATO, at least. :-)

arc: adults are overrated

Not my president. He's only president for Nazis and rednecks

dwd: what about my message earlier?

ralphm, FWIW, I wanted the messages node to hold all messages. But I lost that one. My solution was to have MAM flags which could "condense" the items, for example by eliding retractions and corrected messages. I've been toying with the idea of such things to flag messages which have been acknowledged by '184, etc.

ralphm, But in any case, the MAM/MIX vs MAM/'60-on-messages-node is just a curiosity. Subscribing to the messages node with '60 syntax should/might give you a pubsub-syntax event stream of messages.

I am happy for something that condenses archived messages in general, although keeping in mind the discussion on multiple dimensions as discussed with Kev on the Summit.

ralphm, I don't remember the discussion being discussed.

ralphm, Did we discuss a discussion during the discussion?

However, I strongly believe that MAM on PubSub nodes currently are defined as a protocol to the item archive, not the message archive, and this is just a choice. I don't think right now that it is a bad one.

To cover the weird semantics of the messages node sending out notifications is a non-XEP-0060 syntax is something we could encode in the node configuration.

There is (some) precedent for this.

If it was an archive of events then you'd probably want a way to filter them for the data payloads

I remember that when Twitter supported XMPP pubsub (yes, really), it sent out atom elements and bodies in the bare message instead of using the event structure.

The goal was to make it easier for client devs to work with.

pubsub#include_body is pretty nice

yes and no

it would only cover the body

True

But we could define something similar

like

Including a picture would be neat in some cases actually.

a config item that represents 'bare notifications'

What do you mean by "bare"?

Zash: a notification without the pubsub event wrapper

Wait did they send `<message><{atom}item>atom stuff</item><body>someone tweeted hello</body></message>` without the pubsub container?

Zash: we are considering what notifications a user gets when subscribed to the messages node in a mix channel, right?

So at some point we said we wanted 'regular' messages to be sent as 'regular' messages originating in the channels' JID

However, you still have to subscribe to the 'messages' node.

For all other nodes, you'd get notifications with the normal pubsub event wrapper.

I'm afraid I haven't managed to read the MIX specs yet.

So if we want to change that for this purpose, you'd have to have a flag to represent this behaviour.

I remember discussions about that and thought containerless was the case already.

Zash: yes, but implicit in the sense that if you just throw a pubsub service onto the jid representing a channel, it doesn't work that way.

Zash: XEP-0369 now says something weird:

“The Messages node is used to distribute messages. The Messages node is a transient node and so no PubSub items are held. Messages MUST go to the associated MAM archive and history is retrieved by use of MAM. Users subscribe to this node to indicate that messages from the channel are to be sent to them. Private Messages are not distributed by the Messages node. ”

So this is not very clear on the details.

why isn't it clear?

I find it pretty clear

I implemented this part, didn't find caveates

zinid: if the pubsub item 'messages' doesn't hold items, what MAM archive does this text refer to?

pubsub node, I mean

I don't know, but that's how MIX designed

it's just weird

zinid: so instead of saying 'it is weird', we are trying to properly define it.

zinid

I didn't say the intent wasn't clear

I mean the sentence "as an instruction" is clear. we have tons of weirdness in our XEPs, I kinda don't pay attention already

whatever

zinid: so to retrieve the MAM archive for the channel's messages, where does the client direct it? The channel JID? With or without a node?

ralphm, to channel jid, no need for any nodes

zinid, +1

this node is kinda "ephemeral"

zinid: ok, I'll buy that, but that means that the 'messages' node is not really a pubsub node.

ralphm, We assume that directing XEP-0060 traffic to individual nodes gives you "classic" XEP-0060 stuff.

dwd: sure, I'm happy with that

but the text I quoted should be more explicit on this

ralphm, And FWIW, the "subscription" to nodes is fully mediated by the MIX channel itself, so it can do "special" subscriptions for the messages node if done through a join. WHich is what I'm doing right now.

Because even if it is a transient node, subscribing to it would normally yield empty notifications like this: <message from='pubsub.shakespeare.lit' to='francisco@denmark.lit'> <event xmlns='http://jabber.org/protocol/pubsub#event'> <items node='elsinore/doorbell'/> </event> </message>

dwd: maybe it would be better to say: there is no XEP-0060 style 'messages' node.

ralphm, Why empty?

dwd: right, good point. At least without an item id.

> the "subscription" to nodes is fully mediated by the MIX channel itself dwd, and that's the most weird thing of MIX. Back then when we implemented pubsub in ejabberd it was supposed to be a simple front-end to a database, now we introduce some "mediation" layer and the code is just useless now

ralphm, You're showing '60 Ex 3, there, which is explicitly a transient node that *also* doesn't have payloads.

so the complexity now is how to design internal pubsub handlers to work above any "mediated" layer

including database or MIX

ralphm, Whereas if you look at Table 4, bottom right is what we're expecting if you subscribe directly to the messages node.

Right, without item ids

zinid, Right - in my toy implementation, subscriptions internally are callable types (functor objects), and so MIX just uses a different callable type for join'ed message subscriptions.

ralphm, No, they just don't *have* to have message ids.

ralphm, Argh. Item ids.

dwd, sure, it's quite doable if you don't have a ton of code you need to redesign 🙂

ralphm, And probably wouldn't - there's a lot of complexity unspoken about whether the id on a message coming out from MIX is an item id in Pubsub terms or some new Event Id.

zinid, Yes, as well I know from looking at doing similar in Openfire etc.

dwd: it'd be better to say: the messages node is special, no notifications, no archive, you just indicate you want normal message from the channel and *it* has an archive

ralphm, I think it works anyway, doesn't it? We just need to say that asking for the MAM archive from the channel itself is a Different Thing.

ralphm, Noting, of course, that MIX implementations do nto have to offer very much (if any) classic XEP-0060 on the nodes within channels.

And that you don't get notifications from the node itself

Well, I definitely expect other nodes to be proper pubsub nodes

With retract, purge, etc.

Or at least all the normal mandatory stuff

hey guys, any input on my last ranting on standards@ list?

https://mail.jabber.org/pipermail/standards/2019-March/035857.html

nobody read? nobody cares?

I like the idea, haven't read the XEPs yet though

nice to hear, thanks

I stopped reading at "it just sucks"

oh, sorry

yeah, I shouldn't have said that

oh I agreed with that part :P

you're guys so fragile, I always forget 😀

whatever, scram sucks 😛

hmm, so I like the whole "cert auth" part, I hate the centralized CA bit

is the only reason for that "spam prevention" ?

moparisthebest, no, also RELOAD requirement

that's Sybil protection

https://xmpp.org/extensions/xep-0415.html#enroll-auth

first part

zinid: how do you protect from sybil attacks, again?

Ge0rG, by concentrating identities checks in a few single place, at CAs

note that for now we don't need the checks to be extremely severe, something like oauth to popular services or sms verification is enough so far

why is a sybil attack even a problem at all?

Ge0rG, in p2p?

because you can polute routing

in the CA thing.

aka Eclipse attack

sybil attack is not a problem is CA, CAs are just centralized to prevent Sybil

*in

idk, the CA looks like a total dealbreaker to me

moparisthebest, why?

maybe that makes it vulnerable to a sybil attack but probably worth it

I recall you like DNS, but DNS is ICANN - same centralized stuff

I don't know anyone or any organization I trust enough to run a CA

zinid: but how is a CA supposed to prevent sybil attacks?

I only trust certificates as far as the public key anyway, hence why I like DNSSEC + DANE etc

Ge0rG, I wrote already: > note that for now we don't need the checks to be extremely severe, something like oauth to popular services or sms verification is enough so far

moparisthebest, but what's the difference?

zinid: but that would mean that you essentially don't verify the identity of the XMPP entity but instead of whatever third-party service you use?

(actually, "in addition to")

Ge0rG, you can verify the entity of course by asking to provide the passport 😀

without CAs you don't have even that: you cannot identify all your online contacts that way

to clarify, I like the idea of identifying an XMPP *account* (not device, could be multiple devices) with a cryptographic key

I don't like the idea of a centralized CA approving that

zinid: I still think that you are conflating multiple different problem domains. There is value in having an XMPP based CA hierarchy, and there is value in whatever sybil attack prevention mechanism you might require for RELOAD. But those are completely separate

Ge0rG, sure they are separate

but I don't solve separate problems

I separate a single meta problem

zinid, if you are looking for "trust" then each domain should have it's own level of trust, which gets passed to it's users, imho

I'm not interested in our permanent bikeshedding with small problems without seeing a complete picture

that keeps it federated

moparisthebest, it's possible to do and I outlined that in XEP-0416

but still Sybil resistance is a problem and you CANNOT address it without centralization, and clever people proved that

then it shouldn't be addressed...

also, what's the point in federated accounts? why not roaming user profiles? we're moving that way in Moved for example

as you pointed out it's already basically centralized and could be solved that way through DNS

so we just rely on CA instead of DNS, both are centralized

if entire domains are trusted/not and excluded/not attackers already can't buy an unlimited number of those

moparisthebest, I can create 100500 accounts on another poorly maintained server and attack your server, what will you do?

block that 1 server

oh great idea 🙂

to block other users on that server?

no problem, they can use their crypto identity to move to another one :D

moparisthebest, how that?

where will they get the identity? how will they move?

from *abandoned* server

not to mention they will of course not move anywhere

they generate the identity, all their contacts have that key, when they get a request from $new_account with that same identity, they know they've moved

and you will trust the identity signed by that rougue server?

it's not signed by anyone

moparisthebest, also, can you please outline your concerns with CAs?

let's encrypt is bad?

1. assuming good intent and all that, commercial CAs get hacked/compromised all the time, what "XMPP CA" would even be as good?

2. I don't like assuming good intent, or being at the whim of anyone else

moparisthebest, and DNS servers are not hacked all the time?

and DNS assumes not intent?

no? also there isn't a single source of truth DNS server

except root servers?

whatever, what you say is possible if every XMPP server maintains CA

tied to their domains

but it sucks

also, good luck create roaming profiles in such system: why would I trust identities signed by some server X?

it's even moot that CA and much easier to compromise

*than

I don't think there should be CAs at all, only account keys, generated on an end device on account creation, sync'd to other devices

moparisthebest, and how will you verify those keys?

manually, or PGP web-of-trust style ? :P

sigh

okay, have fun with it