-
Seve
daniel, Matrix already does that. http://mirror.onet.pl/pub/mirrors/video.fosdem.org/2018/H.1309%20(Van%20Rijn)/matrix_webvr.webm
-
rion
looks cool
-
zinid
daniel: Psi gets crazy when it receives muc self-presence, it renders "empty" participant in muc roster
-
rion
O_o
-
zinid
yeah
-
rion
What server should I install to have such a self-presence? ejabberd?
-
zinid
rion, no, it's on local machine
-
rion
in any case last two day I rewrote muc roster. so it won't happen anymore. not commited yet
-
zinid
good
-
zinid
the problem is that now I'm not sure if other clients don't break though
-
zinid
rion, Psi requests MUC vCard on every join?
-
rion
yep
-
rion
I think with self-presence and caps I'll change this too
-
rion
and I have to improve avatars caching for mucs too as well as their requesting algo. otherwise it's a lot of traffic on join.
-
zinid
yeah...
-
zinid
poezio has the same problem wrt avatars 😉
-
zinid
gajim seems doing fine, doesn't render anything strange
-
Seve
By the way, regarding the email I sent to JUser, is there a better mailing list for that, please?
-
jonasw
I still haven’t subscribed to juser :/
-
Seve
That's why I would like to send it somewhere else too, seems not many people are subscribed there
-
jonasw
Seve, I think operators would be more fitting, maybe
-
jonasw
possibly with a cross-post to members@
-
jonasw
I’m also not sure if a separate MUC for this is a good idea
-
Zash
Posted something to some KDE list?
-
jonasw
frankly, I’m not sure XMPP can fulfil the requirements at this instant in time
-
jonasw
> Stickers are available
-
jonasw
we doomed
-
Zash
Find out which are requirements and which are wishlist
-
jonasw
Zash, they don’t even know that yet
-
jonasw
> Requirements are not prioritized (incl. must-have vs. nice-to-have), that comes at a later step.
-
Zash
jonasw: All are wishlist. There's online one requirement, that everyone you care about is already using it. :)
-
Seve
jonasw, the point of the MUC is to discuss this topic with people interested in helping out. Almost everybody here would find this discussion annoying here.
-
jonasw
Seve, I’m running out of screen space for more MUCs.
-
marc
Ge0rG, jonasw can you review https://git.zapb.de/xeps.git/commit/?h=fix_xep_0401&id=52725e993987f205dc253cc5a4e6937fe3955d81 and merge please?
-
jonasw
marc, EBUSY right now
-
jonasw
marc, if you refuse to make a github PR (which would really be the most useful thing for me), please send an email with the first line of the commit message as subject to editor@xmpp.org
-
marc
jonasw, I can do a GitHub PR if you like
-
jonasw
marc, that would be much appreciated, thanks
-
marc
jonasw, you're welcome
-
jonasw
gotta run now, see you
-
marc
bye
-
marc
done
-
jonasw
zinid, regarding the server capabilities invalidation, how about a message?
-
jonasw
server changes caps -> server sends <message/> to local clients. for server-to-server links, it could simply close the link so that the peer re-opens it, thus getting new stream features
-
zinid
jonasw, closing and opening thousands of s2s will create avalanche effect
-
jonasw
right
-
zinid
we see this on jabber.ru for examlpe
-
zinid
so I just think about nonza
-
zinid
stream-level element
-
jonasw
zinid, thanks for your input, I’ll post a few suggestions to the mailing list and I hope you’ll comment on them :)
-
zinid
sure 😉
-
SaltyBones
Is there a good resource on the problems with message ids?
-
jonasw
zinid, replied, happy to hear from you d)✎ -
jonasw
zinid, replied, happy to hear from you :) ✏
-
jonasw
SaltyBones, I’m not sure
-
SaltyBones
I'm looking at XEP-0359 but the problems mentioned at the summit are not in there.
-
jonasw
maybe the summit notes as first starting point
-
SaltyBones
1.2 Stable IDs Discussion on entropy ensues. dwd: <stanza-ids> don't work for unique addressing because we don't trust clients to do them properly. Kev: lots of possible attacks with spoofable stanza IDs.
-
SaltyBones
Not verbose enough.
-
jonasw
SaltyBones, I think the gist of that is: we need to generate stanza IDs on the server because weak/constructed stanza IDs are a problem. but the client needs to know the stanza ID for archive operations and possibly other things
-
zinid
jonasw, well, dunno, for me message looks too complex
-
zinid
can't we just send <c/> as a nonza?
-
jonasw
zinid, I think RFC 6120 would slap you in the face for that ;-)
-
jonasw
really, it needs negotiation
-
jonasw
it’s a shame that we have no way for a client to send stream features :(
-
zinid
yeah, because some implementation may close a stream if they receive unrecognized nonzas
-
jonasw
yupp
-
jonasw
I know at least one.
-
zinid
however, I tried to implement such behaviour (closing a stream) and got lots of problems and left the idea
-
jonasw
never had issues so far; did you encounter things which sent unsolicited nonzas?
-
zinid
alas, I've already forgotten what that was exactly 😕
-
jonasw
zinid, the best we could do is probably say "okay, if the client sent a presence update with caps, we can send a nonza with a caps update"
-
jonasw
and similarly for servers ("offered caps in stream feature -> send nonza")
-
jonasw
but I don’t know enough about s2s to be sure that this works
-
jonasw
this would definitely need a namespace bump though
-
zinid
well, we're talking about new caps xep?
-
zinid
we can do it there
-
jonasw
yeah, it requires a namespace bump there too ;-)
-
zinid
can't we just negotiate this feature?
-
zinid
not sure how
-
jonasw
negotiation will add a round-trip
-
jonasw
which people will react very adversely to
-
zinid
ah
-
jonasw
but please comment on-list, hoping to get more input on that
-
zinid
I don't know what to say
-
jonasw
what you said here, essentially?
-
SaltyBones
There are stanza-id and origin-id in XEP-0359. Somebody at the summit also mentioned "message-id" is that defined somewhere?
-
MattJ
They were probably referring to the id attribute, I guess?
-
SaltyBones
MattJ, this https://tools.ietf.org/html/rfc6120#section-8.1.3 ?
-
MattJ
Yes
-
SaltyBones
thx
-
MattJ
The reason XEP-0359 exists is because the id attribute is controlled (and can only be trusted by) the original sender of the stanza
-
MattJ
It's really just designed for tracking errors, though a couple of XEPs have re-used it for other purposes
-
SaltyBones
The id-attribute you mean?
-
jonasw
yes
-
SaltyBones
So, I guess, stanza-id is used by MAM and origin-id is used to detect MUC reflections (that's what 0359 says anyway). And id-attribute is used for errors but the error will have the same id-attribute iiuc, right?
-
jonasw
yes, pretty much
-
jonasw
except that origin-id won’t work with all MUCs
-
SaltyBones
Why?
-
jonasw
because not all MUCs may be able to reflect that ID for whatever implementation specific reason
-
jonasw
(just like not all mucs can reflect the message @id)
-
MattJ
There is only one implementation I'm aware of that doesn't (didn't?) reflect @id, and I'm not even sure of the current status of that
-
SaltyBones
Okay, so why does the standard not just mandate that it be reflected?
-
MattJ
Simple oversight I suspect
-
Dave Cridland
SaltyBones, Non-MUC things pretending to be MUC, in part.
-
Dave Cridland
SaltyBones, Also by the time people had noticed this was a problem, enough implementations didn't.
-
SaltyBones
So, could this be fixed by changing the standard and requiring that origin-ids be reflected?
-
SaltyBones
Dave Cridland, what do you mean by "non-muc things pretending to be muc"?
-
jonasw
SaltyBones, changing the standard doesn’t fix the implementations magically
-
MattJ
Dave Cridland, enough = 1? Also in the case of bridging to non-MUC, isn't it as simple as 1) if the non-MUC supports acks, wait for the ack and reflect the id or 2) reflect the id immediately?
-
jonasw
SaltyBones, IRC transports are "non-muc things pretending to be MUC" for example
-
SaltyBones
jonasw, not magically but ...
-
Dave Cridland
MattJ, More than one, I think. One classic MUC implementation, but quite a few transports.
-
Dave Cridland
FWIW, I've gone back and forth over whether MUC ought to reflect ids. It's a special case of maintaining the id on bradcast, which is itself both useful in some cases and bad in others.
-
SaltyBones
Why is this reflection necessary?
-
Dave Cridland
SaltyBones, Reflection isn't - we could add in a subelement which indicated the original id.
-
Dave Cridland
SaltyBones, Broadcast ids are harder to work around - there's a few protocols which use id as reference.
-
Dave Cridland
This all said I *really* dislike having a zillion ids as subelements of stanzas when we already have an id attribute.
-
SaltyBones
Well, what is the subelement necessary for? Is this just as an ACK to the client?
-
SaltyBones
What do you mean by broadcast-id?
-
MattJ
SaltyBones, if you're asking why MUC reflects messages to the sender in the first place (some systems, e.g. IRC don't) - it has a few benefits
-
MattJ
Such as ensuring everyone in the room sees the same messages in the same order
-
jonasw
it also is natural as soon as multiple clients are in the play, somewhat like message carbons
-
Dave Cridland
SaltyBones, So the id attribute in the message stanza for a groupchat message sent to the MUC can be reused in the reflected message, and/or the broadcast messages to other participants in the group.
-
MattJ
In IRC if two people send a message at the "same time", the messages will be shown in a different order on both their clients
-
MattJ
and yes, multiple clients from one user in the room
-
jonasw
it allows MUCs to do fancy things to messages and still have everyone see the same thing
-
Dave Cridland
(As a side-note, Matrix achieves a similar goal by a complex hash chain)
-
MattJ
SaltyBones, and also servers modifying messages (e.g. in the Prosody MUC we automatically convert long multi-line messages to a pastebin link). The reflection allows the sending client to see what everyone else sees.
-
jonasw
(in contrast to IRC, where you don’t see that your message was truncated)
-
SaltyBones
Okay, pretty convincing...
-
MattJ
Dave Cridland, and as many people at FOSDEM noted, much RAM
-
Dave Cridland
MattJ, Yes but BLOCKCHAIN.
-
jonasw
#bingo #triggered
-
SaltyBones
🔲⛓️
-
Dave Cridland
I hope there's a ZWJ there.
-
SaltyBones
*Yes, there is.*
-
SaltyBones
Okay, so we want reflected messages. Sounds fair. Let's suppose for a second that we can just make them mandatory in the standard and people would add them.
-
jonasw
they *are* mandatory
-
jonasw
the issue is with message IDs
-
jonasw
clients need to be able to recognize the reflection
-
SaltyBones
You mean id-attribute?
-
jonasw
or any ID really
-
jonasw
id attribute would be easiest
-
Dave Cridland
We could indicate in disco if ids were stable.
-
jonasw
wasn’t that proposal rejected-ish?
-
SaltyBones
Okay, 1. is the origin-id used for anything else and 2. is the whole point of origin-id not to detect reflection?
-
jonasw
speaking of MUC, I think the vote on https://github.com/xsf/xeps/pull/559 expired, didn’t it?
-
Ge0rG
I've attemtped to fix MUC reflected IDs some years back, not even in the normative language but merely by fixing the examples
-
Ge0rG
And got significant flack for it.
-
Ge0rG
I also attempted to make origin-id == message-id, but it was refused by the XEP author.
- SaltyBones groans.
-
Ge0rG
Now I'll just sit and wait, and ocassionaly proclaim: *I told you so!*
-
SaltyBones
So, is the point of origin-id to be used for reflection or is there something else?
-
Flow
SaltyBones, that is one use case
-
SaltyBones
Ge0rG, great, if you could also occasionally chime in with reasons for things and explanations that would be awesome.
-
SaltyBones
Flow, like?
-
Ge0rG
SaltyBones: it's mainly for reflection, except that MUCs are not guaranteed to keep non-body elements in the reflection
-
Flow
other use cases include finding your own messages in a archive
-
Ge0rG
SaltyBones: https://wiki.xmpp.org/web/XEP-Remarks/XEP-0045:_Multi-User_Chat#Matching_Your_Reflected_Message
-
SaltyBones
Flow, MAM uses stanza-id, right?
-
Flow
SaltyBones, it does
-
Ge0rG
SaltyBones: yes, but you don't know the stanza-id for the messages you sent
-
SaltyBones
Ge0rG, would you say this is a requirement by nature or just a problem with implementations?
-
daniel
i agree that the xep should specify that the sender should set origin-id=message-id
-
Ge0rG
SaltyBones: what?
-
Ge0rG
daniel: MUST set.
-
SaltyBones
Ge0rG, the fact that MUCs don't reflect non-body attributes.
-
daniel
yes
-
Ge0rG
because anything different is just a path into insanity
-
daniel
(i didn't mean SHOULD)
-
SaltyBones
Ge0rG, also, why do I need the stanza-id for a message I sent?
-
Ge0rG
SaltyBones: have a look at biboumi, a modern IRC transport. It doesn't reflect your message ID, it doesn't reflect any non-body elements and it mangles multi-line messages
-
Ge0rG
Welcome to message tracking hell.
-
daniel
i think thinks will already break if some client doesn't do that and then expects a delivery reciept or something
-
Ge0rG
SaltyBones: because when you ask for a MAM sync, you'll get yout sent messages copied to you as well
-
daniel
when sending messages to Conversations i mean
-
Ge0rG
Yes, message correction, delivery receipts and anything else that references IDs is a mess already.
-
SaltyBones
Ge0rG, and why is that a problem? They should have their origin-id and be recognizable, right?
-
Ge0rG
SaltyBones: you can't rely on origin-id being there, and you can't rely on it matching the message-id.
-
SaltyBones
daniel, what breaks when a client does what?
-
Ge0rG
SaltyBones: but yes, you can match MAM archives based on origin-id
-
daniel
if an origin id is set Conversations will use that as a reference in the receipt
-
daniel
so if your client doesn't do id=origin-id and expects the receipt for the id it won't work
-
daniel
same with everything else that references ids
-
SaltyBones
So, there seems to be consensus at least in this MUC right now that attribute-id should be the same as origin-id?
-
daniel
i'd argue there is no good reason not do make this a MUST
-
daniel
i mean most client would naturally do this anyway
-
SaltyBones
Ge0rG, who was the xep-author who was against this and do you remember why?
-
jonasw
SaltyBones, Flow
-
daniel
because why would you generate a second id if you can reuse the existing one
-
zinid
Who is generating origin-id? A client?
-
Ge0rG
SaltyBones: https://mail.jabber.org/pipermail/standards/2017-September/033415.html
-
jonasw
zinid, the original sender
-
daniel
zinid, yes
-
jonasw
i.e. most of the times a client
-
daniel
note that muc rewrites a irrelevant to the scenerio described above
-
Dave Cridland
jonasw, #559 expired, yes.
-
zinid
This is to track its own messages?
-
daniel
zinid, yes
-
daniel
or references to that
-
Dave Cridland
jonasw, But with no veto and the rest are +1, so apply it.
-
SaltyBones
jonasw, what is this ID-rewriting MUC shit? :)
-
daniel
some mucs are known to change the 'attribute' id
-
Ge0rG
SaltyBones: have a look at examples 44 and 45 in https://xmpp.org/extensions/xep-0045.html#message
-
daniel
so tracking your own message are parsing references don't work
-
zinid
daniel: why, you can rely on origin-id in this case
-
Ge0rG
If we have a MUC that rewrites message IDs, can we mandate that it also MUST rewrite references in all XEP payloads that reference message IDs, please?
-
Ge0rG
zinid: https://wiki.xmpp.org/web/XEP-Remarks/XEP-0045:_Multi-User_Chat#Matching_Your_Reflected_Message
-
daniel
zinid, in the case of muc? because most mucs won't remove child elements from the stanza
-
zinid
I'm lost
-
zinid
daniel: but that's exactly what's needed
-
daniel
now i'm lost :-)
-
zinid
So you can fetch origin-id and check if this is your id
-
zinid
Why do you need to rely on message-id if you inject origin-id
-
daniel
i just said the sender should set id=origin-id
-
SaltyBones
zinid, I think there was a sub-discussion about forcing message-id = origin-id
-
daniel
and/or deal with delivery receipts that reference the origin-id instead of the id
-
daniel
which is made easier if you id=origin-id
-
zinid
Ok, I don't get it 🤔
-
daniel
doesn't matter. that client to client stuff anyway :-)
-
zinid
From what I understand we just need to ditch IRC transports
-
zinid
😃
-
zinid
That's really their problem
-
daniel
oh yeah. or make irc transports track the messages themselves and re-add origin-id and stuff on reflection
-
daniel
it's probably better if the irc transport does the right thing(tm) than letting each and every client figure it out
-
zinid
daniel: agreed, nobody said writing transport is simple
-
daniel
irc transport should also reassemble the message if they previously split it and stuff like that
-
daniel
because they know best if they did
-
SaltyBones
Hm.
-
SaltyBones
Okay, are there any actual problems with the current state of three IDs except that we don't like having so many?
-
daniel
yes
-
daniel
read what i said before
-
SaltyBones
Uh, assuming a well-behaved server?
-
daniel
yes
-
daniel
my argumentation has nothing to do with servers
-
jonasw
daniel, IRC doesn’t even reflect
-
jonasw
so the transport generates the reflections on its own
-
SaltyBones
Okay, can you please point out what you are referring to then?
-
jonasw
biboumi however decided to reflect the split version of the message, and there’s some argument to doing that
-
daniel
assume i sent: <message id="1"><body>hi</body><request/><origin-id="2"></message> and then i will receive from conversations: <message><receipt id="2"></message>
-
zinid
who cares about IRC, really, are we designing a protocol for old nerds?
-
daniel
but we are talking about two different things. the how should irc transports behave has noting to do with the problem i'm describing
-
daniel
this applies to 1:1 chats as well
-
jonasw
daniel, yes, I don’t argue that
-
jonasw
I’m just replying to your message from "13:49:15Z" because I was away from keyboard.
-
daniel
the irc transport with the name i can not pronounce or remember does a few things that i don't like :-)
-
zinid
daniel: if we don't care about IRC then we probably don't need origin-id and thus there is no the problem you just described
-
daniel
there are non transport mucs which also rewrite the id
-
Ge0rG
daniel: https://lab.louiz.org/louiz/biboumi/issues/3283
-
daniel
zinid, if it weren't only for transports i wouldn't care
-
daniel
Ge0rG, is that an issue to change the name to something i can remember?
-
daniel
or pronounce?
-
zinid
daniel: if they are abandoned, then I personally give zero fucks
-
Ge0rG
daniel: no, it's about maintaining IDs on reflection
-
Ge0rG
zinid: everything in XMPP is abandoned. Now stop giving fucks.
-
daniel
i should probably write my own irc transport
-
edhelas
biboumi is not good ?
-
Ge0rG
daniel: you should just patch biboumis two or three warts.
-
daniel
i haven't used irc though ever since counterstrike 1.6 came out
-
zinid
Ge0rG: not everything, so I still have a few fucks to give
-
daniel
there are a number of things i don't like that seem to be design decisions rather than bugs
-
daniel
so i'm not sure if they even want me to change them
-
SaltyBones
daniel, so regarding your example, what you are saying is that the read receipt might reference the message-id or the origin-id and it is not properly specified which?
-
zinid
Ok, whatever, so what do you guys think about muc subscriptions?
-
SaltyBones
Or are you getting at the fact that origin-id requires "by=" and is therefore sometimes not applicable?
-
daniel
> daniel, so regarding your example, what you are saying is that the read receipt might reference the message-id or the origin-id and it is not properly specified which? This
-
daniel
not just read recepits but everything that references something
-
daniel
but yes
-
zinid
daniel: isn't there a place in the xep which says you should copy the message-id?
-
daniel
zinid, no.
-
zinid
Really?
-
daniel
that's what Ge0rG and I want to add to the XEP
-
daniel
that's pretty much what the entire discussion is about :-)
-
zinid
So add it 😀
-
daniel
i can't without permission of the author
-
zinid
No shit?
-
SaltyBones
So, there is some problem that I am still not aware of....
-
SaltyBones
Flow, why is the by attribute in the origin-id?
-
zinid
Who's the author?
-
SaltyBones
The one that causes the privacy problems...
-
Kev
Not entirely true, BTW, that it's impossible to do things without the author. But it's the path of least resistance.
-
daniel
i don't think there is a by attribute in the origin id
-
SaltyBones
Ah, sorry that is only for stanza IDs. origin-id does not have by.
-
zinid
Peter is the author
-
SaltyBones
That might almost always be correct buy 0359 is Florian Schmaus✎ -
SaltyBones
That might almost always be correct but 0359 is Florian Schmaus ✏
-
zinid
> In addition, it SHOULD include an 'id' attribute that echoes the 'id' attribute of the content message.
-
zinid
And you want this SHOULD to be a MUST?
-
SaltyBones
zinid, what some people here want is that origin-id = message-id
-
SaltyBones
Ge0rG, you had an objection to that in https://mail.jabber.org/pipermail/standards/2017-September/033415.html but I don't quite understand it. Why do you want to know if somebody creates strong message-id?
-
zinid
SaltyBones: but the sender controls this itself, so what is a problem to set them equal?
-
daniel
zinid, we want wording that tells the client developer to do this
-
daniel
there is no 'problem'
-
Ge0rG
SaltyBones: I thought it would matter, but it doesn't, because there always can be malicious entities
-
zinid
daniel: if they don't then they only harm themselves, I guess
-
Ge0rG
zinid: no, they harm the other participants
-
zinid
Ge0rG: ah
-
SaltyBones
How?
-
zinid
Yeah, how?
-
Ge0rG
zinid: also, it's well possible that stanza ids are generated by the xmpp library, and origin ids by the client, causing a mismatch
-
Ge0rG
By making all message references break
-
Ge0rG
We had that above already.
-
SaltyBones
Before we move to stanza-ids....
-
SaltyBones
There is nothing wrong with requiring origin-id = message-id?
-
zinid
If we require this then why origin-id is needed, wtf?
-
SaltyBones
zinid, it's possible that it was a mistake
-
SaltyBones
daniel, does requiring origin-id = message-id actually solve your problem or are there cases when stanza-id might be used to refer to a message instead??
-
daniel
It solves my problem
-
SaltyBones
daniel, wouldn't it be better to just mandate that clients always use the origin-id to avoid the problem of dealing with id-rewriting MUCs?
-
SaltyBones
I mean, certainly removing an unnecessary id would also be desirable but this might be a good intermediate step.
-
SaltyBones
Would everything be better if clients would generate proper stanza-ids?
-
SaltyBones
For ...uh...some definition of proper that makes them UUIDish.
-
zinid
they should be strictly monotonically increasing, so we don't need XEP-0198
-
SaltyBones
And by "would everything be better" I specifically also mean, wouldn't that allow us to ditch the other IDs?
-
zinid
and not UUIDs
-
jonasw
zinid, "strictly monotonically increasing" is not gonna happen
-
zinid
because?
-
SaltyBones
zinid, would increasing IDs really solve all the same problems as stream management? Seems unlikely?
-
jonasw
zinid, also, increasing IDs have security implications
-
jonasw
or rather: predictable IDs
-
zinid
SaltyBones, it solves in data replication, but of course XMPP is too unique
-
zinid
version vectors are based on such ids
-
zinid
jonasw, what security implications?
-
jonasw
zinid, I think there were some IQ response injection attacks based on predictable IDs
-
jonasw
even though in that case you probably already made the mistake of not verifying the sender of the response
-
jonasw
error injections would most likely work though because errors can come from entities different than the original recipient
-
zinid
jonasw, this can be fixed by adding routing information
-
SaltyBones
jonasw, maybe this is too much of an assumption for all of XMPP but don't we have transport encryption?
-
zinid
we anyway need this functionality already
-
jonasw
SaltyBones, that doesn’t stop me (jonas@zombofant.net) from sending an iq type="error" id="whateverIguessed" to="you" to break whatever you were doing
-
SaltyBones
zinid, I agree, if this is a problem I don't see why it cannot be exploited right now by somebody who randomly sees the message first
-
jonasw
SaltyBones, if I can guess the ID, I can attack you from off-path
-
jonasw
I can’t do that when I can’t guess the ID
-
jonasw
if I’m on path, you’re right, everything is lost already in XMPP.
-
zinid
but if you cannot match incoming errors against requests you sent then you should really consider to change the job
-
jonasw
zinid, how’d you match incoming errors against requests other than the ID?
-
jonasw
you can’t really use from, because as I said, it might come from an entity you didn’t know about yet
-
MattJ
jonasw, no?
-
zinid
jonasw, you can use (from, ID) I think
-
MattJ
errors come from the original recipient that you addressed
-
jonasw
MattJ, even if an s2s error causes an error?
-
MattJ
Yes
-
jonasw
oh okay
-
MattJ
Otherwise it would be a nightmare
-
Dave Cridland
jonasw, There's a "by" attribute, if I remember right, that tells you the generator/reporter of the error.
-
SaltyBones
So, I assume that if I run a h4xx0r server and try to send replies to messages that were not directed to me to some other server they will be discarded?
-
SaltyBones
And the same happens between client and server...?
-
MattJ
SaltyBones, by "replies", you mean error replies?
-
SaltyBones
Whatever magical interferring replies that jonasw was referring to :)
-
SaltyBones
yes, errors ;)
-
MattJ
For that to work you would have to know the original sent message id, and the original sender's full JID
-
SaltyBones
But if I did, I could?
-
MattJ
and then yes, you could send them whatever you wanted - it would be up to their client to match it up (or not) to one of its outgoing messages
-
SaltyBones
So, the statement that predictable IDs would allow me to spoof responses remains correct?
-
MattJ
so as said above, the client should use (from, id) to identify error responses, not just the id
-
SaltyBones
Because I cannot spoof "from"?
-
MattJ
Correct
-
SaltyBones
Yes, okay...
-
SaltyBones
That's what I was looking for. :)
-
MattJ
id "abc123" from userA@domainA is different from "abc123" from userB@domainB
-
SaltyBones
And servers only accept s2s with from=...@domainA if they know that the other server is in charge of domainA?
-
Dave Cridland
SaltyBones, That's what they're supposed to do, yes.
-
SaltyBones
^^
-
MattJ
SaltyBones, yes, they do
-
Dave Cridland
MattJ, No, they don't, but I love your optimism.
-
MattJ
If they don't, it's a bad bug or not an XMPP server
-
Dave Cridland
SaltyBones, So Metre (my S2S proxy mad thing) does this very strictly, and as a result I can't join this chatroom from my personal server.
-
Dave Cridland
SaltyBones, Not sure if MattJ is saying that's a bad bug in this server, or if he's saying it's not an XMPP server. :-)
-
MattJ
Dave Cridland, for what reason?
-
SaltyBones
Dave Cridland, so which messages do you get that don't pass?
-
SaltyBones
Bunneh, version xmpp.org
-
Bunneh
SaltyBones: xmpp.org is running Prosody version 0.9.12 on Linux
-
Dave Cridland
MattJ, It's sending a response down the stream obtained by reversing the stream to/from, and not by reversing the stanza to/from. I forget the details, Zash knows.
-
Dave Cridland
MattJ, I don't know if Prosody would accept that or not. Metre doesn't, Openfire does (but because it knows it's multiplexed on the outbound, so weirdness.)
-
Dave Cridland
MattJ, For maximum fun, Openfire used to do similar "implicit" auth on outbound, but I stopped that as from 4.2.0.
-
MattJ
I don't really understand, but maybe Zash will enlighten me
-
Dave Cridland
MattJ, Metre sends traffic for d.c.n -> muc.xmpp.org over a stream for d.c.n -> xmpp.org (after adding that domain via dialback).
-
Zash
How far back should I read to have any idea what you are talking about?
-
SaltyBones
15:58 should suffice
-
MattJ
SaltyBones, and a time machine
-
Dave Cridland
MattJ, Prosody (0.9) responds to the traffic on the reverse stream, xmpp.org -> d.c.n - which it hasn't requested authorization for yet.
-
SaltyBones
Do you mean a timezone machine or a MAM machine?
-
Zash
No MAM here
-
Dave Cridland
SaltyBones, For your amusement and mild confusion, XMPP authorizes streams by a 3-tuple of (from-domain, to-domain, direction).
-
Zash
Page up button tho
-
Dave Cridland
SaltyBones, Well. One or more of those 3-tuples anyway.
-
MattJ
Dave Cridland, who hasn't requested authorization for what? You mean Prosody connects to d.c.n as 'xmpp.org' and sends stanzas from 'muc.xmpp.org'?
-
Dave Cridland
MattJ, Yup, that.
-
Zash
The thing
-
Dave Cridland
MattJ, Like this: DEBUG 2018-02-12T15:14:35 /home/dwd/src/Metre/src/xmlstream.cc:95 : NS296914 - G ot [399] : <?xml version='1.0'?><stream:stream xmlns:db='jabber:server:dialback' xmlns:stream='http://etherx.jabber.org/streams' version='1.0' from='xmpp.org' t o='dave.cridland.net' xml:lang='en' xmlns='jabber:server'><iq id='472-452048' ty pe='error' to='dave.cridland.net' from='xsf@muc.xmpp.org/tim@boese-ban.de'><erro r type='cancel'><not-acceptable xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'/></e rror></iq>
-
dwd
This time I'm joined. Timing, I guess.
-
Zash
Uh, what was the thing with that again?
-
Dave Cridland
Zash, The bug itself? Responding to inbound traffic on a multiplexed stream by reversing the wrong domain pair.
-
MattJ
dwd, I don't see how timing would play a part
-
Zash
Dave Cridland: Do you remember where we discussed this?
-
Dave Cridland
MattJ, I think whether I can join is dependent on what streams are open, which is dependent on the session lifetime and activity.
-
Dave Cridland
MattJ, Not timing as in a race condition, mind.
-
Dave Cridland
Zash, Erm. 1:1 messages, I think, until we figured out it definitely wasn't a security issue.
-
Dave Cridland
Zash, Possibly some discussion in jdev.
-
dwd
FWIW, I have to put in something similar to an implicit auth for X2X, anyway. So this may provide a workaround.
-
Zash
MattJ: https://hg.prosody.im/trunk/file/0de0018bdf91/plugins/mod_s2s/mod_s2s.lua#l203
-
MattJ
Zash, exactly where I ended up
-
Zash
stanza from/to may differ from the stream to/from in case of dialback multiplexing
-
Dave Cridland
MattJ, You switching to FreeBSD?
-
Dave Cridland
MattJ, https://svnweb.freebsd.org/base?view=revision&revision=329166
-
zinid
svn o_O
-
edhelas
gitis too mainstream
-
zinid
capitalist pigs use git!
-
Zash
gititis
-
Holger
NetBSD has Lua in the kernel for years!
-
Zash
Dave Cridland: Heh, nice
-
Holger
https://www.phoronix.com/scan.php?page=news_item&px=MTMwMTU
-
Holger
And it has CVS :-)
-
zinid
damn, I only get used to svn...
-
Dave Cridland
Holger, Can you download NetBSD yet, or is it still only available on tape?
-
zinid
Holger, cool stuff btw
-
SamWhited
More importantly, once I download it can I run it on any machines made after 1995? (this is always the problem I had trying to run NetBSD; a few of the ops people at work run it, but they all use *very* old machines)
- Dave Cridland gets prepared to explain "tape" to the younger readers.
-
Dave Cridland
SamWhited, Sure! It runs pretty well even modern machines like the Amiga 4000.
-
Dave Cridland
(Sorry, that was 1993).
-
SamWhited
Exactly.
-
Holger
Well yes it's about as dead as XMPP :-P
-
SamWhited
I would like to use at *least* a P3.
-
Holger
But it always worked okay-ish for me on non-recent ThinkPads.
-
Holger
I no longer do that though.
-
Holger
cpu0 at mainbus0 apid 0: Intel(R) Core(TM) i3-2120T CPU
-
Holger
... is the one box I still run NetBSD on.
-
Zash
Uh, was NetBSD the one being difficult, or OpenBSD?
-
Holger
Difficult?
-
SamWhited
Actually, jokes aside, that might be a nice thing to do with my broken old thinkpad; doesn't work very well as a laptop anymore, but it could be a {Free,Net,Open}BSD or Illumos "desktop".
-
Holger
Zash: Isn't all computers difficult except for Apple?
-
SamWhited
This is true… ⤴
-
Zash
Et Apple, Holger
-
Dave Cridland
I find Apple incomprehensibly difficult to use, I must admit.
-
Dave Cridland
I've yet to figure out how to go up a directory consistently in file dialogs.
-
Kev
cd ..
-
Kev
Happy to help.
-
Holger
Managing FreeBSD feels more or less like Debian to me, the two others feel a bit more basic but not really harder to use. All dead simple compared to the dances you had to go through when partitioning hard disks or getting X11 to run 20 years ago with any Linux or BSD.
-
Dave Cridland
Holger, Ah, X11. Kids these days don't know they're born.
-
Dave Cridland
Kev, Don't get me started on the antiquated command line tools Apple foists upon you.
-
Holger
Yes, we're horribly old.
-
Dave Cridland
Holger, We're very experienced. I'm sure that's what you meant.
-
Zash
I'm sure wayland will bring back the good old times for those who miss configuring Xorg
-
Holger
Precisely.
-
SamWhited
The only thing I want out of a machine is to sync an old ipod without Rhythmbox crashing…
-
Dave Cridland
SamWhited, Rhythmbox never crashes.
-
Dave Cridland
SamWhited, At that speed it's called "parking".
-
SamWhited
Rhythmbox is one of those people who parallel parks by backing up until they hit the car behind them, then driving forward until they hit the car in front.
-
SaltyBones
So, why is it that a client cannot create the stanza-id?
-
SaltyBones
Is this a thing about malicious clients or what's going on there?
-
jonasw
SaltyBones, no, unaware clients are sufficient; colliding IDs would lead to interesting issues with MAM
-
SaltyBones
jonasw, but if the client is only unaware the server could just return an error informing the client that this ID has been used.
-
Dave Cridland
SaltyBones, But we could use, say, stream-id + attribute-id to form a stable, non-colliding reference identifier.
-
jonasw
SaltyBones, that’d be annoying to handle in the client, and the server would need an O(1) (or similar) way to determine that the ID has been used already...
-
SaltyBones
Yeah, or we could hash things and god knows what else...it seems very solvable...
-
Dave Cridland
SaltyBones, I mean, unless the client isn';t generating unique ids within its stream, in which case it's presumably not going to be fixed to use some other identifier.
-
jonasw
Dave Cridland, stream id + sm-counter?
-
jonasw
that’s verifiable by the server
-
jonasw
and predictable for both
-
Dave Cridland
jonasw, True. But I think we had some ideas on predictability outside the stream?
-
SaltyBones
throw a hash on top to avoid privacy issues, done
-
SaltyBones
Dave Cridland, I thought we dicussed those into oblivion earlier? :)
-
Dave Cridland
SaltyBones, Really I've lost track.
-
jonasw
Dave Cridland, hmmm ... hmac(stream-id, sm-counter)?
-
Dave Cridland
SaltyBones, I have the feeling that nobody quite knows the entire picture here.
-
SaltyBones
I almost lost track and I basically didn't work at all today and just discuss here. :p
-
Dave Cridland
jonasw, HMAC() requires a secret to be of any use.
-
jonasw
Dave Cridland, stream-id.
-
SaltyBones
Seems reasonable on first sight...
-
jonasw
(post-TLS obviously...)
-
Dave Cridland
jonasw, Is that secret? And why HMAC over a hash?
-
SaltyBones
Oh, that would leave out the hash...
-
jonasw
Dave Cridland, I’m fine with hash(stream-id || sm-counter) too, but that’s nearly an hmac ;-)
-
SaltyBones
hmac basically IS a hash
-
SaltyBones
yeah :)
-
Dave Cridland
jonasw, I'm pretty sure it's not. :-)
-
SaltyBones
I am very sure it is. :)
-
jonasw
that’s why I said "nearly"
-
jonasw
I think the concat is slightly different
-
SaltyBones
(yes, nearly)
-
SaltyBones
https://wikimedia.org/api/rest_v1/media/math/render/svg/4edcf0bd8b403c93564b8d7ea91338b3208dea03
-
jonasw
doesn’t matter anyways
-
Dave Cridland
jonasw, An HMAC is two nested hashed with concats and masks, yes.
-
SaltyBones
Okay, so we only disagree on our definition of nearly. ;)
-
Dave Cridland
jonasw, But the security properties are distinct.
-
jonasw
Dave Cridland, I don’t argue that
-
jonasw
(and I’m aware)
-
SaltyBones
Anyway, the suggestion of HMAC(key=stream-id, msg=sm-counter) seems good, doesn't it?
-
Dave Cridland
In any case, given a stanza with a known id on a given stream, we clearly want to be able to predict the MAM id.
-
SaltyBones
I mean, I am also fine with SHA-*(stream-id || counter-sm)
-
SaltyBones
Dave Cridland, but mam-id = stanza-id which is what I am proposing to set to this...
-
Dave Cridland
Question is, do we think the MAM id is likely to become public, and if so, can someone relatively easily figure out the next (or previous) value, and then do Something Bad?
-
SaltyBones
Dave Cridland, that might be the question bit if it only costs us a hash per message to not answer it might also not be...✎ -
SaltyBones
Dave Cridland, that might be the question but if it only costs us a hash per message to not answer it might also not be...✎ ✏ -
SaltyBones
Dave Cridland, that might be the question but if it only costs us a hash per message to not answer it, it might also not be... ✏
-
Dave Cridland
What I'm wondering, you see, is whether we give clients an algorithm to generate attribute-id and then let them signal to the server that they're going to use globally-unique attribute-ids and the server is allowed to call them out if they don't.
-
jonasw
Dave Cridland, okay, right, this is about the MAM ID. this *probably* doesn’t matter, but if the goal is to consolidate all IDs at some point, choosing a way where the ID can become public is safer
-
Dave Cridland
jonasw, That's roughly my thought, yes.
-
jonasw
and at that point, HMAC(…) seems like a sane choice.
-
Dave Cridland
jonasw, Well, that and "But we *HAVE* stanza ids - right there in the stanza!"
-
Kev
How would the server know that they didn't use a globally unique id?
-
SaltyBones
Kev, if the server can simply re-run the generation algorithm and compare results..?
-
jonasw
Kev, if the ID doesn’t follow the algorithm for a globally (predictable, for the server and client only) unique ID
-
Dave Cridland
Kev, Well, it'd know if they used no id at all, and if it saw a collision within a stream.
-
Kev
If we only cared about collisions within a stream, we could trivially solve all issues already.
-
SaltyBones
And that is how you can tell that Kev is a VIP
-
SaltyBones
when he says something three people reply!
-
Dave Cridland
Kev, Well, we care about collisions for a user's account for MAM, and no further.
-
Kev
But having a useful id to refer to a message would be jolly useful, and I don't see how we can get there yet.
-
SaltyBones
Okay, so you're saying when two servers federate and you assume one of them is malicious how can the other make sure he doesn't get duplicate IDs?
-
Kev
Or non-malicious server with a malicious client.
-
Dave Cridland
SaltyBones, Do they care? ANd if not, should they?
-
jonasw
SaltyBones, you don’t use other servers stanza-ids in your own MAM
-
SaltyBones
Well, the malicious client would be detected by the server.
-
Kev
jonasw: Bloody useful if you could, though, no?
-
Kev
SaltyBones: How would the server know it's malicious, if it doesn't know all globally generated ids?
-
SaltyBones
Kev, it can simply check if the client generates their IDs correctly...
-
Dave Cridland
Kev, Again, why would a server care?
-
Kev
Without exploding the size of IDs, at least.
-
jonasw
(we should mandate the use of shakespeare-inspired adjectives in each bloody sentence in this room, by the way)
-
jonasw
Kev, how would it be useful?
-
Kev
I'd like to send a reference to a stanza, where the reference makes sense in my archive, the MUC/MIX archive, and the user's archive.
-
Dave Cridland
jonasw, I don't think "bloody" is Shakespeare. "Submarine" is, though, as far as I remember.
-
SaltyBones
Kev, you simply make it a hash or hmac of something that both the client and the server know and the server can check the calculation.
-
Kev
"simply" :)
-
SaltyBones
Kev, but doesn't that only involve you, the muc/mix server and $other_client? i.e. only one server...?
-
Dave Cridland
Kev, Is the (from, id) attributes of the stanza sufficient? If not, why?
-
Zash
If this is for MAM purposes, then I will cry if I can't stick some time based bits into the ID
-
Kev
Dave Cridland: No, because the from gets rewritten.
-
Dave Cridland
Kev, When?
-
SaltyBones
Zash, what?
-
Kev
In a MUC or a MIX.
-
Zash
My MAM ids are basically yyyy-mm-dd-random
-
jonasw
Kev, those are different use-cases I think.
-
Kev
jonasw: If we cherry-pick the trivial use cases, it's going to be trivial to solve them :)
-
jonasw
Kev, this is where origin-id makes more sense. It is generated by the client. If the client doesn’t ensure that there’s enough entropy in there, references to their message won’t work.
-
SaltyBones
Zash, and this is important because?
-
Dave Cridland
Kev, Ah, so given a message from a MUC fanout, do you want to be able to identify the originating id? I'm not actually sure you always want to allow that.
-
SaltyBones
I still don't get it. Why does from get rewritten?
-
SaltyBones
And what does that have to do with anything? :D
-
jonasw
SaltyBones, when you send a message from saltybones@yourdomain.example/client-resource to this MUC, we see the message from xsf@muc.xmpp.org/SaltyBones
-
jonasw
so the from is being rewritten.
-
jonasw
and if references are based on (from, id) pairs, you can’t use the same reference we all can use for your message
-
jonasw
(simplified, of course we could still refer to the reflection of the message, but that would break down with MIX, too, I think)
-
Kev
jonasw: Well, no, because if you can fake someone else's id, suddenly you can maliciously change the target of a reference.
-
jonasw
Kev, good point. so we indeed need something like (from, id) :(
-
SaltyBones
Wait what?
-
zinid
lol
-
SaltyBones
How can you now fake stuff again?
-
jonasw
but given that all our fanouts and from-rewriting things do actually do reflections, I’m not sure if that’s so bad, Kev?
-
jonasw
your local archive woudl still be able to resolve everything
-
Kev
jonasw: Well, it enforces a round-trip before you can refer to or correct anything, which isn't ideal.
-
jonasw
Kev, you normally know the "from" you’ll be having.
-
jonasw
and since you set the origin-id yourself, you already know everything you need
-
jonasw
(meh, races with server-enforced nickname changes in MUC)
-
Kev
Point.
-
jonasw
(but I guess those will be rare enough for us not to care)
-
SaltyBones
halp! :(
-
Kev
SaltyBones: Because there are potentially malicious entities.
-
SaltyBones
Are these only clients or servers as well?
-
Kev
jonasw: Still doesn't help where a user generates the same id twice, I think, and you're left with ambiguous references/corrections/whatever.
-
Kev
SaltyBones: Both.
-
SaltyBones
So why do you connect to a malicious server and expect things to work?
-
SaltyBones
Maybe that's not a good approach.
-
jonasw
Kev, that’s not an issue. put 256 bits of entropy in there and you are officially allowed to not care
-
Kev
Ah, if you have a mechanism for identifying malicious intent on the Internet, you could be a very famous person.
-
jonasw
Kev, and if we want to have defined behaviour in that case, always assume the most-recent message
-
Kev
jonasw: That only works for accidents, rather than manipulation, no?
-
SaltyBones
Kev, but you cannot manipulate as long as the server is checking.
-
jonasw
Kev, indeed
-
jonasw
but if you manipulate your own origin-ids, it’s your own fault?
-
SaltyBones
Of course if we have malicious servers now a bit more work is required. :)
-
Kev
And I'm concerned that 'most recent' falls apart if you can manage different people receiving different subsets.
-
jonasw
can one inflict harm by producing duplicate origin-ids?
-
jonasw
I’m not convinced that this is actually an issue.
-
Kev
Most likely.
-
Kev
If I can cause the 'wrong' message to get corrected.
-
jonasw
maybe with something like Reactions based on (from, origin-id) references.
-
Kev
Or a like to apply to the wrong message.
-
Kev
If I can manipulate you into liking a Godwin instead of something sensible, that's not ideal.
-
jonasw
meh
-
SaltyBones
But messages aren't authenticated anyway, if you are a malicious server you can claim to have received whatever likes from me...
-
jonasw
Kev, then the only way is a MUC/MIX stanza ID generated by the MUC/MIX and waiting for a round-trip before references can be done.
-
Kev
SaltyBones: Other way around.
-
jonasw
SaltyBones, all it needs for now is a malicious client though
-
jonasw
SaltyBones, or a malicious client+server pair if we do the hmac-stanza-id thing for origin-id
-
jonasw
I can still run my own server and make it fake origin-ids
-
zinid
your own malicious server?
-
jonasw
Kev, I’d however argue that actively, from a client/own server side, make participants in a MUC/MIX receive only parts of the conversation would be rather trikcy
-
jonasw
and targeted parts even
-
jonasw
and that without them suspecting that things are broken in funny ways and thus not trusting the system anymore
-
Kev
jonasw: Ah, that's ok then, no exploits have ever involved doing anything tricky :)
-
jonasw
Kev, I see your point, but I’m not sure this is actually something which is reasonably exploitable.
-
jonasw
but yes
-
jonasw
unless we let the ID be generated by the fanout service, there’s no way to be sure I’m afraid
-
jonasw
I mean, I don’t have an issue with that, I’m fine with that even.
-
Kev
Yes. I don't see a way of solving this, which was why I brushed it under the carpet at the Summit.
-
jonasw
complicates things though
-
jonasw
Kev, I’m having a really hard time constructing a successful attack which wouldn’t be seen by the victim in the MUC case
-
jonasw
even when I omit arbitrary messages
-
jonasw
to only some participants
-
jonasw
(which would be really really tricky to achieve targetedly I think)
-
jonasw
ah, now I have it
-
jonasw
this is the scenario: 1. user A, "bad statement", origin-id=1 2. user A, "good statement", origin-id=1 3. user B, like (from=user A, origin-id=1) if (2) isn’t seen by all users, they see user B liking "bad statement" instead of "good statement"
-
jonasw
with an arbitrary amount of messages between (2) and (3), it is also not too difficult to make people not see (2) in a MAM-less MUC.
-
Kev
Indeed.
-
jonasw
I don’t think there’s a way unless the MUC does things there
-
jonasw
(i.e. if the MUC generates the ID?)✎ -
SaltyBones
But, all it requires to prevent that is for the MUC to check that the ID is unique just like the assumed-non-malicious server did in our previous discussionn...
-
jonasw
(i.e. if the MUC generates the ID) ✏
-
jonasw
SaltyBones, but that is a rather expensive check to do
-
jonasw
you need to record IDs for all eternity, or have a defined way to generate the ID which can be executed by clients
-
jonasw
the latter would be tremendously tricky to get to work synchronously
-
jonasw
but possible...
-
SaltyBones
Which is a little more tricky because there is no obvious common "secret" like the stream-id but a simple Hash("counter") would do
-
jonasw
something like hash(message-counter || presence-id), where presence-id is an ID assigned to the client on join
-
SaltyBones
Kev, not that in this case I used the word simple again but I was totally lying...✎ -
SaltyBones
Kev, note that in this case I used the word simple again but I was totally lying... ✏
-
jonasw
hash counter doesn’t really work with multi-session nicks I think, it would lead to collisions or races
-
SaltyBones
jonasw, yeah it was just to get the idea. Indeed you would at least need some salt.
-
SaltyBones
But if the server gives you the salt on join and you use that to generate the IDs it should be good again.
-
jonasw
that would be verifable by the MUC and the MUC would drop stanzas which do not adhere to this schema
-
jonasw
(or rather, reject)
-
SaltyBones
and since the counter can be per-muc there is no issue with having it
-
jonasw
per client even
-
SaltyBones
I think :p
-
jonasw
Kev, ^
-
SaltyBones
well, yes per client and per muc
-
SaltyBones
just saying it doesn't seem to be a privacy issue
-
jonasw
SaltyBones, yeah
-
jonasw
so origin-id = one-way-function(presence-id, message-counter), where presence-id is assigned on MUC join
-
jonasw
only need to define what happens when message-counter wraps around or becomes large or something
-
jonasw
(same thing for SM by the way, SM has wraparound semantics)
-
SaltyBones
O_o
-
SaltyBones
Y U SEND SO MANY MESSAGES!?!!
-
jonasw
SaltyBones, itym "y u have so long sessions"
-
SaltyBones
still, if this is defined somewhere we can probably renegotiate the salt at that point...
-
jonasw
while wrapping around a 64bit counter in a single session is sure challenging, we need to be prepared if this is to be solid :)
-
jonasw
yeah
-
zinid
I hear counter?
-
zinid
sorry, I don't track the discussion
-
jonasw
zinid, I’m not going to repeat everything you can read in the backlog :)
-
zinid
as you wish
-
zinid
not that I care
-
SaltyBones
counter works in this case because you can restart counting when rejoining the muc
-
zinid
whatever you will end up will be shit anyways, so
-
zinid
you already created 4 fucking ids: stanza-id, origin-id, attribute-id and SM id
-
zinid
maybe it's time to stop and think?
-
jonasw
what is an sm ID?
-
zinid
h
-
jonasw
right
-
jonasw
zinid, all of this is about reducing the number of IDs, so I thnik this is kinda what we’re doing?
-
zinid
jonasw, from what I see you want to add yet another counter
-
jonasw
zinid, to generate origin-ids, yes
-
jonasw
for MUCs and MIXes
-
jonasw
to make them verifiable by the service
-
jonasw
zinid, or rather, we’re replacing origin-id by some one-way-function(counter)
-
zinid
how this will cover sm id?
-
SaltyBones
We haven't discussed sm-id, yet. Only the other three.
-
SaltyBones
I actually have no clue what sm-id is. :)
-
zinid
SaltyBones, then I need to wait while you recognize the problem 😉
-
SaltyBones
zinid, so reducing 4 -> 2 is not enough, eh? :p
-
jonasw
zinid, sm-id stays, but stanza-id (and attribute-id) could potentially become one-way-function(stream-id, sm-id)
-
zinid
2 is enough, however I think first should be counter and second should be routing information, and not an ID
-
jonasw
stanza-id being used for the archive only, origin-id being used throughout the network together with the sender address for refernecing a specific message
-
zinid
what ID will be used to sync?
-
zinid
with SM IDs you just create a pointless queue in c2s state
-
SaltyBones
Okay, I think we are trying to solve a problem that is very orthogonal to stream management. But we have only discussed this a bit and it might very well not work even for what we want it to do.
-
zinid
SaltyBones, I don't think SM should be separated from archive
-
zinid
it's pointless
-
SaltyBones
Why?
-
jonasw
zinid, stanza-id is used for ysnc
-
jonasw
(for MAM sync that is)
-
zinid
SaltyBones, because why you need this separation?
-
zinid
SaltyBones, you keep messages in MAM and then put them in c2s SM queue
-
zinid
why can't you just inc(counter), store in MAM and send via c2s?
-
SaltyBones
Why are you asking me that? I am 100 % sure that you know more about SM than I do!
-
zinid
what I know about SM is that I need to maintain stupid queues inside c2s processes, which sucks as hell
-
zinid
even though a client can request those messages via MAM
-
zinid
if you receive an ID out of order, you just reconnect and ask everything started from the ID you received
-
zinid
and server will send you this from archive
-
zinid
and vice versa
-
SaltyBones
But SM-IDs are per-stream not per-message, right?
-
SaltyBones
At least that's what it looks like in https://xmpp.org/extensions/xep-0198.html
-
zinid
when you connect a server, you provide last seen ID and it will resend you everyting what's greater than this ID
-
zinid
SaltyBones, yes, they are totally separated instances
-
SamWhited
What if the thing you missed was an IQ or a presence that isn't stored in MAM?
-
SamWhited
If you temporarily disconnect and miss something, SM acks allow you to find out. Doesn't help as much if it only covers messages.
-
zinid
SamWhited, I think we can drop them
-
zinid
SamWhited, we already don't care about IQs with Push, so why would we start care?
-
zinid
try to make jingle call when I'm in "push" mode
-
SamWhited
Does that not work? That does seem like something we need to care about to me
-
zinid
SamWhited, but we don't and we cant with push stuff
-
jonasw
shouldn’t an IQ trigger a push? :-O
-
zinid
SamWhited, what if I want to receive your software version and you're in push mode?
-
SamWhited
That seems like a problem that needs fixing then, not an example of something being done right that we should copy.
-
zinid
anyway, you can keep IQs in MAM if you prefer
-
zinid
you need to keep subscription requests for sure there
-
zinid
we keep them already, but in a separate database due to historic reasons only
-
jonasw
IQs (which are inherent request-response, with exactly one response) in MAM sounds like a terrible idea.
-
SamWhited
yah, now you have to try and figure out which IQs are time sensitive, which are important to store, etc. it seems like a comlicated way to work around having a stanza counter.
-
zinid
SamWhited, but you have to decide now too: for example, after 5 minutes of inactivity (by default) a server just bounce all IQs from SM queue
-
zinid
*bounces
-
SamWhited
I'm not saying it's perfect, just that this doesn't seem like a good fix.
-
zinid
a fix? the behaviour is the same
-
zinid
but we let a client to decide which IQ to reply
-
SamWhited
Except now you have to store all stanzas in MAM, or not store some stanzas and risk those being the ones that are dropped and you don't know you missed something. The point of SM is to make sure you know if you lose something, which is often important.
-
zinid
do you really want to loose incoming call?
-
zinid
that's how it works now: if somebody calls you and you're not connected you loose any track
-
SamWhited
As I said, I'm not claiming that the current solution is good; just that this makes it worse.
-
zinid
no, it makes it better
-
zinid
you just need to define what to store and what not, well, it's too much to redesign, yes
-
SamWhited
If you don't store everything you now lose the ability to detect connection drops though
-
zinid
sigh
-
zinid
not sure how will you address missing call though
-
zinid
with your approach
-
SamWhited
I do not have an approach; I agree that is a problem that needs solving though.
-
SaltyBones
It seems like we need storage for management messages and storage for actual messages. I don't see why these should be mixed up.
-
zinid
we need to decide what to store and what not
-
SaltyBones
hm..that distinction was wrong, yeah
-
SaltyBones
there should be a storage until delivered and an optional long term storage
-
SaltyBones
users might not even want mam :p
-
zinid
if you don't want mam then just lose messages
-
SaltyBones
Okay, let me rephrase, some users might not want long-term storage of messages...
-
zinid
not sure why this should be specific to any approach
-
SaltyBones
Because I think that's what MAM is intended for and that's why using it for all messages as you suggest is strange and might have unexpected results if people built it under the assumption that it is for long term storage...
-
zinid
servers can drop MAM archives on reconnect for example, what's the problem?
-
SaltyBones
Okay...
-
zinid
or later, when it's delivered
-
SaltyBones
this is just me confusing MAM and the other thing again...
-
SaltyBones
for the one-millionth time...
-
Ge0rG
What if I want my messages both on my pc and my mobile? I can't just drop MAM when my pc is online
-
zinid
you can if everything is delivered
-
zinid
anyway, what you are trying to say is a partial replication, and this problem is very hard to resolve
-
SaltyBones
what is the other thing again, server side archiving?
-
SaltyBones
zinid, does MAM know if everything was delivered?
- Ge0rG had a very concerning realization about guessable IDs and packet filters in smack earlier today.
-
zinid
SaltyBones, well, no, I think you cannot know that in general case
-
SaltyBones
zinid, but that is what SM does, right?
-
zinid
SaltyBones, well, if you introduce acks, then yes
-
SaltyBones
Hm.... :)
-
Holger
All this is orthogonal to the question whether having two separate stanza/message queues is sane. I agree with zinid that it isn't.
-
Zash
Two whatnow
-
Ge0rG
I'm saying for many years now that we need to replace 0184, 0198, 0280 and 0313 with one single proper message syncing thing.
-
Holger
SM is already mostly just an optimization, and I think we should fix the remaining issues to make stream management superfluous.
-
SaltyBones
I just want to point out that this is orthogonal to our earlier discussion about IDs :)
-
zinid
SaltyBones, I said "no" in general case because we have Bysantine General Problem, it's unresolvable, what we can gurarantee is sequential consistency
-
Ge0rG
SaltyBones: please show your ID before being allowed into this room :P
-
SaltyBones
Holger, does anybody already know what those issues are?
-
Zash
Ge0rG: Kidnap some server devs and lock yourself in a room until that one single proper message syncing thing to rule them all is properly implemented and XEP'd
-
Holger
SaltyBones: Syncing of outgoing messages (in a sane way) and maybe avoiding some round trips during session startup.
-
zinid
Zash, and watch them die 😉
-
Ge0rG
Zash: I'll lock you and zinid up in that room, and watch the live stream
-
SaltyBones
Holger, so, can we just turn off stream management and leave MAM and see what happens to find out?
-
Holger
Sure you can :-)
-
SaltyBones
Or do you think fixing up MAM is a bad idea and something new is required?
-
Zash
SaltyBones: That's what I did, actually. Haven't died from SM-lessness yet.
-
Holger
SaltyBones: I think you can already implement proper sync with MAM as-is. But something new is required to let clients implement this in a sane way, without having to de-duplicate and whatnot.
-
zinid
that's my point: what we need is to store messages and some other restricted stuff and call it a day
-
SaltyBones
Holger, so would unique message IDs solve this?
-
SaltyBones
I mean, at least de-duplication would be reasonably easy then, I guess.
-
SaltyBones
Of course some sort of counter makes more sense in this case...
-
Zash
SM has counters. MAM has server-issued, guarante to be unique message ids.
-
SaltyBones
Why are IDs not a counter?
-
Holger
Not sure what sort of uniqueness we need to solve what problem. Didn't read the backlog, sorry :-) The thing missing for proper sync of outgoing messages is an algorithm to compute the MAM IDs of outgoing messages.
-
SaltyBones
I mean the MAM-IDs....or is that the same as the stanza-ids?
-
Holger
(Sorry, typed this before reading the last few messages.)
-
zinid
Zash, now answer the question "get me last messages I didn't receive" with current SM and MAM approach 😉
-
SaltyBones
Holger, what do you mean by outgoing? Messages that we sent?
-
Holger
Yes.
-
zinid
if you say "time", then no
-
Zash
zinid: query after = id of last message I saw
-
SaltyBones
Okay, that should be solved by the hash idea....if it works :p
-
Zash
cry over outgoing messages sent after that
-
Holger
SamWhited: I need to know their IDs so I can tell the server "give me all messages since $id".
-
SaltyBones
But indeed if we want to query MAM by a "point in time" or "counter" unique IDs are not really the best thing :)
-
SaltyBones
Holger, but then the server still has to search the MAM archive for that ID and give you everything after it....
-
zinid
Zash, define after in distributed system 😉
-
SaltyBones
So a counter would be much better.
-
Holger
SaltyBones: Sure?
-
Zash
zinid: MAM archive ids on incoming messages
-
SaltyBones
Right?
-
Zash
zinid: after is a MAM term
-
zinid
Zash, but you need some ordering
-
Zash
zinid: huh
-
Zash
zinid: XMPP streams are ordered
-
zinid
yes, but you need to maintain a timestamp index in the database
-
Zash
You've lost me
-
zinid
well, you're probably right that if we assume timestamp ordering then we don't need counters and SM at all
-
Zash
Huh?
-
Zash
In a MAM query, 'after' is a field that carries a MAM archive ID
-
zinid
so, what's the ordering? 🙂
-
zinid
ID+1?
-
zinid
from what I understand you use timestamp+id, which means time ordering
-
Zash
Ordering?
-
Zash
As I said, you lost me. I have no idea what any of us are talking about anymore.
-
zinid
ok
-
Ge0rG
https://summerofcode.withgoogle.com/organizations/?sp-page=5 no xsf?
-
zinid
BEAM community is there 😛
-
moparisthebest
SaltyBones: like 98% of what you were talking about is here https://github.com/moparisthebest/xmpp-ircd
-
moparisthebest
Basically it works if IRC users don't want nickserv or chanserv but they do and I never got back to it :)
-
SaltyBones
moparisthebest, you mean of a way to connect to a muc using the irc protocol?
-
moparisthebest
Yes
-
moparisthebest
Running that makes a muc look like an IRC server to an IRC client
-
SaltyBones
I'm pretty sure nobody wants that. At least I don't! :D
-
SaltyBones
But I think it might provide an excuse for people who have to convince irc users ;)
-
moparisthebest
SaltyBones: yesterday you said
-
moparisthebest
Hm...maybe the transport should be the other way round. Offer an IRC server that connects to MUCs.
-
moparisthebest
That's what I was referring to
-
vanitasvitae
Ignite didnt make it into GSoC
-
vanitasvitae
:(
-
SaltyBones
moparisthebest, I know, I said that regarding the discussion of the KDE folks looking for an IM solution
-
Flow
`> * Ge0rG had a very concerning realization about guessable IDs and packet filters in smack earlier today Care to share?
-
Flow
Or is it just something in the ancient smack library yaxim uses?
-
moparisthebest
SaltyBones, right so they could use a MUC, but also have an IRC server that IRC users could use
-
Dave Cridland
moparisthebest, I like this, BTW.
-
moparisthebest
and everyone would end up in the same place, but it'd be a muc
-
Dave Cridland
moparisthebest, Is the GPLv3 your addition or from telepaatti?
-
Dave Cridland
Anyone know what servers support XEP-0288 these days?
-
Dave Cridland
Does this Prosody instance, for example?
-
moparisthebest
Dave Cridland, looks like the original telepaatti is gone, but GPLv3 goes back to at least the next fork looks like
-
moparisthebest
honestly I kind of abandoned it because I like rust so much more than python nowadays but can't be asked to rewrite everything yet haha :)
-
Flow
-xep 288
-
Bunneh
Flow: Bidirectional Server-to-Server Connections (Standards Track, Draft, 2016-10-17) See: https://xmpp.org/extensions/xep-0288.html
-
moparisthebest
I still run an IRC server I would love to rm -rf
-
SamWhited
> I need to know their IDs so I can tell the server "give me all messages since $id". I know, apparently I wasn't clear about something, sorry. I'm not suggesting we get rid of MAM or leave everything exactly as it is today, just that MAM doesn't cover some important parts of SM and probably can't be made to cover it without significant downsides.
-
Ge0rG
Flow: it's affecting the old smack for sure, I'll have a look into smack 4 and let you know
-
SamWhited
(sorry for the long delay, got pulled into a meeting and then was AFK for a bit)
- Dave Cridland greps his logs
-
Dave Cridland
So there's actually only one server I talk to that does bidi. That's scary.
-
Flow
Ge0rG, let me know if you didn't just re-discover CVE-2014-0364
-
Ge0rG
Flow: it's related but different
-
Zash
Dave Cridland: Is it mine? :)
-
Dave Cridland
Zash, No, Lance's. Although actually grep might have gone into Annoying Binary Mode.
-
SaltyBones
-I ?
-
Dave Cridland
-a, but yeah.
-
Dave Cridland
OK, so that's actually lots of servers doing bidi. Happy bunny, now. I've been putting the support into Metre.
-
Zash
Oh? Hm, feeling up for doing a survey?
-
Dave Cridland
Zash, What sort of survey?
-
Zash
"How many servers do 288?"
-
Zash
Hooooold on now
-
Zash
Is that the same number as ...
-
Zash
https://www.youtube.com/watch?v=azEvfD4C6ow !!!
-
Dave Cridland
Zash, Looks like it's PSYC and Prosody.
-
Zash
Prosody doesn't do it out of the box, you need to go a bit out of your way to install a community module.
-
Dave Cridland
Really? Quite a few people have, then.
-
Zash
Dave Cridland: Question is, how much self-selection bias is there among people that have you in their roster? :)
-
Dave Cridland
Zash, Lots.
-
Dave Cridland
.
-
Zash
,
-
Kev
M-Link does bidi, but we disabled it because of bug reports from Prosody about us not accepting stanzas down the right streams. Some of which I'm starting to question :)
-
Dave Cridland
:-)
-
dwd
So, Metre now does Bidi.
-
dwd
OK, this is weird. Metre is successfully negotiating Bidi with various Prosody servers. OK, great. But absolutely nothing ever tries to negotiate bidi with it, despite it offering the feature.