By the way, regarding the email I sent to JUser, is there a better mailing list for that, please?
jonasw
I still haven’t subscribed to juser :/
Seve
That's why I would like to send it somewhere else too, seems not many people are subscribed there
jonasw
Seve, I think operators would be more fitting, maybe
rionhas left
jonasw
possibly with a cross-post to members@
jonasw
I’m also not sure if a separate MUC for this is a good idea
Zash
Posted something to some KDE list?
rionhas left
rionhas left
jonasw
frankly, I’m not sure XMPP can fulfil the requirements at this instant in time
jonasw
> Stickers are available
jonasw
we doomed
Zash
Find out which are requirements and which are wishlist
jonasw
Zash, they don’t even know that yet
rionhas left
jonasw
> Requirements are not prioritized (incl. must-have vs. nice-to-have), that comes at a later step.
rionhas left
Martinhas left
jubalhhas joined
rionhas left
danielhas left
Zash
jonasw: All are wishlist. There's online one requirement, that everyone you care about is already using it. :)
andyhas joined
tim@boese-ban.dehas left
rionhas left
Seve
jonasw, the point of the MUC is to discuss this topic with people interested in helping out. Almost everybody here would find this discussion annoying here.
jonasw
Seve, I’m running out of screen space for more MUCs.
rionhas left
rionhas joined
tim@boese-ban.dehas joined
rionhas left
andyhas left
marc
Ge0rG, jonasw can you review https://git.zapb.de/xeps.git/commit/?h=fix_xep_0401&id=52725e993987f205dc253cc5a4e6937fe3955d81 and merge please?
uchas joined
moparisthebesthas joined
jonasw
marc, EBUSY right now
jonasw
marc, if you refuse to make a github PR (which would really be the most useful thing for me), please send an email with the first line of the commit message as subject to editor@xmpp.org
marc
jonasw, I can do a GitHub PR if you like
jonasw
marc, that would be much appreciated, thanks
jubalhhas left
marc
jonasw, you're welcome
jonasw
gotta run now, see you
marc
bye
Martinhas joined
moparisthebesthas joined
rionhas joined
marc
done
Alexhas left
jjrhhas left
Zashhas left
rionhas left
rionhas joined
rionhas left
jubalhhas joined
valohas left
valohas joined
jubalhhas left
rionhas joined
andyhas joined
suzyohas joined
Zashhas joined
hanneshas joined
danielhas left
hanneshas joined
remkohas left
remkohas joined
jubalhhas joined
andyhas left
jubalhhas left
andyhas joined
jonasw
zinid, regarding the server capabilities invalidation, how about a message?
jonasw
server changes caps -> server sends <message/> to local clients. for server-to-server links, it could simply close the link so that the peer re-opens it, thus getting new stream features
Alexhas joined
remkohas joined
zinid
jonasw, closing and opening thousands of s2s will create avalanche effect
jonasw
right
zinid
we see this on jabber.ru for examlpe
zinid
so I just think about nonza
zinid
stream-level element
jonasw
zinid, thanks for your input, I’ll post a few suggestions to the mailing list and I hope you’ll comment on them :)
zinid
sure 😉
suzyohas joined
intosihas left
andyhas left
moparisthebesthas joined
Dave Cridlandhas left
moparisthebesthas joined
Dave Cridlandhas left
Dave Cridlandhas left
SaltyBones
Is there a good resource on the problems with message ids?
I'm looking at XEP-0359 but the problems mentioned at the summit are not in there.
jonasw
maybe the summit notes as first starting point
hanneshas joined
jubalhhas joined
andyhas joined
jubalhhas left
Syndacehas left
Syndacehas joined
suzyohas joined
SaltyBones
1.2 Stable IDs
Discussion on entropy ensues.
dwd: <stanza-ids> don't work for unique addressing because we don't trust clients to do them properly.
Kev: lots of possible attacks with spoofable stanza IDs.
SaltyBones
Not verbose enough.
jonasw
SaltyBones, I think the gist of that is: we need to generate stanza IDs on the server because weak/constructed stanza IDs are a problem. but the client needs to know the stanza ID for archive operations and possibly other things
zinid
jonasw, well, dunno, for me message looks too complex
Martinhas left
zinid
can't we just send <c/> as a nonza?
jonasw
zinid, I think RFC 6120 would slap you in the face for that ;-)
jonasw
really, it needs negotiation
jonasw
it’s a shame that we have no way for a client to send stream features :(
zinid
yeah, because some implementation may close a stream if they receive unrecognized nonzas
jonasw
yupp
jonasw
I know at least one.
zinid
however, I tried to implement such behaviour (closing a stream) and got lots of problems and left the idea
jonasw
never had issues so far; did you encounter things which sent unsolicited nonzas?
zinid
alas, I've already forgotten what that was exactly 😕
jonasw
zinid, the best we could do is probably say "okay, if the client sent a presence update with caps, we can send a nonza with a caps update"
jonasw
and similarly for servers ("offered caps in stream feature -> send nonza")
jonasw
but I don’t know enough about s2s to be sure that this works
jonasw
this would definitely need a namespace bump though
zinid
well, we're talking about new caps xep?
zinid
we can do it there
jonasw
yeah, it requires a namespace bump there too ;-)
zinid
can't we just negotiate this feature?
Dave Cridlandhas left
zinid
not sure how
jonasw
negotiation will add a round-trip
jonasw
which people will react very adversely to
zinid
ah
jonasw
but please comment on-list, hoping to get more input on that
zinid
I don't know what to say
Dave Cridlandhas left
jonasw
what you said here, essentially?
la|r|mahas joined
lskdjfhas joined
andyhas left
Dave Cridlandhas left
remkohas joined
Dave Cridlandhas left
Dave Cridlandhas left
Dave Cridlandhas left
Dave Cridlandhas left
Dave Cridlandhas left
Dave Cridlandhas left
Dave Cridlandhas left
SaltyBones
There are stanza-id and origin-id in XEP-0359. Somebody at the summit also mentioned "message-id" is that defined somewhere?
MattJ
They were probably referring to the id attribute, I guess?
Dave Cridlandhas left
SaltyBones
MattJ, this https://tools.ietf.org/html/rfc6120#section-8.1.3 ?
MattJ
Yes
SaltyBones
thx
MattJ
The reason XEP-0359 exists is because the id attribute is controlled (and can only be trusted by) the original sender of the stanza
MattJ
It's really just designed for tracking errors, though a couple of XEPs have re-used it for other purposes
Dave Cridlandhas left
suzyohas joined
Martinhas joined
Dave Cridlandhas left
suzyohas joined
remkohas joined
Dave Cridlandhas left
SaltyBones
The id-attribute you mean?
jonasw
yes
SaltyBones
So, I guess, stanza-id is used by MAM and origin-id is used to detect MUC reflections (that's what 0359 says anyway). And id-attribute is used for errors but the error will have the same id-attribute iiuc, right?
jonasw
yes, pretty much
jonasw
except that origin-id won’t work with all MUCs
SaltyBones
Why?
andyhas joined
jonasw
because not all MUCs may be able to reflect that ID for whatever implementation specific reason
jonasw
(just like not all mucs can reflect the message @id)
Zashhas left
MattJ
There is only one implementation I'm aware of that doesn't (didn't?) reflect @id, and I'm not even sure of the current status of that
SaltyBones
Okay, so why does the standard not just mandate that it be reflected?
MattJ
Simple oversight I suspect
Dave Cridland
SaltyBones, Non-MUC things pretending to be MUC, in part.
Dave Cridland
SaltyBones, Also by the time people had noticed this was a problem, enough implementations didn't.
SaltyBones
So, could this be fixed by changing the standard and requiring that origin-ids be reflected?
Dave Cridlandhas left
SaltyBones
Dave Cridland, what do you mean by "non-muc things pretending to be muc"?
jonasw
SaltyBones, changing the standard doesn’t fix the implementations magically
MattJ
Dave Cridland, enough = 1? Also in the case of bridging to non-MUC, isn't it as simple as 1) if the non-MUC supports acks, wait for the ack and reflect the id or 2) reflect the id immediately?
jonasw
SaltyBones, IRC transports are "non-muc things pretending to be MUC" for example
SaltyBones
jonasw, not magically but ...
Dave Cridland
MattJ, More than one, I think. One classic MUC implementation, but quite a few transports.
uchas joined
Dave Cridland
FWIW, I've gone back and forth over whether MUC ought to reflect ids. It's a special case of maintaining the id on bradcast, which is itself both useful in some cases and bad in others.
SaltyBones
Why is this reflection necessary?
Dave Cridland
SaltyBones, Reflection isn't - we could add in a subelement which indicated the original id.
Dave Cridland
SaltyBones, Broadcast ids are harder to work around - there's a few protocols which use id as reference.
Dave Cridland
This all said I *really* dislike having a zillion ids as subelements of stanzas when we already have an id attribute.
Kevhas left
SaltyBones
Well, what is the subelement necessary for? Is this just as an ACK to the client?
SaltyBones
What do you mean by broadcast-id?
Dave Cridlandhas left
MattJ
SaltyBones, if you're asking why MUC reflects messages to the sender in the first place (some systems, e.g. IRC don't) - it has a few benefits
MattJ
Such as ensuring everyone in the room sees the same messages in the same order
jonasw
it also is natural as soon as multiple clients are in the play, somewhat like message carbons
Dave Cridland
SaltyBones, So the id attribute in the message stanza for a groupchat message sent to the MUC can be reused in the reflected message, and/or the broadcast messages to other participants in the group.
MattJ
In IRC if two people send a message at the "same time", the messages will be shown in a different order on both their clients
MattJ
and yes, multiple clients from one user in the room
jonasw
it allows MUCs to do fancy things to messages and still have everyone see the same thing
Dave Cridland
(As a side-note, Matrix achieves a similar goal by a complex hash chain)
andyhas left
MattJ
SaltyBones, and also servers modifying messages (e.g. in the Prosody MUC we automatically convert long multi-line messages to a pastebin link). The reflection allows the sending client to see what everyone else sees.
remkohas joined
andyhas joined
jonasw
(in contrast to IRC, where you don’t see that your message was truncated)
SaltyBones
Okay, pretty convincing...
MattJ
Dave Cridland, and as many people at FOSDEM noted, much RAM
Dave Cridland
MattJ, Yes but BLOCKCHAIN.
jonasw
#bingo #triggered
SaltyBones
🔲⛓️
Dave Cridland
I hope there's a ZWJ there.
moparisthebesthas joined
SaltyBones
*Yes, there is.*
Dave Cridlandhas left
SaltyBones
Okay, so we want reflected messages. Sounds fair. Let's suppose for a second that we can just make them mandatory in the standard and people would add them.
jonasw
they *are* mandatory
jonasw
the issue is with message IDs
jonasw
clients need to be able to recognize the reflection
SaltyBones
You mean id-attribute?
jonasw
or any ID really
jonasw
id attribute would be easiest
Dave Cridland
We could indicate in disco if ids were stable.
andyhas left
jonasw
wasn’t that proposal rejected-ish?
SaltyBones
Okay, 1. is the origin-id used for anything else and 2. is the whole point of origin-id not to detect reflection?
moparisthebesthas joined
jonasw
speaking of MUC, I think the vote on https://github.com/xsf/xeps/pull/559 expired, didn’t it?
Ge0rG
I've attemtped to fix MUC reflected IDs some years back, not even in the normative language but merely by fixing the examples
Ge0rG
And got significant flack for it.
Ge0rG
I also attempted to make origin-id == message-id, but it was refused by the XEP author.
SaltyBonesgroans.
Ge0rG
Now I'll just sit and wait, and ocassionaly proclaim: *I told you so!*
SaltyBones
So, is the point of origin-id to be used for reflection or is there something else?
Flow
SaltyBones, that is one use case
lumihas joined
SaltyBones
Ge0rG, great, if you could also occasionally chime in with reasons for things and explanations that would be awesome.
SaltyBones
Flow, like?
Ge0rG
SaltyBones: it's mainly for reflection, except that MUCs are not guaranteed to keep non-body elements in the reflection
Flow
other use cases include finding your own messages in a archive
SaltyBones: yes, but you don't know the stanza-id for the messages you sent
SaltyBones
Ge0rG, would you say this is a requirement by nature or just a problem with implementations?
daniel
i agree that the xep should specify that the sender should set origin-id=message-id
Ge0rG
SaltyBones: what?
Ge0rG
daniel: MUST set.
SaltyBones
Ge0rG, the fact that MUCs don't reflect non-body attributes.
daniel
yes
Ge0rG
because anything different is just a path into insanity
daniel
(i didn't mean SHOULD)
SaltyBones
Ge0rG, also, why do I need the stanza-id for a message I sent?
Ge0rG
SaltyBones: have a look at biboumi, a modern IRC transport. It doesn't reflect your message ID, it doesn't reflect any non-body elements and it mangles multi-line messages
Ge0rG
Welcome to message tracking hell.
daniel
i think thinks will already break if some client doesn't do that and then expects a delivery reciept or something
Ge0rG
SaltyBones: because when you ask for a MAM sync, you'll get yout sent messages copied to you as well
daniel
when sending messages to Conversations i mean
Ge0rG
Yes, message correction, delivery receipts and anything else that references IDs is a mess already.
SaltyBones
Ge0rG, and why is that a problem? They should have their origin-id and be recognizable, right?
Ge0rG
SaltyBones: you can't rely on origin-id being there, and you can't rely on it matching the message-id.
SaltyBones
daniel, what breaks when a client does what?
Ge0rG
SaltyBones: but yes, you can match MAM archives based on origin-id
daniel
if an origin id is set Conversations will use that as a reference in the receipt
daniel
so if your client doesn't do id=origin-id and expects the receipt for the id it won't work
daniel
same with everything else that references ids
SaltyBones
So, there seems to be consensus at least in this MUC right now that attribute-id should be the same as origin-id?
daniel
i'd argue there is no good reason not do make this a MUST
daniel
i mean most client would naturally do this anyway
SaltyBones
Ge0rG, who was the xep-author who was against this and do you remember why?
jonasw
SaltyBones, Flow
daniel
because why would you generate a second id if you can reuse the existing one
note that muc rewrites a irrelevant to the scenerio described above
Dave Cridland
jonasw, #559 expired, yes.
zinid
This is to track its own messages?
daniel
zinid, yes
daniel
or references to that
Dave Cridland
jonasw, But with no veto and the rest are +1, so apply it.
Dave Cridlandhas left
SaltyBones
jonasw, what is this ID-rewriting MUC shit? :)
daniel
some mucs are known to change the 'attribute' id
Ge0rG
SaltyBones: have a look at examples 44 and 45 in https://xmpp.org/extensions/xep-0045.html#message
daniel
so tracking your own message are parsing references don't work
zinid
daniel: why, you can rely on origin-id in this case
Ge0rG
If we have a MUC that rewrites message IDs, can we mandate that it also MUST rewrite references in all XEP payloads that reference message IDs, please?
zinid, in the case of muc? because most mucs won't remove child elements from the stanza
zinid
I'm lost
zinid
daniel: but that's exactly what's needed
daniel
now i'm lost :-)
zinid
So you can fetch origin-id and check if this is your id
Kevhas joined
jjrhhas left
zinid
Why do you need to rely on message-id if you inject origin-id
daniel
i just said the sender should set id=origin-id
SaltyBones
zinid, I think there was a sub-discussion about forcing message-id = origin-id
daniel
and/or deal with delivery receipts that reference the origin-id instead of the id
daniel
which is made easier if you id=origin-id
zinid
Ok, I don't get it 🤔
daniel
doesn't matter. that client to client stuff anyway :-)
la|r|mahas left
la|r|mahas joined
remkohas joined
zinid
From what I understand we just need to ditch IRC transports
zinid
😃
jjrhhas left
zinid
That's really their problem
daniel
oh yeah. or make irc transports track the messages themselves and re-add origin-id and stuff on reflection
daniel
it's probably better if the irc transport does the right thing(tm) than letting each and every client figure it out
zinid
daniel: agreed, nobody said writing transport is simple
daniel
irc transport should also reassemble the message if they previously split it and stuff like that
daniel
because they know best if they did
SaltyBones
Hm.
SaltyBones
Okay, are there any actual problems with the current state of three IDs except that we don't like having so many?
daniel
yes
daniel
read what i said before
SaltyBones
Uh, assuming a well-behaved server?
daniel
yes
daniel
my argumentation has nothing to do with servers
jonasw
daniel, IRC doesn’t even reflect
jonasw
so the transport generates the reflections on its own
SaltyBones
Okay, can you please point out what you are referring to then?
jonasw
biboumi however decided to reflect the split version of the message, and there’s some argument to doing that
daniel
assume i sent: <message id="1"><body>hi</body><request/><origin-id="2"></message>
and then i will receive from conversations: <message><receipt id="2"></message>
zinid
who cares about IRC, really, are we designing a protocol for old nerds?
daniel
but we are talking about two different things. the how should irc transports behave has noting to do with the problem i'm describing
daniel
this applies to 1:1 chats as well
jonasw
daniel, yes, I don’t argue that
jonasw
I’m just replying to your message from "13:49:15Z" because I was away from keyboard.
remkohas left
daniel
the irc transport with the name i can not pronounce or remember does a few things that i don't like :-)
zinid
daniel: if we don't care about IRC then we probably don't need origin-id and thus there is no the problem you just described
daniel
there are non transport mucs which also rewrite the id
zinid, if it weren't only for transports i wouldn't care
daniel
Ge0rG, is that an issue to change the name to something i can remember?
daniel
or pronounce?
zinid
daniel: if they are abandoned, then I personally give zero fucks
Ge0rG
daniel: no, it's about maintaining IDs on reflection
Ge0rG
zinid: everything in XMPP is abandoned. Now stop giving fucks.
daniel
i should probably write my own irc transport
edhelas
biboumi is not good ?
Ge0rG
daniel: you should just patch biboumis two or three warts.
daniel
i haven't used irc though ever since counterstrike 1.6 came out
zinid
Ge0rG: not everything, so I still have a few fucks to give
daniel
there are a number of things i don't like that seem to be design decisions rather than bugs
andyhas joined
daniel
so i'm not sure if they even want me to change them
SaltyBones
daniel, so regarding your example, what you are saying is that the read receipt might reference the message-id or the origin-id and it is not properly specified which?
zinid
Ok, whatever, so what do you guys think about muc subscriptions?
SaltyBones
Or are you getting at the fact that origin-id requires "by=" and is therefore sometimes not applicable?
daniel
> daniel, so regarding your example, what you are saying is that the read receipt might reference the message-id or the origin-id and it is not properly specified which?
This
daniel
not just read recepits but everything that references something
daniel
but yes
jubalhhas joined
zinid
daniel: isn't there a place in the xep which says you should copy the message-id?
daniel
zinid, no.
zinid
Really?
daniel
that's what Ge0rG and I want to add to the XEP
daniel
that's pretty much what the entire discussion is about :-)
zinid
So add it 😀
daniel
i can't without permission of the author
zinid
No shit?
SaltyBones
So, there is some problem that I am still not aware of....
SaltyBones
Flow, why is the by attribute in the origin-id?
zinid
Who's the author?
SaltyBones
The one that causes the privacy problems...
Kev
Not entirely true, BTW, that it's impossible to do things without the author. But it's the path of least resistance.
daniel
i don't think there is a by attribute in the origin id
SaltyBones
Ah, sorry that is only for stanza IDs. origin-id does not have by.
zinid
Peter is the author
SaltyBones
That might almost always be correct buy 0359 is Florian Schmaus✎
SaltyBones
That might almost always be correct but 0359 is Florian Schmaus ✏
zinid
> In addition, it SHOULD include an 'id' attribute that echoes the 'id' attribute of the content message.
zinid
And you want this SHOULD to be a MUST?
lskdjfhas joined
SaltyBones
zinid, what some people here want is that origin-id = message-id
SaltyBones
Ge0rG, you had an objection to that in https://mail.jabber.org/pipermail/standards/2017-September/033415.html but I don't quite understand it. Why do you want to know if somebody creates strong message-id?
zinid
SaltyBones: but the sender controls this itself, so what is a problem to set them equal?
daniel
zinid, we want wording that tells the client developer to do this
daniel
there is no 'problem'
efrithas joined
Ge0rG
SaltyBones: I thought it would matter, but it doesn't, because there always can be malicious entities
zinid
daniel: if they don't then they only harm themselves, I guess
Ge0rG
zinid: no, they harm the other participants
zinid
Ge0rG: ah
SaltyBones
How?
zinid
Yeah, how?
Ge0rG
zinid: also, it's well possible that stanza ids are generated by the xmpp library, and origin ids by the client, causing a mismatch
Ge0rG
By making all message references break
Ge0rG
We had that above already.
SaltyBones
Before we move to stanza-ids....
SaltyBones
There is nothing wrong with requiring origin-id = message-id?
Martinhas left
zinid
If we require this then why origin-id is needed, wtf?
SaltyBones
zinid, it's possible that it was a mistake
SaltyBones
daniel, does requiring origin-id = message-id actually solve your problem or are there cases when stanza-id might be used to refer to a message instead??
daniel
It solves my problem
andyhas left
andyhas joined
SaltyBones
daniel, wouldn't it be better to just mandate that clients always use the origin-id to avoid the problem of dealing with id-rewriting MUCs?
SaltyBones
I mean, certainly removing an unnecessary id would also be desirable but this might be a good intermediate step.
Martinhas joined
SaltyBones
Would everything be better if clients would generate proper stanza-ids?
SaltyBones
For ...uh...some definition of proper that makes them UUIDish.
zinid
they should be strictly monotonically increasing, so we don't need XEP-0198
SaltyBones
And by "would everything be better" I specifically also mean, wouldn't that allow us to ditch the other IDs?
zinid
and not UUIDs
jonasw
zinid, "strictly monotonically increasing" is not gonna happen
zinid
because?
SaltyBones
zinid, would increasing IDs really solve all the same problems as stream management? Seems unlikely?
jonasw
zinid, also, increasing IDs have security implications
jonasw
or rather: predictable IDs
zinid
SaltyBones, it solves in data replication, but of course XMPP is too unique
zinid
version vectors are based on such ids
zinid
jonasw, what security implications?
jubalhhas left
jonasw
zinid, I think there were some IQ response injection attacks based on predictable IDs
jonasw
even though in that case you probably already made the mistake of not verifying the sender of the response
jonasw
error injections would most likely work though because errors can come from entities different than the original recipient
zinid
jonasw, this can be fixed by adding routing information
SaltyBones
jonasw, maybe this is too much of an assumption for all of XMPP but don't we have transport encryption?
zinid
we anyway need this functionality already
jonasw
SaltyBones, that doesn’t stop me (jonas@zombofant.net) from sending an iq type="error" id="whateverIguessed" to="you" to break whatever you were doing
SaltyBones
zinid, I agree, if this is a problem I don't see why it cannot be exploited right now by somebody who randomly sees the message first
jonasw
SaltyBones, if I can guess the ID, I can attack you from off-path
jonasw
I can’t do that when I can’t guess the ID
jonasw
if I’m on path, you’re right, everything is lost already in XMPP.
zinid
but if you cannot match incoming errors against requests you sent then you should really consider to change the job
jonasw
zinid, how’d you match incoming errors against requests other than the ID?
jonasw
you can’t really use from, because as I said, it might come from an entity you didn’t know about yet
MattJ
jonasw, no?
zinid
jonasw, you can use (from, ID) I think
MattJ
errors come from the original recipient that you addressed
jonasw
MattJ, even if an s2s error causes an error?
MattJ
Yes
jonasw
oh okay
MattJ
Otherwise it would be a nightmare
Dave Cridland
jonasw, There's a "by" attribute, if I remember right, that tells you the generator/reporter of the error.
Dave Cridlandhas left
SaltyBones
So, I assume that if I run a h4xx0r server and try to send replies to messages that were not directed to me to some other server they will be discarded?
SaltyBones
And the same happens between client and server...?
MattJ
SaltyBones, by "replies", you mean error replies?
SaltyBones
Whatever magical interferring replies that jonasw was referring to :)
SaltyBones
yes, errors ;)
MattJ
For that to work you would have to know the original sent message id, and the original sender's full JID
SaltyBones
But if I did, I could?
MattJ
and then yes, you could send them whatever you wanted - it would be up to their client to match it up (or not) to one of its outgoing messages
SaltyBones
So, the statement that predictable IDs would allow me to spoof responses remains correct?
MattJ
so as said above, the client should use (from, id) to identify error responses, not just the id
SaltyBones
Because I cannot spoof "from"?
MattJ
Correct
SaltyBones
Yes, okay...
SaltyBones
That's what I was looking for. :)
remkohas joined
MattJ
id "abc123" from userA@domainA is different from "abc123" from userB@domainB
SaltyBones
And servers only accept s2s with from=...@domainA if they know that the other server is in charge of domainA?
Dave Cridland
SaltyBones, That's what they're supposed to do, yes.
remkohas left
SaltyBones
^^
MattJ
SaltyBones, yes, they do
Dave Cridland
MattJ, No, they don't, but I love your optimism.
MattJ
If they don't, it's a bad bug or not an XMPP server
Dave Cridland
SaltyBones, So Metre (my S2S proxy mad thing) does this very strictly, and as a result I can't join this chatroom from my personal server.
Dave Cridland
SaltyBones, Not sure if MattJ is saying that's a bad bug in this server, or if he's saying it's not an XMPP server. :-)
MattJ
Dave Cridland, for what reason?
jubalhhas left
andyhas left
SaltyBones
Dave Cridland, so which messages do you get that don't pass?
Alexhas left
SaltyBones
Bunneh, version xmpp.org
Bunneh
SaltyBones: xmpp.org is running Prosody version 0.9.12 on Linux
Dave Cridland
MattJ, It's sending a response down the stream obtained by reversing the stream to/from, and not by reversing the stanza to/from. I forget the details, Zash knows.
Dave Cridland
MattJ, I don't know if Prosody would accept that or not. Metre doesn't, Openfire does (but because it knows it's multiplexed on the outbound, so weirdness.)
Dave Cridland
MattJ, For maximum fun, Openfire used to do similar "implicit" auth on outbound, but I stopped that as from 4.2.0.
MattJ
I don't really understand, but maybe Zash will enlighten me
Dave Cridland
MattJ, Metre sends traffic for d.c.n -> muc.xmpp.org over a stream for d.c.n -> xmpp.org (after adding that domain via dialback).
Zash
How far back should I read to have any idea what you are talking about?
SaltyBones
15:58 should suffice
MattJ
SaltyBones, and a time machine
Dave Cridland
MattJ, Prosody (0.9) responds to the traffic on the reverse stream, xmpp.org -> d.c.n - which it hasn't requested authorization for yet.
SaltyBones
Do you mean a timezone machine or a MAM machine?
Alexhas left
Zash
No MAM here
Dave Cridland
SaltyBones, For your amusement and mild confusion, XMPP authorizes streams by a 3-tuple of (from-domain, to-domain, direction).
Zash
Page up button tho
Dave Cridland
SaltyBones, Well. One or more of those 3-tuples anyway.
MattJ
Dave Cridland, who hasn't requested authorization for what? You mean Prosody connects to d.c.n as 'xmpp.org' and sends stanzas from 'muc.xmpp.org'?
Dave Cridland
MattJ, Yup, that.
dwdhas joined
Zash
The thing
Dave Cridland
MattJ, Like this: DEBUG 2018-02-12T15:14:35 /home/dwd/src/Metre/src/xmlstream.cc:95 : NS296914 - G
ot [399] : <?xml version='1.0'?><stream:stream xmlns:db='jabber:server:dialback'
xmlns:stream='http://etherx.jabber.org/streams' version='1.0' from='xmpp.org' t
o='dave.cridland.net' xml:lang='en' xmlns='jabber:server'><iq id='472-452048' ty
pe='error' to='dave.cridland.net' from='xsf@muc.xmpp.org/tim@boese-ban.de'><erro
r type='cancel'><not-acceptable xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'/></e
rror></iq>
dwd
This time I'm joined. Timing, I guess.
Zash
Uh, what was the thing with that again?
Dave Cridland
Zash, The bug itself? Responding to inbound traffic on a multiplexed stream by reversing the wrong domain pair.
MattJ
dwd, I don't see how timing would play a part
Zash
Dave Cridland: Do you remember where we discussed this?
Dave Cridland
MattJ, I think whether I can join is dependent on what streams are open, which is dependent on the session lifetime and activity.
Dave Cridland
MattJ, Not timing as in a race condition, mind.
Dave Cridland
Zash, Erm. 1:1 messages, I think, until we figured out it definitely wasn't a security issue.
Dave Cridland
Zash, Possibly some discussion in jdev.
dwd
FWIW, I have to put in something similar to an implicit auth for X2X, anyway. So this may provide a workaround.
Holger, Can you download NetBSD yet, or is it still only available on tape?
zinid
Holger, cool stuff btw
SamWhited
More importantly, once I download it can I run it on any machines made after 1995? (this is always the problem I had trying to run NetBSD; a few of the ops people at work run it, but they all use *very* old machines)
Dave Cridlandgets prepared to explain "tape" to the younger readers.
Dave Cridland
SamWhited, Sure! It runs pretty well even modern machines like the Amiga 4000.
nycohas left
Dave Cridland
(Sorry, that was 1993).
SamWhited
Exactly.
Holger
Well yes it's about as dead as XMPP :-P
SamWhited
I would like to use at *least* a P3.
Holger
But it always worked okay-ish for me on non-recent ThinkPads.
Holger
I no longer do that though.
Holger
cpu0 at mainbus0 apid 0: Intel(R) Core(TM) i3-2120T CPU
Holger
... is the one box I still run NetBSD on.
Zash
Uh, was NetBSD the one being difficult, or OpenBSD?
Holger
Difficult?
SamWhited
Actually, jokes aside, that might be a nice thing to do with my broken old thinkpad; doesn't work very well as a laptop anymore, but it could be a {Free,Net,Open}BSD or Illumos "desktop".
Holger
Zash: Isn't all computers difficult except for Apple?
SamWhited
This is true… ⤴
Dave Cridlandhas left
Zash
Et Apple, Holger
Dave Cridland
I find Apple incomprehensibly difficult to use, I must admit.
Dave Cridland
I've yet to figure out how to go up a directory consistently in file dialogs.
Kev
cd ..
Kev
Happy to help.
Holger
Managing FreeBSD feels more or less like Debian to me, the two others feel a bit more basic but not really harder to use. All dead simple compared to the dances you had to go through when partitioning hard disks or getting X11 to run 20 years ago with any Linux or BSD.
Dave Cridland
Holger, Ah, X11. Kids these days don't know they're born.
Dave Cridland
Kev, Don't get me started on the antiquated command line tools Apple foists upon you.
remkohas joined
Holger
Yes, we're horribly old.
Dave Cridland
Holger, We're very experienced. I'm sure that's what you meant.
Zash
I'm sure wayland will bring back the good old times for those who miss configuring Xorg
Holger
Precisely.
SamWhited
The only thing I want out of a machine is to sync an old ipod without Rhythmbox crashing…
Dave Cridland
SamWhited, Rhythmbox never crashes.
Dave Cridland
SamWhited, At that speed it's called "parking".
SamWhited
Rhythmbox is one of those people who parallel parks by backing up until they hit the car behind them, then driving forward until they hit the car in front.
efrithas left
jubalhhas joined
Alexhas joined
SaltyBones
So, why is it that a client cannot create the stanza-id?
SaltyBones
Is this a thing about malicious clients or what's going on there?
Dave Cridlandhas left
jonasw
SaltyBones, no, unaware clients are sufficient; colliding IDs would lead to interesting issues with MAM
SaltyBones
jonasw, but if the client is only unaware the server could just return an error informing the client that this ID has been used.
Dave Cridland
SaltyBones, But we could use, say, stream-id + attribute-id to form a stable, non-colliding reference identifier.
jonasw
SaltyBones, that’d be annoying to handle in the client, and the server would need an O(1) (or similar) way to determine that the ID has been used already...
SaltyBones
Yeah, or we could hash things and god knows what else...it seems very solvable...
Dave Cridland
SaltyBones, I mean, unless the client isn';t generating unique ids within its stream, in which case it's presumably not going to be fixed to use some other identifier.
jonasw
Dave Cridland, stream id + sm-counter?
jonasw
that’s verifiable by the server
jonasw
and predictable for both
Dave Cridland
jonasw, True. But I think we had some ideas on predictability outside the stream?
SaltyBones
throw a hash on top to avoid privacy issues, done
SaltyBones
Dave Cridland, I thought we dicussed those into oblivion earlier? :)
Dave Cridland
SaltyBones, Really I've lost track.
jonasw
Dave Cridland, hmmm ... hmac(stream-id, sm-counter)?
Dave Cridland
SaltyBones, I have the feeling that nobody quite knows the entire picture here.
SaltyBones
I almost lost track and I basically didn't work at all today and just discuss here. :p
Dave Cridland
jonasw, HMAC() requires a secret to be of any use.
jonasw
Dave Cridland, stream-id.
SaltyBones
Seems reasonable on first sight...
jonasw
(post-TLS obviously...)
Dave Cridland
jonasw, Is that secret? And why HMAC over a hash?
SaltyBones
Oh, that would leave out the hash...
jonasw
Dave Cridland, I’m fine with hash(stream-id || sm-counter) too, but that’s nearly an hmac ;-)
jonasw, An HMAC is two nested hashed with concats and masks, yes.
SaltyBones
Okay, so we only disagree on our definition of nearly. ;)
Dave Cridland
jonasw, But the security properties are distinct.
jonasw
Dave Cridland, I don’t argue that
jonasw
(and I’m aware)
SaltyBones
Anyway, the suggestion of HMAC(key=stream-id, msg=sm-counter) seems good, doesn't it?
Dave Cridland
In any case, given a stanza with a known id on a given stream, we clearly want to be able to predict the MAM id.
SaltyBones
I mean, I am also fine with SHA-*(stream-id || counter-sm)
SaltyBones
Dave Cridland, but mam-id = stanza-id which is what I am proposing to set to this...
Dave Cridland
Question is, do we think the MAM id is likely to become public, and if so, can someone relatively easily figure out the next (or previous) value, and then do Something Bad?
SaltyBones
Dave Cridland, that might be the question bit if it only costs us a hash per message to not answer it might also not be...✎
SaltyBones
Dave Cridland, that might be the question but if it only costs us a hash per message to not answer it might also not be...✎✏
SaltyBones
Dave Cridland, that might be the question but if it only costs us a hash per message to not answer it, it might also not be... ✏
Dave Cridland
What I'm wondering, you see, is whether we give clients an algorithm to generate attribute-id and then let them signal to the server that they're going to use globally-unique attribute-ids and the server is allowed to call them out if they don't.
jonasw
Dave Cridland, okay, right, this is about the MAM ID. this *probably* doesn’t matter, but if the goal is to consolidate all IDs at some point, choosing a way where the ID can become public is safer
zinidhas left
Dave Cridland
jonasw, That's roughly my thought, yes.
jonasw
and at that point, HMAC(…) seems like a sane choice.
Dave Cridland
jonasw, Well, that and "But we *HAVE* stanza ids - right there in the stanza!"
Kev
How would the server know that they didn't use a globally unique id?
SaltyBones
Kev, if the server can simply re-run the generation algorithm and compare results..?
jonasw
Kev, if the ID doesn’t follow the algorithm for a globally (predictable, for the server and client only) unique ID
Dave Cridland
Kev, Well, it'd know if they used no id at all, and if it saw a collision within a stream.
Kev
If we only cared about collisions within a stream, we could trivially solve all issues already.
SaltyBones
And that is how you can tell that Kev is a VIP
SaltyBones
when he says something three people reply!
Dave Cridland
Kev, Well, we care about collisions for a user's account for MAM, and no further.
Kev
But having a useful id to refer to a message would be jolly useful, and I don't see how we can get there yet.
SaltyBones
Okay, so you're saying when two servers federate and you assume one of them is malicious how can the other make sure he doesn't get duplicate IDs?
Kev
Or non-malicious server with a malicious client.
Dave Cridland
SaltyBones, Do they care? ANd if not, should they?
jonasw
SaltyBones, you don’t use other servers stanza-ids in your own MAM
SaltyBones
Well, the malicious client would be detected by the server.
Kev
jonasw: Bloody useful if you could, though, no?
Kev
SaltyBones: How would the server know it's malicious, if it doesn't know all globally generated ids?
Alexhas left
SaltyBones
Kev, it can simply check if the client generates their IDs correctly...
Dave Cridland
Kev, Again, why would a server care?
Kev
Without exploding the size of IDs, at least.
jonasw
(we should mandate the use of shakespeare-inspired adjectives in each bloody sentence in this room, by the way)
jonasw
Kev, how would it be useful?
Kev
I'd like to send a reference to a stanza, where the reference makes sense in my archive, the MUC/MIX archive, and the user's archive.
Dave Cridland
jonasw, I don't think "bloody" is Shakespeare. "Submarine" is, though, as far as I remember.
SaltyBones
Kev, you simply make it a hash or hmac of something that both the client and the server know and the server can check the calculation.
Kev
"simply" :)
SaltyBones
Kev, but doesn't that only involve you, the muc/mix server and $other_client? i.e. only one server...?
Dave Cridland
Kev, Is the (from, id) attributes of the stanza sufficient? If not, why?
Zash
If this is for MAM purposes, then I will cry if I can't stick some time based bits into the ID
Kev
Dave Cridland: No, because the from gets rewritten.
Dave Cridland
Kev, When?
SaltyBones
Zash, what?
Kev
In a MUC or a MIX.
Zash
My MAM ids are basically yyyy-mm-dd-random
jonasw
Kev, those are different use-cases I think.
Kev
jonasw: If we cherry-pick the trivial use cases, it's going to be trivial to solve them :)
jonasw
Kev, this is where origin-id makes more sense. It is generated by the client. If the client doesn’t ensure that there’s enough entropy in there, references to their message won’t work.
SaltyBones
Zash, and this is important because?
Dave Cridland
Kev, Ah, so given a message from a MUC fanout, do you want to be able to identify the originating id? I'm not actually sure you always want to allow that.
Alexhas joined
jubalhhas joined
SaltyBones
I still don't get it. Why does from get rewritten?
SaltyBones
And what does that have to do with anything? :D
jonasw
SaltyBones, when you send a message from saltybones@yourdomain.example/client-resource to this MUC, we see the message from xsf@muc.xmpp.org/SaltyBones
jonasw
so the from is being rewritten.
jonasw
and if references are based on (from, id) pairs, you can’t use the same reference we all can use for your message
jonasw
(simplified, of course we could still refer to the reflection of the message, but that would break down with MIX, too, I think)
Dave Cridlandhas left
Kev
jonasw: Well, no, because if you can fake someone else's id, suddenly you can maliciously change the target of a reference.
jonasw
Kev, good point. so we indeed need something like (from, id) :(
SaltyBones
Wait what?
zinid
lol
SaltyBones
How can you now fake stuff again?
jonasw
but given that all our fanouts and from-rewriting things do actually do reflections, I’m not sure if that’s so bad, Kev?
jonasw
your local archive woudl still be able to resolve everything
Kev
jonasw: Well, it enforces a round-trip before you can refer to or correct anything, which isn't ideal.
jonasw
Kev, you normally know the "from" you’ll be having.
jonasw
and since you set the origin-id yourself, you already know everything you need
jonasw
(meh, races with server-enforced nickname changes in MUC)
Kev
Point.
jonasw
(but I guess those will be rare enough for us not to care)
SaltyBones
halp! :(
Dave Cridlandhas left
Kev
SaltyBones: Because there are potentially malicious entities.
tim@boese-ban.dehas left
danielhas left
SaltyBones
Are these only clients or servers as well?
Kev
jonasw: Still doesn't help where a user generates the same id twice, I think, and you're left with ambiguous references/corrections/whatever.
Kev
SaltyBones: Both.
SaltyBones
So why do you connect to a malicious server and expect things to work?
SaltyBones
Maybe that's not a good approach.
jonasw
Kev, that’s not an issue. put 256 bits of entropy in there and you are officially allowed to not care
Kev
Ah, if you have a mechanism for identifying malicious intent on the Internet, you could be a very famous person.
jonasw
Kev, and if we want to have defined behaviour in that case, always assume the most-recent message
ralphmhas left
Kev
jonasw: That only works for accidents, rather than manipulation, no?
marchas left
SaltyBones
Kev, but you cannot manipulate as long as the server is checking.
jonasw
Kev, indeed
jonasw
but if you manipulate your own origin-ids, it’s your own fault?
SaltyBones
Of course if we have malicious servers now a bit more work is required. :)
Kev
And I'm concerned that 'most recent' falls apart if you can manage different people receiving different subsets.
jonasw
can one inflict harm by producing duplicate origin-ids?
jonasw
I’m not convinced that this is actually an issue.
Kev
Most likely.
Kev
If I can cause the 'wrong' message to get corrected.
jonasw
maybe with something like Reactions based on (from, origin-id) references.
Kev
Or a like to apply to the wrong message.
Kev
If I can manipulate you into liking a Godwin instead of something sensible, that's not ideal.
andyhas joined
jonasw
meh
SaltyBones
But messages aren't authenticated anyway, if you are a malicious server you can claim to have received whatever likes from me...
jonasw
Kev, then the only way is a MUC/MIX stanza ID generated by the MUC/MIX and waiting for a round-trip before references can be done.
Kev
SaltyBones: Other way around.
jonasw
SaltyBones, all it needs for now is a malicious client though
jonasw
SaltyBones, or a malicious client+server pair if we do the hmac-stanza-id thing for origin-id
jonasw
I can still run my own server and make it fake origin-ids
danielhas left
andyhas left
andyhas joined
zinid
your own malicious server?
jonasw
Kev, I’d however argue that actively, from a client/own server side, make participants in a MUC/MIX receive only parts of the conversation would be rather trikcy
jonasw
and targeted parts even
jonasw
and that without them suspecting that things are broken in funny ways and thus not trusting the system anymore
Kev
jonasw: Ah, that's ok then, no exploits have ever involved doing anything tricky :)
jonasw
Kev, I see your point, but I’m not sure this is actually something which is reasonably exploitable.
jonasw
but yes
jonasw
unless we let the ID be generated by the fanout service, there’s no way to be sure I’m afraid
jonasw
I mean, I don’t have an issue with that, I’m fine with that even.
Kev
Yes. I don't see a way of solving this, which was why I brushed it under the carpet at the Summit.
jonasw
complicates things though
Guushas joined
jjrhhas left
marchas left
jonasw
Kev, I’m having a really hard time constructing a successful attack which wouldn’t be seen by the victim in the MUC case
jonasw
even when I omit arbitrary messages
jonasw
to only some participants
jonasw
(which would be really really tricky to achieve targetedly I think)
jonasw
ah, now I have it
jonasw
this is the scenario:
1. user A, "bad statement", origin-id=1
2. user A, "good statement", origin-id=1
3. user B, like (from=user A, origin-id=1)
if (2) isn’t seen by all users, they see user B liking "bad statement" instead of "good statement"
tuxhas joined
jonasw
with an arbitrary amount of messages between (2) and (3), it is also not too difficult to make people not see (2) in a MAM-less MUC.
Kev
Indeed.
Dave Cridlandhas left
jonasw
I don’t think there’s a way unless the MUC does things there
But, all it requires to prevent that is for the MUC to check that the ID is unique just like the assumed-non-malicious server did in our previous discussionn...
SaltyBones, but that is a rather expensive check to do
jonasw
you need to record IDs for all eternity, or have a defined way to generate the ID which can be executed by clients
jonasw
the latter would be tremendously tricky to get to work synchronously
jonasw
but possible...
SaltyBones
Which is a little more tricky because there is no obvious common "secret" like the stream-id but a simple Hash("counter") would do
jonasw
something like hash(message-counter || presence-id), where presence-id is an ID assigned to the client on join
SaltyBones
Kev, not that in this case I used the word simple again but I was totally lying...✎
SaltyBones
Kev, note that in this case I used the word simple again but I was totally lying... ✏
jonasw
hash counter doesn’t really work with multi-session nicks I think, it would lead to collisions or races
SaltyBones
jonasw, yeah it was just to get the idea. Indeed you would at least need some salt.
SaltyBones
But if the server gives you the salt on join and you use that to generate the IDs it should be good again.
jonasw
that would be verifable by the MUC and the MUC would drop stanzas which do not adhere to this schema
jonasw
(or rather, reject)
SaltyBones
and since the counter can be per-muc there is no issue with having it
jonasw
per client even
SaltyBones
I think :p
jonasw
Kev, ^
SaltyBones
well, yes per client and per muc
SaltyBones
just saying it doesn't seem to be a privacy issue
jonasw
SaltyBones, yeah
remkohas joined
jonasw
so origin-id = one-way-function(presence-id, message-counter), where presence-id is assigned on MUC join
andyhas left
jonasw
only need to define what happens when message-counter wraps around or becomes large or something
jonasw
(same thing for SM by the way, SM has wraparound semantics)
efrithas joined
SaltyBones
O_o
SaltyBones
Y U SEND SO MANY MESSAGES!?!!
ralphmhas joined
jonasw
SaltyBones, itym "y u have so long sessions"
andyhas joined
SaltyBones
still, if this is defined somewhere we can probably renegotiate the salt at that point...
jonasw
while wrapping around a 64bit counter in a single session is sure challenging, we need to be prepared if this is to be solid :)
jonasw
yeah
Marandahas joined
zinid
I hear counter?
zinid
sorry, I don't track the discussion
jonasw
zinid, I’m not going to repeat everything you can read in the backlog :)
zinid
as you wish
zinid
not that I care
SaltyBones
counter works in this case because you can restart counting when rejoining the muc
zinid
whatever you will end up will be shit anyways, so
Marandahas left
zinid
you already created 4 fucking ids: stanza-id, origin-id, attribute-id and SM id
zinid
maybe it's time to stop and think?
jonasw
what is an sm ID?
zinid
h
jonasw
right
jonasw
zinid, all of this is about reducing the number of IDs, so I thnik this is kinda what we’re doing?
zinid
jonasw, from what I see you want to add yet another counter
jonasw
zinid, to generate origin-ids, yes
jonasw
for MUCs and MIXes
jonasw
to make them verifiable by the service
jonasw
zinid, or rather, we’re replacing origin-id by some one-way-function(counter)
zinid
how this will cover sm id?
waqashas joined
SaltyBones
We haven't discussed sm-id, yet. Only the other three.
SaltyBones
I actually have no clue what sm-id is. :)
zinid
SaltyBones, then I need to wait while you recognize the problem 😉
SaltyBones
zinid, so reducing 4 -> 2 is not enough, eh? :p
jonasw
zinid, sm-id stays, but stanza-id (and attribute-id) could potentially become one-way-function(stream-id, sm-id)
zinid
2 is enough, however I think first should be counter and second should be routing information, and not an ID
jonasw
stanza-id being used for the archive only, origin-id being used throughout the network together with the sender address for refernecing a specific message
lovetoxhas joined
zinid
what ID will be used to sync?
ralphmhas joined
zinid
with SM IDs you just create a pointless queue in c2s state
Ge0rGhas left
SaltyBones
Okay, I think we are trying to solve a problem that is very orthogonal to stream management. But we have only discussed this a bit and it might very well not work even for what we want it to do.
zinid
SaltyBones, I don't think SM should be separated from archive
zinid
it's pointless
SaltyBones
Why?
jonasw
zinid, stanza-id is used for ysnc
jonasw
(for MAM sync that is)
zinid
SaltyBones, because why you need this separation?
zinid
SaltyBones, you keep messages in MAM and then put them in c2s SM queue
zinid
why can't you just inc(counter), store in MAM and send via c2s?
SaltyBones
Why are you asking me that? I am 100 % sure that you know more about SM than I do!
zinid
what I know about SM is that I need to maintain stupid queues inside c2s processes, which sucks as hell
zinid
even though a client can request those messages via MAM
zinid
if you receive an ID out of order, you just reconnect and ask everything started from the ID you received
zinid
and server will send you this from archive
zinid
and vice versa
SaltyBones
But SM-IDs are per-stream not per-message, right?
andyhas left
SaltyBones
At least that's what it looks like in https://xmpp.org/extensions/xep-0198.html
zinid
when you connect a server, you provide last seen ID and it will resend you everyting what's greater than this ID
zinid
SaltyBones, yes, they are totally separated instances
SamWhited
What if the thing you missed was an IQ or a presence that isn't stored in MAM?
SamWhited
If you temporarily disconnect and miss something, SM acks allow you to find out. Doesn't help as much if it only covers messages.
zinid
SamWhited, I think we can drop them
ralphmhas joined
zinid
SamWhited, we already don't care about IQs with Push, so why would we start care?
zinid
try to make jingle call when I'm in "push" mode
SamWhited
Does that not work? That does seem like something we need to care about to me
zinid
SamWhited, but we don't and we cant with push stuff
jonasw
shouldn’t an IQ trigger a push? :-O
zinid
SamWhited, what if I want to receive your software version and you're in push mode?
SamWhited
That seems like a problem that needs fixing then, not an example of something being done right that we should copy.
Steve Killehas left
zinid
anyway, you can keep IQs in MAM if you prefer
zinid
you need to keep subscription requests for sure there
zinid
we keep them already, but in a separate database due to historic reasons only
Martinhas left
andyhas joined
Steve Killehas joined
jonasw
IQs (which are inherent request-response, with exactly one response) in MAM sounds like a terrible idea.
SamWhited
yah, now you have to try and figure out which IQs are time sensitive, which are important to store, etc. it seems like a comlicated way to work around having a stanza counter.
zinid
SamWhited, but you have to decide now too: for example, after 5 minutes of inactivity (by default) a server just bounce all IQs from SM queue
zinid
*bounces
SamWhited
I'm not saying it's perfect, just that this doesn't seem like a good fix.
zinid
a fix? the behaviour is the same
SaltyBoneshas left
zinid
but we let a client to decide which IQ to reply
SamWhited
Except now you have to store all stanzas in MAM, or not store some stanzas and risk those being the ones that are dropped and you don't know you missed something. The point of SM is to make sure you know if you lose something, which is often important.
andyhas left
zinid
do you really want to loose incoming call?
zinid
that's how it works now: if somebody calls you and you're not connected you loose any track
SamWhited
As I said, I'm not claiming that the current solution is good; just that this makes it worse.
zinid
no, it makes it better
zinid
you just need to define what to store and what not, well, it's too much to redesign, yes
SamWhited
If you don't store everything you now lose the ability to detect connection drops though
zinid
sigh
Dave Cridlandhas left
Guushas left
Guushas joined
zinid
not sure how will you address missing call though
zinid
with your approach
SamWhited
I do not have an approach; I agree that is a problem that needs solving though.
Dave Cridlandhas left
Dave Cridlandhas left
Dave Cridlandhas left
Dave Cridlandhas left
SaltyBones
It seems like we need storage for management messages and storage for actual messages. I don't see why these should be mixed up.
Guushas left
rionhas left
Dave Cridlandhas left
Guushas joined
remkohas joined
zinid
we need to decide what to store and what not
Dave Cridlandhas left
SaltyBones
hm..that distinction was wrong, yeah
SaltyBones
there should be a storage until delivered and an optional long term storage
Dave Cridlandhas left
SaltyBones
users might not even want mam :p
zinid
if you don't want mam then just lose messages
SaltyBones
Okay, let me rephrase, some users might not want long-term storage of messages...
zinid
not sure why this should be specific to any approach
SaltyBones
Because I think that's what MAM is intended for and that's why using it for all messages as you suggest is strange and might have unexpected results if people built it under the assumption that it is for long term storage...
brahas left
brahas joined
zinid
servers can drop MAM archives on reconnect for example, what's the problem?
SaltyBones
Okay...
zinid
or later, when it's delivered
SaltyBones
this is just me confusing MAM and the other thing again...
SaltyBones
for the one-millionth time...
Ge0rG
What if I want my messages both on my pc and my mobile? I can't just drop MAM when my pc is online
zinid
you can if everything is delivered
zinid
anyway, what you are trying to say is a partial replication, and this problem is very hard to resolve
SaltyBones
what is the other thing again, server side archiving?
ralphmhas joined
SaltyBones
zinid, does MAM know if everything was delivered?
Seve/SouLhas left
Ge0rGhad a very concerning realization about guessable IDs and packet filters in smack earlier today.
zinid
SaltyBones, well, no, I think you cannot know that in general case
SaltyBones
zinid, but that is what SM does, right?
zinid
SaltyBones, well, if you introduce acks, then yes
SaltyBones
Hm.... :)
Holger
All this is orthogonal to the question whether having two separate stanza/message queues is sane. I agree with zinid that it isn't.
Zash
Two whatnow
Ge0rG
I'm saying for many years now that we need to replace 0184, 0198, 0280 and 0313 with one single proper message syncing thing.
Holger
SM is already mostly just an optimization, and I think we should fix the remaining issues to make stream management superfluous.
SaltyBones
I just want to point out that this is orthogonal to our earlier discussion about IDs :)
zinid
SaltyBones, I said "no" in general case because we have Bysantine General Problem, it's unresolvable, what we can gurarantee is sequential consistency
Dave Cridlandhas left
Ge0rG
SaltyBones: please show your ID before being allowed into this room :P
Tobiashas joined
SaltyBones
Holger, does anybody already know what those issues are?
danielhas left
Guushas left
Guushas joined
Zash
Ge0rG: Kidnap some server devs and lock yourself in a room until that one single proper message syncing thing to rule them all is properly implemented and XEP'd
Holger
SaltyBones: Syncing of outgoing messages (in a sane way) and maybe avoiding some round trips during session startup.
zinid
Zash, and watch them die 😉
Ge0rG
Zash: I'll lock you and zinid up in that room, and watch the live stream
SaltyBones
Holger, so, can we just turn off stream management and leave MAM and see what happens to find out?
Holger
Sure you can :-)
SaltyBones
Or do you think fixing up MAM is a bad idea and something new is required?
Zash
SaltyBones: That's what I did, actually. Haven't died from SM-lessness yet.
Holger
SaltyBones: I think you can already implement proper sync with MAM as-is. But something new is required to let clients implement this in a sane way, without having to de-duplicate and whatnot.
zinid
that's my point: what we need is to store messages and some other restricted stuff and call it a day
SaltyBones
Holger, so would unique message IDs solve this?
SaltyBones
I mean, at least de-duplication would be reasonably easy then, I guess.
jjrhhas left
SaltyBones
Of course some sort of counter makes more sense in this case...
Zash
SM has counters. MAM has server-issued, guarante to be unique message ids.
ralphmhas joined
SaltyBones
Why are IDs not a counter?
Holger
Not sure what sort of uniqueness we need to solve what problem. Didn't read the backlog, sorry :-) The thing missing for proper sync of outgoing messages is an algorithm to compute the MAM IDs of outgoing messages.
SaltyBones
I mean the MAM-IDs....or is that the same as the stanza-ids?
Holger
(Sorry, typed this before reading the last few messages.)
zinid
Zash, now answer the question "get me last messages I didn't receive" with current SM and MAM approach 😉
SaltyBones
Holger, what do you mean by outgoing? Messages that we sent?
Holger
Yes.
zinid
if you say "time", then no
Zash
zinid: query after = id of last message I saw
SaltyBones
Okay, that should be solved by the hash idea....if it works :p
Zash
cry over outgoing messages sent after that
Holger
SamWhited: I need to know their IDs so I can tell the server "give me all messages since $id".
SaltyBones
But indeed if we want to query MAM by a "point in time" or "counter" unique IDs are not really the best thing :)
remkohas joined
SaltyBones
Holger, but then the server still has to search the MAM archive for that ID and give you everything after it....
zinid
Zash, define after in distributed system 😉
SaltyBones
So a counter would be much better.
Holger
SaltyBones: Sure?
Zash
zinid: MAM archive ids on incoming messages
SaltyBones
Right?
Zash
zinid: after is a MAM term
zinid
Zash, but you need some ordering
pep.has left
Zash
zinid: huh
Zash
zinid: XMPP streams are ordered
zinid
yes, but you need to maintain a timestamp index in the database
Zash
You've lost me
Dave Cridlandhas left
Guushas left
Guushas joined
zinid
well, you're probably right that if we assume timestamp ordering then we don't need counters and SM at all
Zash
Huh?
Zash
In a MAM query, 'after' is a field that carries a MAM archive ID
zinid
so, what's the ordering? 🙂
andyhas joined
zinid
ID+1?
zinid
from what I understand you use timestamp+id, which means time ordering
Zash
Ordering?
Zash
As I said, you lost me. I have no idea what any of us are talking about anymore.
zinid
ok
vanitasvitaehas left
lskdjfhas left
Guushas left
Tobiashas joined
Dave Cridlandhas left
remkohas joined
la|r|mahas joined
danielhas left
rionhas joined
Dave Cridlandhas left
suzyohas joined
Dave Cridlandhas left
andyhas left
ralphmhas joined
Dave Cridlandhas left
rionhas left
Syndacehas left
Syndacehas joined
Dave Cridlandhas left
blablahas left
blablahas joined
remkohas joined
Dave Cridlandhas left
ralphmhas joined
Dave Cridlandhas left
Ge0rG
https://summerofcode.withgoogle.com/organizations/?sp-page=5 no xsf?
zinid
BEAM community is there 😛
andyhas joined
jjrhhas left
remkohas joined
ralphmhas joined
andyhas left
moparisthebest
SaltyBones: like 98% of what you were talking about is here https://github.com/moparisthebest/xmpp-ircd
Dave Cridlandhas left
Dave Cridlandhas left
Dave Cridlandhas left
moparisthebest
Basically it works if IRC users don't want nickserv or chanserv but they do and I never got back to it :)
SaltyBones
moparisthebest, you mean of a way to connect to a muc using the irc protocol?
moparisthebest
Yes
moparisthebest
Running that makes a muc look like an IRC server to an IRC client
Dave Cridlandhas left
SaltyBones
I'm pretty sure nobody wants that. At least I don't! :D
SaltyBones
But I think it might provide an excuse for people who have to convince irc users ;)
Dave Cridlandhas left
moparisthebest
SaltyBones: yesterday you said
moparisthebest
Hm...maybe the transport should be the other way round.
Offer an IRC server that connects to MUCs.
moparisthebest
That's what I was referring to
vanitasvitae
Ignite didnt make it into GSoC
vanitasvitae
:(
jubalhhas joined
Dave Cridlandhas left
danielhas left
jubalhhas left
jubalhhas joined
SaltyBones
moparisthebest, I know, I said that regarding the discussion of the KDE folks looking for an IM solution
andyhas joined
Flow
`> * Ge0rG had a very concerning realization about guessable IDs and packet filters in smack earlier today
Care to share?
marchas joined
jubalhhas left
Flow
Or is it just something in the ancient smack library yaxim uses?
Dave Cridlandhas left
remkohas joined
moparisthebest
SaltyBones, right so they could use a MUC, but also have an IRC server that IRC users could use
Dave Cridland
moparisthebest, I like this, BTW.
moparisthebest
and everyone would end up in the same place, but it'd be a muc
Dave Cridland
moparisthebest, Is the GPLv3 your addition or from telepaatti?
ralphmhas joined
lskdjfhas joined
Dave Cridland
Anyone know what servers support XEP-0288 these days?
Dave Cridland
Does this Prosody instance, for example?
moparisthebest
Dave Cridland, looks like the original telepaatti is gone, but GPLv3 goes back to at least the next fork looks like
andyhas left
andyhas joined
moparisthebest
honestly I kind of abandoned it because I like rust so much more than python nowadays but can't be asked to rewrite everything yet haha :)
> I need to know their IDs so I can tell the server "give me all messages since $id".
I know, apparently I wasn't clear about something, sorry. I'm not suggesting we get rid of MAM or leave everything exactly as it is today, just that MAM doesn't cover some important parts of SM and probably can't be made to cover it without significant downsides.
Ge0rG
Flow: it's affecting the old smack for sure, I'll have a look into smack 4 and let you know
SamWhited
(sorry for the long delay, got pulled into a meeting and then was AFK for a bit)
Dave Cridlandgreps his logs
Dave Cridlandhas left
Dave Cridland
So there's actually only one server I talk to that does bidi. That's scary.
Dave Cridlandhas left
moparisthebesthas left
Dave Cridlandhas left
winfriedhas left
Flow
Ge0rG, let me know if you didn't just re-discover CVE-2014-0364
Dave Cridlandhas left
Syndacehas left
Syndacehas joined
jubalhhas joined
Dave Cridlandhas left
Dave Cridlandhas left
Ge0rG
Flow: it's related but different
Zash
Dave Cridland: Is it mine? :)
marchas left
Dave Cridland
Zash, No, Lance's. Although actually grep might have gone into Annoying Binary Mode.
SaltyBones
-I ?
Dave Cridlandhas left
Dave Cridland
-a, but yeah.
Dave Cridlandhas left
Syndacehas joined
Dave Cridlandhas left
Dave Cridlandhas left
Dave Cridland
OK, so that's actually lots of servers doing bidi. Happy bunny, now. I've been putting the support into Metre.
Zash
Oh? Hm, feeling up for doing a survey?
Dave Cridland
Zash, What sort of survey?
Zash
"How many servers do 288?"
Zash
Hooooold on now
Zash
Is that the same number as ...
Zash
https://www.youtube.com/watch?v=azEvfD4C6ow !!!
Dave Cridlandhas left
tuxhas left
Dave Cridland
Zash, Looks like it's PSYC and Prosody.
Zash
Prosody doesn't do it out of the box, you need to go a bit out of your way to install a community module.
Dave Cridlandhas left
Dave Cridland
Really? Quite a few people have, then.
Dave Cridlandhas left
Zash
Dave Cridland: Question is, how much self-selection bias is there among people that have you in their roster? :)
Dave Cridland
Zash, Lots.
ralphmhas joined
remkohas joined
Dave Cridlandhas left
Dave Cridlandhas left
winfriedhas joined
winfriedhas joined
jubalhhas left
andyhas left
blablahas left
danielhas left
danielhas left
blablahas joined
moparisthebesthas joined
Dave Cridlandhas left
jubalhhas left
remkohas joined
Dave Cridlandhas left
zinidhas left
Dave Cridlandhas left
Dave Cridlandhas left
danielhas left
Alexhas left
Dave Cridlandhas left
Dave Cridlandhas left
remkohas joined
brahas left
brahas joined
danielhas joined
SamWhitedhas left
Alexhas joined
marchas joined
Tobiashas joined
Tobiashas joined
Dave Cridlandhas left
remkohas joined
brahas left
brahas joined
danielhas left
dwdhas joined
dwdhas joined
brahas left
brahas joined
danielhas joined
Dave Cridland
.
Zash
,
Dave Cridlandhas left
moparisthebesthas left
andyhas joined
Kev
M-Link does bidi, but we disabled it because of bug reports from Prosody about us not accepting stanzas down the right streams. Some of which I'm starting to question :)
Dave Cridland
:-)
mimi89999has joined
Dave Cridlandhas left
Dave Cridlandhas left
brahas left
brahas joined
Kevhas left
moparisthebesthas joined
remkohas joined
brahas left
brahas joined
tuxhas left
jubalhhas left
pep.has left
moparisthebesthas joined
vanitasvitaehas left
Dave Cridlandhas left
dwdhas joined
blablahas left
tuxhas left
marchas left
SaltyBoneshas left
andyhas left
remkohas joined
vanitasvitaehas left
danielhas left
dwd
So, Metre now does Bidi.
danielhas joined
marchas left
lumihas left
dwd
OK, this is weird. Metre is successfully negotiating Bidi with various Prosody servers. OK, great. But absolutely nothing ever tries to negotiate bidi with it, despite it offering the feature.