-
Zash
Does anyone remeber what the point of https://xmpp.org/extensions/xep-0292.html#self-iq was?
- Maranda eyes anjan broadcasting one presence any second 🙄
-
Ge0rG
What's wrong with that? MUCs are made to scale!
-
Maranda
Ge0rG: but my eyes don't scale 😞
-
Ge0rG
Maranda: are you reading the XML log?
-
Ge0rG
...and only see the blondes, the brunettes, ...
-
Maranda
Ge0rG: trying to catch something from the log tail actually 🙄☺️
- Maranda thinks: luckily now CSI deduplicates 😆
-
Ge0rG
depending on the CSI implementation and violation of RFCs.
- Maranda thinks he doesn't break in order processing.
-
Maranda
Although staying in CSI theme...
-
Maranda
"For example: A client sends a CSI active nonza, followed by an XMPP Ping request to the server. The server first changes the CSI state to active and flushes all eventually queued *stanzsa*."
-
Ge0rG
Maranda: but you are dropping stanzas
-
Maranda
Ge0rG, so whenever I filter a stanza silently I'm breaking in order processing...?
-
Maranda
Perhaps everyone violates rfcs
-
flow
Ge0rG, Does the RFC forbid that?
-
Guus
The RFC states that stanza order MUST be preserved.
-
flow
Yep, but I meant "dropping stanzas"
-
Maranda
Ge0rG, so since *everyone* does that I guess there isn't a real problem isn't it? :P
-
Guus
dropping != reordering, I believe.
-
Maranda
or you can't filter and drop stanzas anymore
-
Maranda
Guus, I'm not reordering
-
Guus
I'm not saying you are. 🙂
-
Maranda
:)
-
Guus
I do wonder how effective CSI can be though, given that there's bound to be a stanza that you don't want to queue, and you need to maintain stanza order.
-
Guus
doesn't give much room to play, I think
-
Guus
at least not for "drop all but the last" type of filtering.
-
Maranda
Guus, I'm just queuing presences so it's... rather easy to do
-
Guus
Maranda, sure, but as soon as you get an IQ ping, you'd have to flush all that, right? Before responding to the IQ ping, I mean.
-
Guus
(or any other non-queued stanza)
-
flow
Guus depends on who send the ping. but yes, usually you would flush if you have to send something anyway
-
Maranda
Guus, hmm that I didn't think about.
-
Guus
flow, an inbound ping to the client, I mean. I don't think it matters much who sent the ping? The server needs to preserve the order of all stanzas, not depending on who is the originator?
-
Ge0rG
So you want to tell me it is okay with the RFC to drop stanzas, but not ok to reorder them?
-
flow
Guus, I believe that XMPP could fall into the category where the order only matters with regard to two endpoints, not necesarily globally
-
Guus
flow: I'm not sure if that works. How would that work with private messages in MUCs, for example
-
flow
So if the ping originated from an entity which has no queued stanzas, you could only deliver the ping without flushing (but there is no reason you shouldn't flush)
-
Guus
Ge0rG: I'm not sure, to be honest. My point is even if it were allowed, I wonder if there's much practical benefit to CSI.
-
Guus
got to pick up kid from school. bbl
-
MattJ
Guus: flow: in practical terms the order between two bare JIDs must be preserved
-
Guus
Ge0rG: but isn't that how privacy lists work too, btw?
-
flow
MattJ, why bare JID and not just "JID"?
-
MattJ
flow: imagine reordering presence from a MUC join, which specifies you always receive your own presence last
- Maranda is rather confused now.
-
MattJ
In my opinion servers need to be very careful if they start arbitrarily dropping or reordering stanzas
-
Ge0rG
What MattJ said.
-
Maranda
well if it's bare jids IQs are excluded from the equation I suppose.
-
Maranda
clients won't have to answer to those.
-
Kev
Reordering is worse than not delivery✎ -
MattJ
Maranda: I didn't mean it like that
-
Kev
Reordering is worse than not delivering ✏
-
MattJ
But I'm on my phone so I don't think I can attempt a better explain
-
Kev
(Note that 'not delivering' isn't the same as 'dropping' - if you don't deliver, you still have to follow the rules for when to silently drop and when to error)
-
Ge0rG
My point is: if the RFC mandates in-order delivery, it is kind of implicit that it mandates delivery.
-
Ge0rG
So just dropping stanzas will violate the RFC
-
Maranda
... and so everyone in a way or another violates it.
-
Kev
Ge0rG: Delivery or error, yes.
-
MattJ
It's my understanding that it mandates delivery
-
MattJ
Or error
-
Ge0rG
Some cases allow silent dropping.
-
Zash
But XEPs can override core rules?
-
Ge0rG
Zash: yes
-
MattJ
Depending on stanza type, as you well know
-
Kev
Zash: Yes, by negotiation.
-
flow
MattJ, yep, the xep45 self-presence order is a good point
-
Maranda
Perhaps you're suggesting to do some magic amendment to the CSI spec?
-
Maranda
:P
- Maranda isn't sure about how much he doesn't violate anymore, although he doesn't reorder what's in the queue.
-
MattJ
Maranda: the CSI spec does not tell you to reorder or drop
-
flow
Maranda, some ppl will tell you that you violate the RFC, some will tell you possible the opposite. What matters is: Is it beneficial? Does it break things?
-
MattJ
Unfortunately it has some "examples" which I am not happy about, but consensus was to add them
-
MattJ
They are not part of CSI
-
MattJ
And they are not specified in any detail, which may be worse
-
flow
Then get rid of them from the XEP. But we should consider adding a wiki page for CSI where strategies are collected
-
flow
Working and non-working strategies that is
-
Zash
Pros and cons
-
MattJ
flow: iirc you were one of the main voices requesting their addition :)
-
Zash
And whys
-
Maranda
flow, it's very beneficial and doesn't break anything tbh.
-
MattJ
Maranda: how do you know it is beneficial?
-
Maranda
MattJ, deduplicating presences?
-
MattJ
This is another problem I have with current CSI usage
-
Maranda
MattJ, well it reduces queues *a lot* expecially if you join a lot of big mucs.
-
MattJ
Maranda: OK, spammy broken clients
-
MattJ
But the MUC could filter that
-
MattJ
More simply
-
Maranda
MattJ, irregardless of broken clients
-
MattJ
bbl, train arriving at station
-
flow
MattJ, wait, we are talking about the in order requirement § 5.1? I thought we talk about CSI strategies that are applied
-
Maranda
flow, I guess that if you queued all stanzas there wouldn't be problem with in order processing, the issues come out only if you don't filter all routable payload but only one or two of 'em.✎ -
Maranda
flow, I guess that if you queued all stanzas there wouldn't be problem with in order processing, the issues come out only if you don't filter all routable payloads but only one or two of 'em. ✏
-
Maranda
(like I currently do)
-
flow
Maranda, figures would be interesting. like the bytes that are not send by your implementation
-
Holger
Guus: > I do wonder how effective CSI can be though, given that there's bound to be a stanza that you don't want to queue, and you need to maintain stanza order. For me in practice there's often long periods where many presence stanzas are received, and nothing else.
-
Maranda
flow, well for my mobile account the average queue for staying inactive was like 600 presences without dedup, it went down to as little as 50-60.✎ -
Maranda
flow, well for my mobile account the average queue for staying inactive few hours was like 600 presences without dedup, it went down to as little as 50-60. ✏
-
flow
Maranda, so that is CSI with and without queue (presence) dedup?
-
Maranda
So I guess that matters somehow, about the actual bw you save I'm not sure but you can make an esteem.
-
Maranda
yes
-
flow
k, I was more wondering about CSI vs non-CSI
-
Maranda
so there's for sure a benefit expecially on MUC usage.
-
flow
and especially the longest time period the connection was idle
-
flow
time for mod_csi_stats
-
Maranda
well I suppose you can extract those numbers for my figure if the server queued around 600 presences mostly for mucs during inactive hours✎ -
Holger
flow: Dedup can of course make a great difference if you limit the queue size and flush when it's reached.
-
Maranda
well I suppose you can extract those numbers from my figure if the server queued around 600 presences mostly for mucs during inactive hours ✏
-
Maranda
for sure the reduction is reasonably around x10 magnitude
-
Maranda
sending only the last useful presence state
-
flow
Holger, good point
-
dos
heck, it doesn't need MUCs - I have an account with 700 transport contacts from a network with lousily defined presence semantics and CSI helps there as well in taming the unnecessary traffic
-
flow
dos, which network is this?
-
dos
facebook :P
-
Ge0rG
even grouping the presence XML stanzas over time and sending them in one big flush is good for the battery.
-
Maranda
Ge0rG, yes which is where deduplication helps...
-
Holger
> <MattJ> flow: imagine reordering presence from a MUC join, which specifies you always receive your own presence last I guess the problem with breaking this is that the client might mis-interpret presence received after the final self-presence as *newly* joined members?
-
MattJ
Yes
-
MattJ
dos: any actual stats?
-
Zash
Join a few noicy IRC channels
-
Ge0rG
Holger: imagine how that interacts with GC1.0 joins and intermittent s2s outages.
-
Zash
There's no GC1.0 la-la-la can't hear you
-
Ge0rG
I've done a CSI benchmark on a freenode transport some two years ago.
-
Ge0rG
I prepared a motion to Council to get rid of GC1.0, but it was not approved.
-
Ge0rG
And when I started making a PR to 0045, I realized that significant deletion is required
-
Zash
Next Prosody won't allow GC1.0 joins, have we seen any ill effects of it yet?
-
Holger
So you basically cannot de-dup MUC presence without breaking that 0045 guarantee unless you add special code for handling MUC joins.
-
Zash
Holger: Mmmmm, special cases
-
Holger
They're so elegant!
-
dos
MattJ: I haven't made any stats so far, but when I hacked up my Nokia N900 to actually use CSI, it was the first time I could actually observe prolonged silence on the stream
-
Zash
Specal casing on top of hacks on top of more hacks until your brain explodes
-
dos
I've got around 60 presences in the last few minutes
-
Maranda
fwiw I didn't yet see a single muc break.
-
Holger
Maranda: Do you have a CSI client that shows MUC joins/leaves?
-
Holger
Without one I'm not sure how you'd "see" that breakage.
-
jonasw
and even then it might actually be hard to notice
-
Maranda
Holger, join a small channel compare occupant list with another client?
-
dos
you're likely to join MUCs when CSI indicates "active", for instance
-
Holger
Maranda: The resulting occupant list will be correct in both cases.
-
Ge0rG
dos: another good one is client caps caching
-
Maranda
I'm sure that works, and I just did that because I was worried something like that happened because of deduplication.
-
Maranda
Holger, I didn't mean breaking the spec :P
-
Holger
Maranda: AFAICS the only problem would be bogus "Maranda joined" messages.
-
Maranda
I meant *it actually* broke anything.
-
Ge0rG
dos: my client will join MUCs when inactive, because a MUC ping timeout happened when idle
-
dos
Ge0rG: yup. thankfully, that was already done by the whichevet old Telepathy version is there :)
-
Holger
Maranda: Yes I think it doesn't *actually* break anything because Conversations & friends don't show "Maranda joined" messages anyway.
-
Maranda
Holger, I'd suppose clients look more at status 110 now than the order.
-
Maranda
for self presence
-
dos
Ge0rG: and how likely are you to notice some erroreneous joins displayed by your client in such case? ;D
-
Holger
Maranda: That statement makes no sense to me.
-
Ge0rG
dos: there are issues.
-
Holger
Maranda: The point is, when they receive their status-110-message they assume they have the full list of occupants. Any presence received afterwards would be interpreted as "Maranda joined", while in reality Maranda didn't join but was an occupant before I joined.
-
Kev
Am I being completely daft, or does https://tools.ietf.org/html/rfc6121#section-8.5 completely fail to say how to route presence type=error?
-
Maranda
Holger, hm ok now that's clearer :P
-
Maranda
I was more looking at actual occupant list desyncs rather than that anyways
-
Holger
Yes those shouldn't be broken by this kind of dedup.
-
jonasw
only temporarily
-
Holger
Well yes only until the client processed the flushed stanzas.
-
jonasw
or if your dedup has a bug
-
Holger
:-)
-
Holger
I haven't heard of bugs in XMPP code.
-
jonasw
that’s because they eat the messages telling you about them
-
Maranda
Holger, but question is how many mobile (one presumes most using CSI are those) actually show MUC joins?✎ -
Maranda
Holger, but question is how many mobile clients (one presumes most using CSI are those) actually show MUC joins? ✏
-
dos
Maemo 5 does, and I modified it to use CSI, so I guess I could test it with ejabberd :v
-
Maranda
dos: doo eet 😁
-
Holger
Maranda: Yes, as I said :-)
-
Holger
Maranda: The answer may well be zero.
-
Holger
Maranda: I'm doing that kind of dedup as well as the possible benefit seems significant to me, and I haven't seen any (reports on) breagage. But this *is* hairy of course.
-
Holger
"breagage", I'm drunk.
-
Maranda
Holger, ditto I mean as far as I like strict compliance, the current CSI code works and is (surprisingly) clean and simple for once, so as long as something doesn't break (e.g. like occupant list desync) I'm not inclined to touch it 😁
-
jonasw
MattJ, could you kick anjan temporarily? apparently, they’re flooding this room with useless <presence/> stanzas
-
MattJ
:)
-
MattJ
An unsurprising outcome, in hindsight
-
Ge0rG
GC1.0 FTW!
-
Ge0rG
Unfortunately, there is no way to burn GC1.0 without "breaking" usability.
-
Maranda
damn I actually needed anjan to test something
-
Ge0rG
Maranda: get yourself kickbanned as well, then you are together with anjan again 🤔
-
Maranda
Ge0rG, :P
-
Maranda
Ge0rG, no need it actually works.
-
Maranda
And that problem wasnt gc1.0 related me thinks just an old issue in gajim that made it loop resending muc presences
-
Ge0rG
Maranda: but with GC1.0, the ghost returned after getting kicked
-
Maranda
🤔
-
Andrew Nenakhov
Btw, are there any docs on GC1.0? We didn't find it anywhere, only references in XEP-0045
-
dos
hmm... when client receives a message directed to other client via carbon that has a delivery receipt request, should it send one?
-
daniel
That can make sense
-
dos
nvm, xep mentions it, so I guess it should: "Because it is possible for a given content message to be delivered to multiple XMPP resources controlled by the recipient, the sender of the content message needs to be prepared to receive multiple ack messages."
-
daniel
The xep is probably not talking about carbons though
-
daniel
But still I belive sending them - while not ideal - is the best thing we can do for now
-
dos
I think this is the only mention of a situation with multiple acks, so I'd interpret it as "it is possible, for instance with bare JID messages, message carbons are enabled etc."
-
dos
so nothing really mandates you to send them, but conforming receipt requestor should be prepared to handle them, so it should be fine
-
Andrew Nenakhov
I'd rather send it. Even if sender didn't intend my device to receive this message, I still received it, because my carbons are configured this way. So, it's delivered to me, and I should probably send a receipt.
-
Andrew Nenakhov
I also don't like when remote party dictates how my devices work. "I'll talk with you over this Android phone, not over that desktop client"
-
jonasw
#OMEMO
-
Ge0rG
you could send a carbon of a receipt 😁
-
dos
Psi does it even better right now - it sends a receipt for an outgoing carbon that requests one :D
-
Ge0rG
Heh.
-
Ge0rG
dos: as long as it doesn't request a receipt for a receipt, everything is good.
-
Zash
Something something two generals problem
-
lovetox
in my opinion of course a receipt should be send✎ -
lovetox
in my opinion of course a receipt should be sent ✏
-
lovetox
a carbon means only the server received the message, not the client
-
Ge0rG
lovetox: what?
-
lovetox
what part of what i said is not clear?
-
Ge0rG
> a carbon means only the server received the message, not the client
-
Ge0rG
maybe I'm just lacking the proper context for this
-
lovetox
you receive a "received" carbon, now you can wait 5 seconds or any amount of time you want, if there is no receipt coming in from your other client, you should assume he didnt receive the message
-
lovetox
and ack
-
Ge0rG
I'd just ack anyway
-
Zash
ack for carbons, ack for mam, ack all teh time
-
Ge0rG
we need multi-ack!
-
jonasw
if we only had a mechanism for acknowledging stuff
-
jonasw
like, IQs
-
Zash
or send acks when the server writes into mam
-
jonasw
Zash, I’d like that
-
Zash
wait have we invented email then?
-
lovetox
i think thats exactly not what receipts are about
-
lovetox
receipt mean the client received the message
-
lovetox
not a server archive, which cant even determine if the message is encrypted and the client can decrypt it
-
lovetox
really there are error messages for when s2s or c2s does not work
-
jonasw
except when there aren’t
-
Zash
what if we just stop worrying about everything
-
Zash
messages get through or they don't
-
lovetox
yes i say this often, Gajim has receipts disabled by default
-
lovetox
i use xmpp now for some years, never had a problem with not received messages, except when encryption was involved
-
Ge0rG
.I agree with lovetox in that.
-
jonasw
sure, let’s not worry about reliability
-
jonasw
lovetox, I had
-
Ge0rG
lovetox: sorry, but your story is a huge anomaly. I lose messages all the time
-
jonasw
a lot
-
Ge0rG
sometimes I only lose carbons because my contact just turns off their phone's wifi
-
Zash
did you say something?
-
jonasw
but many of those would probably be solved by MAM
-
daniel
Ge0rG: maybe you smacks implementation is broken?
-
daniel
Or the fact that you don't have mam?
-
Ge0rG
daniel: maybe XMPP is broken?
-
daniel
Apparently not for me and lovetox
-
daniel
Who do implement both mam and 198
-
Ge0rG
daniel: not sure about ejabberd, but another widely-deployed XMPP server will error-bounce all stanzas when a hibernated 0198 session gets destroyed.
-
Ge0rG
besides, I probably just don't know zilch about how xmpp works :P
-
daniel
But a bounced message is not a lost one
-
daniel
Unless you don't track your errors
-
Zash
Schrödingers bouncy deliveries
-
dos
afaik ejabberd has a config entry for that and can either put them to offline storage or bounce
-
daniel
Yeah so does prosody. But complaining about things is always more fun
-
Zash
The day humans stop complaining is the day we gone extinct.
-
Ge0rG
daniel: the prosody module doesn't work if you have multiple clients
-
Ge0rG
But yeah, complaining is more fun.
-
Ge0rG
Also I'm not sure what tracking errors gives you, unless you do automatic retransmission, which is a huge PITA
-
Maranda
We need some ELE first or later anyways
-
jonasw
ELE?
-
Maranda
jonasw: Extinction Level Event
-
Ge0rG
Maranda: https://syonyk.blogspot.com/2015/08/i-am-backsideist.html
-
Maranda
sounds like some complex read which I'm a bit too tired to start :O
-
Maranda
> not sure about ejabberd, but another widely-deployed XMPP server will error-bounce all stanzas when a hibernated 0198 session gets destroyed. well I have the "Ge0rG's maneuver" when there's at least one session online supporting carbons :P
-
Maranda
(aka "assume like it's delivered")
-
Maranda
(... and don't bounce it)
-
Ge0rG
Maranda: is that upstreamed?
-
Maranda
Ge0rG, it's in Metronome from quite a bit yes.
-
Ge0rG
Maranda: that's not what I mean.
-
Maranda
Then I'm too tired to understand what you mean, elaborate plx :P
-
Ge0rG
Maranda: prosody
-
Maranda
And I don't think so, dunno.
-
Maranda
> is quite unpleasantly high, the state of infrastructure is quite sad (when was the last bridge collapse?), 6 days ago.
-
Ge0rG
Not speaking of irc bridges