Hello Kev,
for the Tweets for the Newsletter I would like to ask for access to the Twitter account. And maybe Mastadon aswell, if someone has the access, too.
jcbrand, Seve:
Marandahas left
waqashas left
lovetoxhas joined
Marandahas joined
remkohas left
j.rhas joined
Yagizahas left
Yagizahas joined
adiaholic_has left
adiaholic_has joined
stpeterhas joined
stpeterhas left
remkohas joined
papatutuwawahas joined
adiaholic_has left
adiaholic_has joined
Viktor Lindberghas left
Viktor Lindberghas joined
lovetoxhas left
stpeterhas joined
stpeterhas left
!XSF_Martinhas left
!XSF_Martinhas joined
strypeyhas joined
strypeyhas left
mukt2has joined
strypeyhas joined
strypeyhas left
krauqhas left
krauqhas joined
mukt2has left
Yagizahas left
adiaholic_has left
adiaholic_has joined
stpeterhas joined
stpeterhas left
lorddavidiiihas left
eevvoorhas joined
Guus
What are sensible max size restrictions on pubsub items these days?
lorddavidiiihas joined
sonnyhas left
sonnyhas joined
debaclehas joined
karoshihas left
sonnyhas left
sonnyhas joined
lovetoxhas joined
stpeterhas joined
stpeterhas left
Zash
Size of the items?
Tobiashas left
Tobiashas joined
adiaholic_has left
adiaholic_has joined
karoshihas joined
Marandahas left
Marandahas joined
Zash
Guus: I suppose you want to keep https://xmpp.org/extensions/xep-0060.html#subscriber-retrieve-returnall under 10k or whatever stanza size limit you may have
Zash
So, stanza size limit / max items (per rsm page, if supported) - some margins
Marandahas left
Marandahas joined
flow
it's a pitty that xep60 doesn't use the same scheme as MAM when returning multiple items
karoshihas left
strypeyhas joined
strypeyhas left
stpeterhas joined
stpeterhas left
adiaholic_has left
adiaholic_has joined
werdanhas joined
werdanhas left
Zash
Let's invent time travel and send that idea back to when '60 was designed
adiaholic_has left
adiaholic_has joined
Zash
Wasn't PubSub MAM a thing we discussed at Summit before?
sonnyhas left
sonnyhas joined
Yagizahas joined
mukt2has joined
Kev
PubSub MAM is a thing, we deploy it and it's in the XEP, no?
stpeterhas joined
stpeterhas left
Kev
Unless I'm going mad.
flow
I am probably not very knowledgeable on pubsub, but what exactly is the benefit of pubsub mam?
Kev
You can get the history of items on a node.
flow
given that it appears that there is pubsub+rsm already
Kev
Xep60 gives you the 'current state', MAM gives you the history.
flow
the history of items means that I can access previous revisions of an item with id 'A'?
Kev
As an analogy, xep60 gives you the filesystem, 313 gives you the backups.
Kev
Yes, it would show you the history of things published to the node.
flow
i'd like to point out that I always found that motivation for PubSub+MAM missing in xep313
flow
mam introduction currently states "or to view old items in a pubsub node."
Yagizahas left
Yagizahas joined
flow
and I always wondered "can't I read old items already with plain xep60?"
Yagizahas left
Yagizahas joined
intosihas left
flow
and i guess xep-pubsub-mam does not exist (yet)?
Kev
I thought I wrote pubsub into 313.
Viktor Lindberghas left
flow
you did, but it was removed early this year
Kev
Oh.
Kev
That seems...not good.
jonas’
it was removed a week or two back
jonas’
it’s a separate XEP (not yet)
jonas’
because I forgot to do things editor-wise
flow
Version 0.7.0 (2020-03-20)
jonas’
I was reminded by reading that paragraph
jonas’
0.7.0 was published a few weeks back
jonas’
https://github.com/xsf/xeps/pull/921 this one I need to process, will soon
flow
IIRC the mam pubsub part was not that substantial, e.g. it still wasn't clear to me why and when I would want to use it, and what it can do for me (but maybe that's just me)
flow
simply stating that you can access previous item revisions with it may be a good first step
Viktor Lindberghas joined
flow
but eventually I think there is much more to it, like the retention policy of this "historical" information
intosihas joined
Kev
Isn't that exactly the same as with IM MAM?
Zash
So it'd let you replay notifications, or somesuch?
Kev
That's one way of looking at it I guess.
Kev
But if you're in an environment where pubsub is containing data that's a little bit like a message, rather than just transient notifications, the argument for MAM is the same as it is for IM, I think.
Kev
e.g. if you're doing FDP or something.
Zash
Notifications including retractions and other stuff
lovetox
Found a 8 year old Issue from MattJ in Gajim, where he proposes to use stream management acks instead of pings to detect a timeout
lovetox
whats the opinion about this today?
flow
lovetox, timeout or liveness detection?
lovetox
the later
jonas’
lovetox, "yes please"
Ge0rG
lovetox: if you send multiple <r/> without data in between, you can't decide which of them a resulting <a/> is for
jonas’
Ge0rG, doesn’t matter
jonas’
that you received an <a/> is enough to know that you’re still live
Ge0rG
jonas’: doesn't matter for liveness detection, but does matter for server latency measurement
jonas’
that is true, but you can do that passively
jonas’
(if you really wanted to)
lovetox
Ge0rG, why would i send multiple </r> though?
flow
sm <r/> <a/> has fewer bytes on the wire compared to xmpp' pinging the server
lovetox
like without waiting for the <a>
flow
but IMHO not worth it, still a nice microoptimization
Ge0rG
I think I'm using actual pings because I encountered some limitation in the Smack API...
jonas’
and fewer corner cases ("what if the server does not support ping?")
Ge0rG
lovetox: e.g. when doing it from a periodic timer
jonas’
Ge0rG, also, you can decide because of in-order processing :)
lovetox
yes Ge0rG but i would always wait for the <a> or? before issuing a new one?
jonas’
lovetox, waiting for "the" <a/> is tricky
Ge0rG
jonas’: but what if they get reordered or one is lost?!
jonas’
Ge0rG, tcp?!
eevvoorhas left
jonas’
the main problem tho for corellation is:
> Either party MAY send an <a/> element at any time (e.g., after it has received a certain number of stanzas, or after a certain period of time), even if it has not received an <r/> element from the other party.
Ge0rG
jonas’: remember that fortigate TLS 1.3 issue I talked about recently?
jonas’
please no
lovetox
and second question is, do you think its a good idea to send <r> after a period of time rather than after a number of stanzas?
flow
lovetox, I see no reason why you would request SM acks based on a period of time
jonas’
lovetox, so I’d recommedn to send <r/> when you haven’t received anything from the server for more than N seconds
jonas’
that you send <r/> specifically doesn’t matter, but sending *anything* which provokes a response from the server is good
jonas’
<r/> lends itself because it’s cheap and has a beneficial side effect (updating the SM sync)
flow
you request SM acks to drain your, potentially existing, unack'ed stanza queue
Ge0rG
lovetox: it makes sense to send an <r/> after you sent out all the important stanzas that were queued, and then as jonas’ said after a longer period of idletime
jonas’
then when you don’t receive *anything* (not specifically <a/>, anything from the server is fine) after another timeout, you consider the stream dead
lovetox
2 seperate issues
jonas’
this is much simpler than keeping track of pings and it’s also much better, because it can better deal with saturated high-latency links
lovetox
lets move away from alive detection
jonas’
but you were talking about liveness detection?
lovetox
currently i request r after each stanza
jonas’
that’s certainly a valid strategy
lovetox
yes jonas’ and now i want to talk about the second issue
but I hadn’t answered your first question yet :P ✏
lovetox
:D, you said its a good idea
jonas’
lovetox, I’d say for non-aliveness-detection purposes, do <r/> on each socket flush
lovetox
Ge0rG, has some server latency issue i dont undestand and im not sure i want to care
Ge0rG
I have a server latency issue?
lovetox
jonas’, as i dont fliush the socket myself this is not a option
Ge0rG
-ENOPARSE
lovetox
i can ask for r after i get the callback that my write was successful
lovetox
which is probably the same
Ge0rG
if you don't flush the socket, maybe Nagle will
lovetox
but im not sure that is really a optimization
Ge0rG
lovetox: it's the opposite of an optimization
lovetox
but i would have to meassure that first
Ge0rG
lovetox: ideally, you want the stanza(s) and the <r/> to go out in the same TCP frame
Ge0rG
instead of wasting roundtrips
lovetox
yeah i use a library that abstracts that for me, so i have no idea when and how much is in a tcp frame
Ge0rG
I know this is a controversial opinion, but I consider Gajim to be a mobile client when I'm on 2G tethering, riding a train in the middle of German nowhere.
Ge0rG
lovetox: there is also TLS in between ;)
flow
and compression
Ge0rG
so it's borderline impossible to find out.
jonas’
lovetox, then maybe wait for 1ms after you wrote a stanza to the socket and send a <r/> after that
Ge0rG
jonas’: but why?
flow
yeah, jonas’, why?
jonas’
lovetox, and reset the timer after each stanza
jonas’
flow, Ge0rG, to avoid sending multiple <r/>s for no good reason?
flow
I think I lost slighlty what this discussion is about
lovetox
yeah why multiple r?
mukt2has left
lovetox
look i have a send_stanza method
flow
simply send an <r/> after a few stanzas or when you hit a high watermark of unack'ed stanzas
lovetox
and everytime this is called, i also call send_r
lovetox
its very simple
lovetox
there can never be multiple r for one stanza
lovetox
its alway stanza, r, stanza, r, stanza, r
jonas’
flow, so you send <r/> after N stanzas and the "last" batch of stanzas has N-1. You’re now left with N-1 unacked stanzas "forever".
jonas’
for no good reason.
LNJhas left
Viktor Lindberghas left
LNJhas joined
flow
jonas’, well if the unacked stanzas are not important then this is fine
lovetox
thats why my idea was, something like, every 3 seconds (of course only if there are unacked stanzas=
jonas’
flow, but if they are? what makes a stanza "important"?
flow
smack has predicates that decide when to request an stanza✎
jonas’
lovetox, please don’t send multiple <r/> for any unacked stanza
flow
smack has predicates that decide when to issue an <r/> ✏
jonas’
you’ll just pile on to an apparently already existing network congestion
flow
trivial example is "messages with <body/>"
remkohas left
jonas’
especially not as often as every 3 seconds.
Ge0rG
jonas’: a stanza is important if it is not ephemeral and not going to time out if left in hibernation
lovetox
jonas, of course only one r is in flight
lovetox
please
jonas’
lovetox, it’ll stay in-flight forever though
jonas’
if the server doesn’t reply
jonas’
or if the server takes long to reply
karoshihas joined
lovetox
but it MUST reply
jonas’
yes, but taht may take some time
lovetox
yeah and?
jonas’
e.g. 5 minutes on a crappy mobile link
jonas’
in that time you have amassed 100 <r/>s
jonas’
to which the server MUST reply
jonas’
so not only have you congested your uplink, but also your downlink now
jonas’
unneccesarily
lovetox
i propose to send an r every 3 seconds only if there is no r in flight
lovetox
so how would i amass 100 r
jonas’
oh, you messed up a negation earlier then
lovetox
maybe, although i dont see it
jonas’
oh, I imagined an "if" in a sentence when there was none
jonas’
sorry for that
jonas’
lovetox, so, problem is, tracking <r/> and <a/> isn’t really possible
jonas’
so how would you define "in-flight"?
Ge0rG
every 3 seconds?!
jonas’
Ge0rG, (of course only if there are unacked stanzas)
jonas’
Ge0rG, + the in-flight this results in "do <r/> if there are unacked stanzas 3s after the last stanza has been sent" or something like that
Ge0rG
that's an interesting trade-off.
jonas’
which is kind of what I proposed earlier, just with a much higher delay, causing less battery efficiency
jonas’
and with an unclear definition of "in-flight"
lovetox
one could switch the dealy dynamically
Ge0rG
How long do you wait after generating a stanza and before concluding that there are no more stanzas going to be sent
lovetox
when i have unacked stanzas its 3s, afterwards its 20
lovetox
or soemothing
jonas’
lovetox, what do you send an <r/> for if you don’t have unacke:?✎
jonas’
lovetox, what do you send an <r/> for if you don’t have unacked stanzas? ✏
lovetox
for alive detection ^^
stpeterhas joined
stpeterhas left
lovetox
back to issue one
jonas’
lovetox, see what I wrote in regard of that one please then
Ge0rG
maybe it's a good thing to just put all the stanzas that you want to send into a queue, then let the OS/event API know you have something to send, and then add a single <r> at the end of the queue
Ge0rG
...on delivery
lovetox
generally i want to send every data instantly
jonas’
Ge0rG, a simpler version of that, requiring less cooperation from the OS, is to re-implement that TCP-cork-ish stuff in userspace: N ms after the last stanza has been put in the tx queue of the socket, add <r/>. reset the timer on each stanza.
lovetox
to put it into a queue then wait some time, maybe more comes
lovetox
seems not very fast
jonas’
lovetox, the OS will do that for you anyways. and it’s ok, since the time is generally rather short.
Ge0rG
jonas’: but then you are competing with TCP cork and end up sending the <r> in a separate TLS frame in a separate TCP frame
Ge0rG
so you add something like 20x the overhead to <r/>
jonas’
Ge0rG, only if your delay is longer than the tcp delay
jonas’
which you can query from the OS
jonas’
and if that’s too complex for you (I don’t blame you), then just send <r/> after each stanza
lovetox, https://docs.zombofant.net/aioxmpp/devel/api/internal/protocol.html#aioxmpp.protocol.XMLStream.deadtime_soft_limit here’s how liveness detection it works in aioxmpp.
The docs at that place are written from a very internal perspective, hence user == the aioxmpp client class in this case.
-> when the soft timeout expires, the stream sends <r/>. when the hard timeout expires, the stream is considered dead. timeouts reset when (any) data is received (it doesn’t even have to be a stanza).
remkohas joined
jonas’
the frontend to that is https://docs.zombofant.net/aioxmpp/devel/api/public/stream.html#timeouts-stream-aliveness-checks
jonas’
("frontend")
lovetox
thanks i look into it
jonas’
if anything is unclear, feel free to ask. if anything seems stupid, please let me know, because currently I think this is the best anyone can do given networking realities.
jonas’
(and what I particularly love about it is the simplicity, because it only uses two timers. no tracking of stanzas or whatnot)
jonas’
(and the building blocks can easily be reused for MUC aliveness checking and stuff like that)
jonas’
(because the concepts are really the same everywhere)
jonas’
also relevant: https://docs.zombofant.net/aioxmpp/devel/api/internal/utils.html?highlight=alive#aioxmpp.utils.AlivenessMonitor✎
jonas’
also relevant: https://docs.zombofant.net/aioxmpp/devel/api/internal/utils.html#aioxmpp.utils.AlivenessMonitor ✏
Viktor Lindberghas left
jonas’
oh and this is also an amazing side-effect: "If the peer is pinging us in an interval which works for us (i.e. is less than the soft limit), we don’t need to ping the peer; no extra logic required."
thorstenhas left
LNJhas left
lovetox
yeah less timers the better
lovetox
i hate timers
Viktor Lindberghas joined
LNJhas joined
jonas’
two timers, no tracking.
karoshihas left
thorstenhas joined
karoshihas joined
stpeterhas joined
stpeterhas left
mukt2has joined
sonnyhas left
sonnyhas joined
krauqhas left
krauqhas joined
mukt2has left
lovetoxhas left
mukt2has joined
lovetoxhas joined
mukt2has left
mukt2has joined
lovetoxhas left
lovetoxhas joined
stpeterhas joined
stpeterhas left
karoshihas left
eevvoorhas joined
adiaholic_has left
adiaholic_has joined
mukt2has left
mukt2has joined
debaclehas left
krauqhas left
krauqhas joined
Guushas left
Guushas joined
Guushas left
Guushas joined
stpeterhas joined
stpeterhas left
Guushas left
mukt2has left
Guushas joined
Guushas left
Guushas joined
bearhas joined
mimi89999has left
mimi89999has joined
karoshihas joined
lovetoxhas left
karoshihas left
karoshihas joined
stpeterhas joined
stpeterhas left
mukt2has joined
lovetoxhas joined
lovetoxhas left
lovetoxhas joined
lovetoxhas left
lovetoxhas joined
stpeterhas joined
stpeterhas left
karoshihas left
mukt2has left
mukt2has joined
edhelashas left
edhelashas joined
lovetoxhas left
remkohas left
karoshihas joined
krauqhas left
krauqhas joined
mukt2has left
remkohas joined
amuza@riseup.nethas joined
karoshihas left
adiaholic_has left
adiaholic_has joined
adiaholic_has left
adiaholic_has joined
lovetoxhas joined
Viktor Lindberghas left
krauqhas left
krauqhas joined
karoshihas joined
Viktor Lindberghas joined
lovetoxhas left
mukt2has joined
LNJhas left
LNJhas joined
lovetoxhas joined
Marandahas left
Marandahas joined
Mikaelahas left
adiaholic_has left
adiaholic_has joined
Mikaelahas joined
MAOhas joined
MAOhas left
karoshihas left
karoshihas joined
lovetoxhas left
lovetoxhas joined
Shellhas left
Shellhas joined
Viktor Lindberghas left
Viktor Lindberghas joined
sonnyhas left
sonnyhas joined
Lancehas joined
sonnyhas left
sonnyhas joined
Wojtekhas joined
Shellhas left
Shellhas joined
adiaholic_has left
adiaholic_has joined
mukt2has left
lovetoxhas left
mukt2has joined
neshtaxmpphas left
debaclehas joined
krauqhas left
krauqhas joined
j.rhas left
krauqhas left
krauqhas joined
Viktor Lindberghas left
krauqhas left
krauqhas joined
j.rhas joined
krauqhas left
krauqhas joined
Viktor Lindberghas joined
neshtaxmpphas joined
Danielhas left
Danielhas joined
Danielhas left
Danielhas joined
Danielhas left
Danielhas joined
Dele Olajidehas left
remkohas left
sonnyhas left
sonnyhas joined
sonnyhas left
sonnyhas joined
eevvoorhas left
mukt2has left
mukt2has joined
sonnyhas left
sonnyhas joined
sonnyhas left
sonnyhas joined
eevvoorhas joined
sonnyhas left
sonnyhas joined
Danielhas left
Danielhas joined
Wojtekhas left
remkohas joined
adiaholic_has left
adiaholic_has joined
winfriedhas left
winfriedhas joined
eevvoorhas left
eevvoorhas joined
remkohas left
lovetoxhas joined
Danielhas left
Danielhas joined
lovetoxhas left
lovetoxhas joined
bearhas left
Danielhas left
Danielhas joined
DebXWoodyhas joined
Vaulorhas left
Vaulorhas joined
karoshihas left
Wojtekhas joined
etahas left
etahas joined
calvinhas joined
lovetoxhas left
lovetoxhas joined
karoshihas joined
debaclehas left
serge90has joined
Shellhas left
Shellhas joined
lovetoxhas left
remkohas joined
karoshihas left
sonnyhas left
sonnyhas joined
adiaholic_has left
sonnyhas left
sonnyhas joined
bearhas joined
karoshihas joined
DebXWoodyhas left
eevvoorhas left
archas joined
sonnyhas left
sonnyhas joined
krauqhas left
krauqhas joined
Shellhas left
florettahas left
florettahas joined
DebXWoodyhas joined
karoshihas left
karoshihas joined
lovetoxhas joined
Yagizahas left
archas left
archas joined
Syndacehas left
karoshihas left
karoshihas joined
archas left
archas joined
archas left
archas joined
archas left
archas joined
Syndacehas joined
bearhas left
karoshihas left
karoshihas joined
remkohas left
remkohas joined
karoshihas left
karoshihas joined
karoshihas left
eevvoorhas joined
karoshihas joined
karoshihas left
archas left
archas joined
archas left
archas joined
remkohas left
remkohas joined
papatutuwawahas left
archas left
archas joined
remkohas left
remkohas joined
ralphmhas left
ralphmhas joined
Shellhas joined
papatutuwawahas joined
karoshihas joined
debaclehas joined
DebXWoodyhas left
bearhas joined
mukt2has left
remkohas left
archas left
archas joined
Lancehas left
karoshihas left
mukt2has joined
karoshihas joined
jonas’has left
jonas’has joined
mukt2has left
mukt2has joined
Lancehas joined
Zashhas left
Zashhas joined
Lancehas left
karoshihas left
Zashhas left
mukt2has left
mukt2has joined
sonnyhas left
sonnyhas joined
karoshihas joined
eevvoorhas left
Nekithas left
Nekithas joined
Zashhas joined
Tobiashas left
karoshihas left
debaclehas left
mukt2has left
edhelashas left
edhelashas joined
edhelashas left
edhelashas joined
lovetoxhas left
karoshihas joined
karoshihas left
mukt2has joined
LNJhas left
winfriedhas left
winfriedhas joined
Wojtek
not sure what's the best place to ask (maybe jdev?) but here goes - is there somewhere a description of the current setup of xmpp.net? or it's sources? "about" (https://xmpp.net/about.php) only states that it's based on xmpppoke and it seems that https://github.com/stpeter/xmppdotnet is not exactly the same thing