emusHello Kev,
for the Tweets for the Newsletter I would like to ask for access to the Twitter account. And maybe Mastadon aswell, if someone has the access, too.
jcbrand, Seve:
Marandahas left
waqashas left
lovetoxhas joined
Marandahas joined
remkohas left
j.rhas joined
Yagizahas left
Yagizahas joined
adiaholic_has left
adiaholic_has joined
stpeterhas joined
stpeterhas left
remkohas joined
papatutuwawahas joined
adiaholic_has left
adiaholic_has joined
Viktor Lindberghas left
Viktor Lindberghas joined
lovetoxhas left
stpeterhas joined
stpeterhas left
!XSF_Martinhas left
!XSF_Martinhas joined
strypeyhas joined
strypeyhas left
mukt2has joined
strypeyhas joined
strypeyhas left
krauqhas left
krauqhas joined
mukt2has left
Yagizahas left
adiaholic_has left
adiaholic_has joined
stpeterhas joined
stpeterhas left
lorddavidiiihas left
eevvoorhas joined
GuusWhat are sensible max size restrictions on pubsub items these days?
lorddavidiiihas joined
sonnyhas left
sonnyhas joined
debaclehas joined
karoshihas left
sonnyhas left
sonnyhas joined
lovetoxhas joined
stpeterhas joined
stpeterhas left
ZashSize of the items?
Tobiashas left
Tobiashas joined
adiaholic_has left
adiaholic_has joined
karoshihas joined
Marandahas left
Marandahas joined
ZashGuus: I suppose you want to keep https://xmpp.org/extensions/xep-0060.html#subscriber-retrieve-returnall under 10k or whatever stanza size limit you may have
ZashSo, stanza size limit / max items (per rsm page, if supported) - some margins
Marandahas left
Marandahas joined
flowit's a pitty that xep60 doesn't use the same scheme as MAM when returning multiple items
karoshihas left
strypeyhas joined
strypeyhas left
stpeterhas joined
stpeterhas left
adiaholic_has left
adiaholic_has joined
werdanhas joined
werdanhas left
ZashLet's invent time travel and send that idea back to when '60 was designed
adiaholic_has left
adiaholic_has joined
ZashWasn't PubSub MAM a thing we discussed at Summit before?
sonnyhas left
sonnyhas joined
Yagizahas joined
mukt2has joined
KevPubSub MAM is a thing, we deploy it and it's in the XEP, no?
stpeterhas joined
stpeterhas left
KevUnless I'm going mad.
flowI am probably not very knowledgeable on pubsub, but what exactly is the benefit of pubsub mam?
KevYou can get the history of items on a node.
flowgiven that it appears that there is pubsub+rsm already
KevXep60 gives you the 'current state', MAM gives you the history.
flowthe history of items means that I can access previous revisions of an item with id 'A'?
KevAs an analogy, xep60 gives you the filesystem, 313 gives you the backups.
KevYes, it would show you the history of things published to the node.
flowi'd like to point out that I always found that motivation for PubSub+MAM missing in xep313
flowmam introduction currently states "or to view old items in a pubsub node."
Yagizahas left
Yagizahas joined
flowand I always wondered "can't I read old items already with plain xep60?"
Yagizahas left
Yagizahas joined
intosihas left
flowand i guess xep-pubsub-mam does not exist (yet)?
KevI thought I wrote pubsub into 313.
Viktor Lindberghas left
flowyou did, but it was removed early this year
KevOh.
KevThat seems...not good.
jonas’it was removed a week or two back
jonas’it’s a separate XEP (not yet)
jonas’because I forgot to do things editor-wise
flowVersion 0.7.0 (2020-03-20)
jonas’I was reminded by reading that paragraph
jonas’0.7.0 was published a few weeks back
jonas’https://github.com/xsf/xeps/pull/921 this one I need to process, will soon
flowIIRC the mam pubsub part was not that substantial, e.g. it still wasn't clear to me why and when I would want to use it, and what it can do for me (but maybe that's just me)
flowsimply stating that you can access previous item revisions with it may be a good first step
Viktor Lindberghas joined
flowbut eventually I think there is much more to it, like the retention policy of this "historical" information
intosihas joined
KevIsn't that exactly the same as with IM MAM?
ZashSo it'd let you replay notifications, or somesuch?
KevThat's one way of looking at it I guess.
KevBut if you're in an environment where pubsub is containing data that's a little bit like a message, rather than just transient notifications, the argument for MAM is the same as it is for IM, I think.
Keve.g. if you're doing FDP or something.
ZashNotifications including retractions and other stuff
lovetoxFound a 8 year old Issue from MattJ in Gajim, where he proposes to use stream management acks instead of pings to detect a timeout
lovetoxwhats the opinion about this today?
flowlovetox, timeout or liveness detection?
lovetoxthe later
jonas’lovetox, "yes please"
Ge0rGlovetox: if you send multiple <r/> without data in between, you can't decide which of them a resulting <a/> is for
jonas’Ge0rG, doesn’t matter
jonas’that you received an <a/> is enough to know that you’re still live
Ge0rGjonas’: doesn't matter for liveness detection, but does matter for server latency measurement
jonas’that is true, but you can do that passively
jonas’(if you really wanted to)
lovetoxGe0rG, why would i send multiple </r> though?
flowsm <r/> <a/> has fewer bytes on the wire compared to xmpp' pinging the server
lovetoxlike without waiting for the <a>
flowbut IMHO not worth it, still a nice microoptimization
Ge0rGI think I'm using actual pings because I encountered some limitation in the Smack API...
jonas’and fewer corner cases ("what if the server does not support ping?")
Ge0rGlovetox: e.g. when doing it from a periodic timer
jonas’Ge0rG, also, you can decide because of in-order processing :)
lovetoxyes Ge0rG but i would always wait for the <a> or? before issuing a new one?
jonas’lovetox, waiting for "the" <a/> is tricky
Ge0rGjonas’: but what if they get reordered or one is lost?!
jonas’Ge0rG, tcp?!
eevvoorhas left
jonas’the main problem tho for corellation is:
> Either party MAY send an <a/> element at any time (e.g., after it has received a certain number of stanzas, or after a certain period of time), even if it has not received an <r/> element from the other party.
Ge0rGjonas’: remember that fortigate TLS 1.3 issue I talked about recently?
jonas’please no
lovetoxand second question is, do you think its a good idea to send <r> after a period of time rather than after a number of stanzas?
flowlovetox, I see no reason why you would request SM acks based on a period of time
jonas’lovetox, so I’d recommedn to send <r/> when you haven’t received anything from the server for more than N seconds
jonas’that you send <r/> specifically doesn’t matter, but sending *anything* which provokes a response from the server is good
jonas’<r/> lends itself because it’s cheap and has a beneficial side effect (updating the SM sync)
flowyou request SM acks to drain your, potentially existing, unack'ed stanza queue
Ge0rGlovetox: it makes sense to send an <r/> after you sent out all the important stanzas that were queued, and then as jonas’ said after a longer period of idletime
jonas’then when you don’t receive *anything* (not specifically <a/>, anything from the server is fine) after another timeout, you consider the stream dead
lovetox2 seperate issues
jonas’this is much simpler than keeping track of pings and it’s also much better, because it can better deal with saturated high-latency links
lovetoxlets move away from alive detection
jonas’but you were talking about liveness detection?
lovetoxcurrently i request r after each stanza
jonas’that’s certainly a valid strategy
lovetoxyes jonas’ and now i want to talk about the second issue
jonas’but unrelated to liveness detection
jonas’oh
jonas’but I didn’t answer your first question yet :P
jonas’but I hadn’t answered your first question yet :P
lovetox:D, you said its a good idea
jonas’lovetox, I’d say for non-aliveness-detection purposes, do <r/> on each socket flush
lovetoxGe0rG, has some server latency issue i dont undestand and im not sure i want to care
Ge0rGI have a server latency issue?
lovetoxjonas’, as i dont fliush the socket myself this is not a option
Ge0rG-ENOPARSE
lovetoxi can ask for r after i get the callback that my write was successful
lovetoxwhich is probably the same
Ge0rGif you don't flush the socket, maybe Nagle will
lovetoxbut im not sure that is really a optimization
Ge0rGlovetox: it's the opposite of an optimization
lovetoxbut i would have to meassure that first
Ge0rGlovetox: ideally, you want the stanza(s) and the <r/> to go out in the same TCP frame
Ge0rGinstead of wasting roundtrips
lovetoxyeah i use a library that abstracts that for me, so i have no idea when and how much is in a tcp frame
Ge0rGI know this is a controversial opinion, but I consider Gajim to be a mobile client when I'm on 2G tethering, riding a train in the middle of German nowhere.
Ge0rGlovetox: there is also TLS in between ;)
flowand compression
Ge0rGso it's borderline impossible to find out.
jonas’lovetox, then maybe wait for 1ms after you wrote a stanza to the socket and send a <r/> after that
Ge0rGjonas’: but why?
flowyeah, jonas’, why?
jonas’lovetox, and reset the timer after each stanza
jonas’flow, Ge0rG, to avoid sending multiple <r/>s for no good reason?
flowI think I lost slighlty what this discussion is about
lovetoxyeah why multiple r?
mukt2has left
lovetoxlook i have a send_stanza method
flowsimply send an <r/> after a few stanzas or when you hit a high watermark of unack'ed stanzas
lovetoxand everytime this is called, i also call send_r
lovetoxits very simple
lovetoxthere can never be multiple r for one stanza
lovetoxits alway stanza, r, stanza, r, stanza, r
jonas’flow, so you send <r/> after N stanzas and the "last" batch of stanzas has N-1. You’re now left with N-1 unacked stanzas "forever".
jonas’for no good reason.
LNJhas left
Viktor Lindberghas left
LNJhas joined
flowjonas’, well if the unacked stanzas are not important then this is fine
lovetoxthats why my idea was, something like, every 3 seconds (of course only if there are unacked stanzas=
jonas’flow, but if they are? what makes a stanza "important"?
flowsmack has predicates that decide when to request an stanza
jonas’lovetox, please don’t send multiple <r/> for any unacked stanza
flowsmack has predicates that decide when to issue an <r/>
jonas’you’ll just pile on to an apparently already existing network congestion
flowtrivial example is "messages with <body/>"
remkohas left
jonas’especially not as often as every 3 seconds.
Ge0rGjonas’: a stanza is important if it is not ephemeral and not going to time out if left in hibernation
lovetoxjonas, of course only one r is in flight
lovetoxplease
jonas’lovetox, it’ll stay in-flight forever though
jonas’if the server doesn’t reply
jonas’or if the server takes long to reply
karoshihas joined
lovetoxbut it MUST reply
jonas’yes, but taht may take some time
lovetoxyeah and?
jonas’e.g. 5 minutes on a crappy mobile link
jonas’in that time you have amassed 100 <r/>s
jonas’to which the server MUST reply
jonas’so not only have you congested your uplink, but also your downlink now
jonas’unneccesarily
lovetoxi propose to send an r every 3 seconds only if there is no r in flight
lovetoxso how would i amass 100 r
jonas’oh, you messed up a negation earlier then
lovetoxmaybe, although i dont see it
jonas’oh, I imagined an "if" in a sentence when there was none
jonas’sorry for that
jonas’lovetox, so, problem is, tracking <r/> and <a/> isn’t really possible
jonas’so how would you define "in-flight"?
Ge0rGevery 3 seconds?!
jonas’Ge0rG, (of course only if there are unacked stanzas)
jonas’Ge0rG, + the in-flight this results in "do <r/> if there are unacked stanzas 3s after the last stanza has been sent" or something like that
Ge0rGthat's an interesting trade-off.
jonas’which is kind of what I proposed earlier, just with a much higher delay, causing less battery efficiency
jonas’and with an unclear definition of "in-flight"
lovetoxone could switch the dealy dynamically
Ge0rGHow long do you wait after generating a stanza and before concluding that there are no more stanzas going to be sent
lovetoxwhen i have unacked stanzas its 3s, afterwards its 20
lovetoxor soemothing
jonas’lovetox, what do you send an <r/> for if you don’t have unacke:?
jonas’lovetox, what do you send an <r/> for if you don’t have unacked stanzas?
lovetoxfor alive detection ^^
stpeterhas joined
stpeterhas left
lovetoxback to issue one
jonas’lovetox, see what I wrote in regard of that one please then
Ge0rGmaybe it's a good thing to just put all the stanzas that you want to send into a queue, then let the OS/event API know you have something to send, and then add a single <r> at the end of the queue
Ge0rG...on delivery
lovetoxgenerally i want to send every data instantly
jonas’Ge0rG, a simpler version of that, requiring less cooperation from the OS, is to re-implement that TCP-cork-ish stuff in userspace: N ms after the last stanza has been put in the tx queue of the socket, add <r/>. reset the timer on each stanza.
lovetoxto put it into a queue then wait some time, maybe more comes
lovetoxseems not very fast
jonas’lovetox, the OS will do that for you anyways. and it’s ok, since the time is generally rather short.
Ge0rGjonas’: but then you are competing with TCP cork and end up sending the <r> in a separate TLS frame in a separate TCP frame
Ge0rGso you add something like 20x the overhead to <r/>
jonas’Ge0rG, only if your delay is longer than the tcp delay
jonas’which you can query from the OS
jonas’and if that’s too complex for you (I don’t blame you), then just send <r/> after each stanza
Viktor Lindberghas joined
Ge0rGor just give it up
lovetoxok i keep sending r after each stanza
lovetoxi dont send many stanzas anyway
jonas’Ge0rG, delay on linux is 200ms
jonas’lovetox, don’t implement IBB then :D
lovetoxhehe
lovetoxyeah
lovetoxi already have
Ge0rGdo you request an <r> for each ibb stanza?
jonas’:)
Ge0rGso it was YOU crashing my server? :D
jonas’how do <r/>s cause a crashing server?
jonas’do I want to know?
Ge0rGjonas’: j/k
jonas’you never know
jonas’you find the weirdest bugs
Ge0rGthank!
Ge0rGthanks!
jonas’lovetox, https://docs.zombofant.net/aioxmpp/devel/api/internal/protocol.html#aioxmpp.protocol.XMLStream.deadtime_soft_limit here’s how liveness detection it works in aioxmpp.
The docs at that place are written from a very internal perspective, hence user == the aioxmpp client class in this case.
-> when the soft timeout expires, the stream sends <r/>. when the hard timeout expires, the stream is considered dead. timeouts reset when (any) data is received (it doesn’t even have to be a stanza).
remkohas joined
jonas’the frontend to that is https://docs.zombofant.net/aioxmpp/devel/api/public/stream.html#timeouts-stream-aliveness-checks
jonas’("frontend")
lovetoxthanks i look into it
jonas’if anything is unclear, feel free to ask. if anything seems stupid, please let me know, because currently I think this is the best anyone can do given networking realities.
jonas’(and what I particularly love about it is the simplicity, because it only uses two timers. no tracking of stanzas or whatnot)
jonas’(and the building blocks can easily be reused for MUC aliveness checking and stuff like that)
jonas’(because the concepts are really the same everywhere)
jonas’oh and this is also an amazing side-effect: "If the peer is pinging us in an interval which works for us (i.e. is less than the soft limit), we don’t need to ping the peer; no extra logic required."
thorstenhas left
LNJhas left
lovetoxyeah less timers the better
lovetoxi hate timers
Viktor Lindberghas joined
LNJhas joined
jonas’two timers, no tracking.
karoshihas left
thorstenhas joined
karoshihas joined
stpeterhas joined
stpeterhas left
mukt2has joined
sonnyhas left
sonnyhas joined
krauqhas left
krauqhas joined
mukt2has left
lovetoxhas left
mukt2has joined
lovetoxhas joined
mukt2has left
mukt2has joined
lovetoxhas left
lovetoxhas joined
stpeterhas joined
stpeterhas left
karoshihas left
eevvoorhas joined
adiaholic_has left
adiaholic_has joined
mukt2has left
mukt2has joined
debaclehas left
krauqhas left
krauqhas joined
Guushas left
Guushas joined
Guushas left
Guushas joined
stpeterhas joined
stpeterhas left
Guushas left
mukt2has left
Guushas joined
Guushas left
Guushas joined
bearhas joined
mimi89999has left
mimi89999has joined
karoshihas joined
lovetoxhas left
karoshihas left
karoshihas joined
stpeterhas joined
stpeterhas left
mukt2has joined
lovetoxhas joined
lovetoxhas left
lovetoxhas joined
lovetoxhas left
lovetoxhas joined
stpeterhas joined
stpeterhas left
karoshihas left
mukt2has left
mukt2has joined
edhelashas left
edhelashas joined
lovetoxhas left
remkohas left
karoshihas joined
krauqhas left
krauqhas joined
mukt2has left
remkohas joined
amuza@riseup.nethas joined
karoshihas left
adiaholic_has left
adiaholic_has joined
adiaholic_has left
adiaholic_has joined
lovetoxhas joined
Viktor Lindberghas left
krauqhas left
krauqhas joined
karoshihas joined
Viktor Lindberghas joined
lovetoxhas left
mukt2has joined
LNJhas left
LNJhas joined
lovetoxhas joined
Marandahas left
Marandahas joined
Mikaelahas left
adiaholic_has left
adiaholic_has joined
Mikaelahas joined
MAOhas joined
MAOhas left
karoshihas left
karoshihas joined
lovetoxhas left
lovetoxhas joined
Shellhas left
Shellhas joined
Viktor Lindberghas left
Viktor Lindberghas joined
sonnyhas left
sonnyhas joined
Lancehas joined
sonnyhas left
sonnyhas joined
Wojtekhas joined
Shellhas left
Shellhas joined
adiaholic_has left
adiaholic_has joined
mukt2has left
lovetoxhas left
mukt2has joined
neshtaxmpphas left
debaclehas joined
krauqhas left
krauqhas joined
j.rhas left
krauqhas left
krauqhas joined
Viktor Lindberghas left
krauqhas left
krauqhas joined
j.rhas joined
krauqhas left
krauqhas joined
Viktor Lindberghas joined
neshtaxmpphas joined
Danielhas left
Danielhas joined
Danielhas left
Danielhas joined
Danielhas left
Danielhas joined
Dele Olajidehas left
remkohas left
sonnyhas left
sonnyhas joined
sonnyhas left
sonnyhas joined
eevvoorhas left
mukt2has left
mukt2has joined
sonnyhas left
sonnyhas joined
sonnyhas left
sonnyhas joined
eevvoorhas joined
sonnyhas left
sonnyhas joined
Danielhas left
Danielhas joined
Wojtekhas left
remkohas joined
adiaholic_has left
adiaholic_has joined
winfriedhas left
winfriedhas joined
eevvoorhas left
eevvoorhas joined
remkohas left
lovetoxhas joined
Danielhas left
Danielhas joined
lovetoxhas left
lovetoxhas joined
bearhas left
Danielhas left
Danielhas joined
DebXWoodyhas joined
Vaulorhas left
Vaulorhas joined
karoshihas left
Wojtekhas joined
etahas left
etahas joined
calvinhas joined
lovetoxhas left
lovetoxhas joined
karoshihas joined
debaclehas left
serge90has joined
Shellhas left
Shellhas joined
lovetoxhas left
remkohas joined
karoshihas left
sonnyhas left
sonnyhas joined
adiaholic_has left
sonnyhas left
sonnyhas joined
bearhas joined
karoshihas joined
DebXWoodyhas left
eevvoorhas left
archas joined
sonnyhas left
sonnyhas joined
krauqhas left
krauqhas joined
Shellhas left
florettahas left
florettahas joined
DebXWoodyhas joined
karoshihas left
karoshihas joined
lovetoxhas joined
Yagizahas left
archas left
archas joined
Syndacehas left
karoshihas left
karoshihas joined
archas left
archas joined
archas left
archas joined
archas left
archas joined
Syndacehas joined
bearhas left
karoshihas left
karoshihas joined
remkohas left
remkohas joined
karoshihas left
karoshihas joined
karoshihas left
eevvoorhas joined
karoshihas joined
karoshihas left
archas left
archas joined
archas left
archas joined
remkohas left
remkohas joined
papatutuwawahas left
archas left
archas joined
remkohas left
remkohas joined
ralphmhas left
ralphmhas joined
Shellhas joined
papatutuwawahas joined
karoshihas joined
debaclehas joined
DebXWoodyhas left
bearhas joined
mukt2has left
remkohas left
archas left
archas joined
Lancehas left
karoshihas left
mukt2has joined
karoshihas joined
jonas’has left
jonas’has joined
mukt2has left
mukt2has joined
Lancehas joined
Zashhas left
Zashhas joined
Lancehas left
karoshihas left
Zashhas left
mukt2has left
mukt2has joined
sonnyhas left
sonnyhas joined
karoshihas joined
eevvoorhas left
Nekithas left
Nekithas joined
Zashhas joined
Tobiashas left
karoshihas left
debaclehas left
mukt2has left
edhelashas left
edhelashas joined
edhelashas left
edhelashas joined
lovetoxhas left
karoshihas joined
karoshihas left
mukt2has joined
LNJhas left
winfriedhas left
winfriedhas joined
Wojteknot sure what's the best place to ask (maybe jdev?) but here goes - is there somewhere a description of the current setup of xmpp.net? or it's sources? "about" (https://xmpp.net/about.php) only states that it's based on xmpppoke and it seems that https://github.com/stpeter/xmppdotnet is not exactly the same thing