lovetoxim interested in the size of the tcp read buffer people use
lovetoxcurrently is use 8192 bytes
lovetoxi wonder if this is too high, seems like a lot of stanzas can fit into this
lovetoxwhich means on every read i would have to process potentially a lot of UI updates
Ge0rGthat sounds like the opposite of a problem
homebeachhas left
Matrix Traveler (bot)has left
Matrix Traveler (bot)has joined
homebeachhas joined
lovetoxwhy?
Ge0rGlovetox: you'll probably only hit this limit if you get flooded by some entity, like right after authenticating to the server or when joining a huge MUC
lovetoxi always get flooded from servers
Ge0rGaggregting actions is much better than doing one thing at a time in the event loop
lovetoxthats the default if you join mucs
Laurahas joined
Ge0rGif you have trouble coping with that in the UI, optimize the UI code ;)
Ge0rGmaybe you have some O(n²) algorithm hidden in there?
J Marinarohas left
jonas’what has the tcp read buffer to do with anything?
jonas’isn't that more of a choice whether you process the actions caused by stanzas synchronously or asynchronously?
J Marinarohas joined
lovetoxthats not much of a choice
Yagizаhas left
jonas’if you say so
J Marinarohas left
lovetoxthe buffer defines the max amout you can receive with a read()
lovetoxhence limits the UI operations done afterwards
Kevhas left
Kevhas joined
lovetoxthe situation is, you can pull more faster data from the network than you have capacity to process in the UI✎
lovetoxthe situation is, you can pull faster data from the network than you have capacity to process in the UI ✏
xnamedhas left
Ge0rGlovetox: have you thought about not updating the ui in small increments, and instead perform all changes, then doing a single redraw() when the read buffers are empty?
rom1dephas left
rom1dephas joined
lovetoxsounds insanly complicated
lovetoxlike i receive a presence, and then wait an amount X maybe i receive an updated presence, and then draw only the last?
Ge0rGdepends on your UI framework and on how you interact with it, I suppose
Ge0rGlovetox: well, some programs will just delay the redraw by .2s, others will only issue the redraw after the read buffer is empty
lovetoxim not sure what you mean by empty, i call read(8192)
Ge0rGis it a blocking or a non-blocking call?
lovetoxnon blocking
Ge0rGso it will return 0 if you are done, and you can issue a redraw then
lovetoxbut it will not return 0 for like a minute
lovetoxif you join100 mucs on connect
lovetoxthats what i mean this is essentially a ddos scenario
Ge0rGwell, you could have two threads, one doing the stanza processing, the other doing UI redraws as fast as it can
lovetoxthe server sends me a potential endless stream of data
lovetoxGe0rG, so you build up a unbounded queue of UI draw events, because the one thread pulls data in faster then the other thread can process
xnamedhas joined
lovetoxthats what i say, async does not help here
lovetoxyou need to limit the input
lovetoxto a level what your computer can process
lovetoxnot stuff it into a queue which you process sometime later, maybe
lovetoxi had 2 ideas, one i limit the read size, meaning i need more time to pull in the data, meaning the ui has more time to update
lovetoxthe second was to just limit the priority of when i call read(), meaning if there any UI events outstanding, i simply not call read() again
Ge0rGlovetox: if you don't do redraws every 8KB, you won't do redraws every 4KB or 2KB
lovetoxpulling 4x 2 kb from the network takes more time, then pulling 8kb once
xnamedhas left
Ge0rGsome microseconds, yes
Ge0rGit's already there in your network stack, waiting for you to fetch it
xnamedhas joined
lovetoxyes nevertheless, its an asnyc call, meaning if i call read(), i will not receive anything (even if its already there) in the mainloop iteration where i issued the call
lovetoxmeaning thats a full iteration where other code can do other things
mathieuihas left
lovetoxbut actually i wanted to know what other people use
lovetoxso Ge0rG what do you use?
Ge0rGlovetox: I have a dedicated thread doing data processing
Ge0rGand I send events to the UI to redraw.
lovetoxso when your UI freezes because it needs to process a lot of events, you pull in just more and more data in the background
Ge0rGno, I always process data on a different thread, not blocking the UI
lovetoxi really having trouble understanding how that technically works, so you receive 200 stanzas, and then you issue 200 events to the ui thread, but it can only process say 10 events / second, so where do the other 190 events go? into some kind of UI event queue? and while that queue builds up you just pull more data in the other thread?
Ge0rGI think I'm pooling things like all presence from a room
goffihas left
lovetoxpresence was an example, great if you are pooling them, but my question was about the how that would work if you getting spammed by a server
Ge0rGlovetox: it's complicated ;) my backend service is storing data in an SQLite DB, and the content provider notifies the UI then
lovetox-has left
goffihas joined
Millesimushas left
lovetox-has joined
Millesimushas joined
lovetox-has left
debaclehas joined
nephelehas joined
nephelehas left
nephelehas joined
nephelehas left
nephelehas joined
nephelehas left
nephelehas joined
oxtypedhas joined
abdullahhas left
nephelehas left
lovetoxone problem i often come across is, when i need to export data from a jid to hard disk, and i need to name the file like the jid
lovetoxis there some save jid to allowed chars for harddrive convert thingy
Ge0rGon linux, you can have anything but / and NUL, so bare JIDs go.
Ge0rGon windows, you are in hell
lovetoxreally anything? because full jids also allow stuff like emoji codepoints
Ge0rGlovetox: depends on the fs of course, but on ext4 everything goes.
Ge0rGyou don't need to have valid utf8 or somesuch
abdullahhas joined
lovetoxhm so i replace / with "-" or something, and look when that breaks
qy> lovetox wrote:
> presence was an example, great if you are pooling them, but my question was about the how that would work if you getting spammed by a server
How frequent or realistic a scenario is it that you'd be spammed by a server so fast that the UI could not keep up? (admittedly in weechat I use the same model, but only begrudgingly)
oxtypedhas left
lovetoxpretty high, because in xmpp you can pretty easily amplify the amount of data you get, i can send 100 join presences, resulting in 100.000 presence stanzas
lovetoxthis all depends of course on how fast your server is, how fast your connection is, how fast your computer is
qyBut surely by that logic, when i applied MR677, my gajim should have become unusable and constantly be playing catchup?
lovetoxNot sure about what you are talking right now, but that is not the Gajim chat and i was not talking about a issue you had
qyHuh
atomicwatchhas joined
oxtypedhas joined
qyStill relevant to me, cause i am technically halfway through rewriting my client to use an unbounded queue instead of direct processing, and best i can tell, even with all my mucs open, there's no real risk of not being able to keep up except at initial connect, so unless the UI redraw takes a very long time, i don't quite follow where the risk is, since the alternative is just freezing anyway
pasdesushihas left
lovetoxthat just does not sound like a sound architecture
lovetoxintroducing an unbounded queue
lovetoxlike instead of ui freezing (one problem) you probably have now much more propblems✎
lovetoxlike instead of ui freezing (one problem) you probably have now much more problems ✏
lovetoxdata loss, memory management problems etc
Ge0rGlovetox: what about joining fewer MUCs in parallel? ;)
lovetoxalso a solution, it falls into the category "reducing input"
lovetoxin my expierience its very easy to write inefficient UI code
jonas’lovetox, re JIDs on the FS: the problem is not what's allowed, the problem is that a JID may be much longer than what a filename may be on ext4 (255 bytes) or even a full pathname may be on linux (it depends™)✎
jonas’lovetox, re JIDs on the FS: on linux, the problem is not what's allowed, the problem is that a JID may be much longer than what a filename may be on ext4 (255 bytes) or even a full pathname may be on linux (it depends™) ✏
lovetoxjonas’, i know there are multiple problems all not pretty, i just have no better idea, if i offer the user a export off all conversations, and want to put them in plain text, then one file per conversation makes sense
jonas’only way to stay sane is sha256 | base32 on the JID if you need to truly key it on the jid and hope that that's hard enough to collide.
lovetoxis there a better solution to this?
lovetoxjonas’, but its plaintext export, people should see the filename and know about what it is
lovetoxnot some sha256 string
jonas’alternatively you could truncate the individual JID parts (thankfully, DNS also only allows 255 chars), so you could e.g. localpart[:128] + '@' + domainpart[:100] + '/' + resourcepart[:10] + sha256sum(jid).to_hex()[:5] or so
jonas’or you might separate domainpart, localpart, resourcepart into a directory structure, which may also be much more navigatable anyway
larmahas joined
lovetoxhm i like that idea on first look
pasdesushihas joined
jonas’(that still requires truncating on long local- and resourceparts tho)
lovetoxim fine with a 99% solution here
nephelehas joined
lovetoxthanks, i think i will try that with the folders
mathieuihas joined
oxtypedhas left
nephelehas left
mathieuihas left
mathieuihas joined
sonnyhas joined
xeckshas left
moparisthebesthas joined
xeckshas joined
Alexhas left
Alexhas joined
sonnyhas left
lovetoxuh resource can also contain /
cedarhas left
cedarhas joined
dezanthas left
lovetox-has joined
debaclehas left
lovetox-has left
xeckshas left
dezanthas joined
rom1dephas left
raghavgururajanhas left
raghavgururajanhas joined
abdullahhas left
xeckshas joined
abdullahhas joined
marmistrzhas left
atomicwatchhas left
xeckshas left
atomicwatchhas joined
xeckshas joined
sonnyhas joined
rom1dephas joined
misterhas joined
dezanthas left
pulkomandyhello, do you have any hints, comments, things to avoid, etc about nickname tab completion? for example:
- any interesting algorithm to detect what substring of the input to try to complete? normally I would split the string on spaces, but xmpp nicknames can have spaces in them
- how do you select which completion to use first when there are multiple matches? depending on who spoke last in the chat? depending on which tab completion was used previously for the same input? any other things to take into account?
- really any feedback on how you did it and what makes oyu happy or unhappy about your current implementation
Ge0rGpulkomandy: splitting on whitespace is generally fine, you'd only end up in conflict if you have multiple users with the same first word
Ge0rGpulkomandy: it makes sense to order tab-completion by last-spoke _and_ last-mentioned
Ge0rGpoezio only does last-spoke, and I'm frustrated every time when messaging the same person multiple times back-to-back
Ge0rGnot sure if last-mentioned-by-you or -by-anybody though
pep.Someone(tm) should push for a wire format to be used for this.
pep.Converse is already using 372, dunno if that's the one, but something is badly needed
Ge0rGpep.: to attach an occupant reference to a message?
Ge0rGthat's quite orthogonal
pep.I think that's pretty much the deal. To me it's the same issue as 0071/0393. Randomly matching stuff in body
pep.And not allowing the sender to express intentions
pulkomandyyes, 372 can do that, I will add it to the message when tab completion is used to complete a nickname
moparisthebestLast spoke is super annoying, if you've ever spent significant time in IRC you'll notice people constantly addressing the wrong person because that person spoke right before tab+enter
Ge0rGmoparisthebest: only if you have nothing to say
Ge0rGI always do tab, then type a message, then press enter
rom1dephas left
pulkomandyalso, do you use any fancy data structure / algorithms for this (possibly a trie or something like that), or do you consider that the number of people in a room is low enough that it isn't worth it, and just compare strings one by one to find all the matches?
pulkomandyalso, case sensitive or not?
Ge0rGcase insensitive. good luck getting that right with unicode and each user's locales.
nephelehas joined
pulkomandywell I don't care about other user's locale, I can just convert everything to lowercase for comparisons locally
nephelehas left
nephelehas joined
pulkomandyah yes and there's also the case where someone has a nickname starting with [ or ` or some other silly char and I may want to ignore it for completion (or maybe I don't)
oxtypedhas joined
pep.In poezio there's a user named ☭ sometimes. Looks like they've changed to another symbol today :)✎
pep.In poezio@ there's a user named ☭ sometimes. Looks like they've changed to another symbol today :) ✏
nepheleYou don't need tab completion if you have only one char in your name :)
nepheleThe only real vector I've had problems with is people namibg themselves /command something in group chats, depending on the input that can be an attack vector
pep.Depends, let's say symbols are not often mapped on keyboards, and not everybody is into clicking
rom1dephas joined
pep.And being able to match that the nick has been mentioned is also necessary
marchas left
nepheleIt's not a completion if you have nothing to base it on, do you do tab conpletions with an empty buffer?
pulkomandyprobably won't have a special case to tab-complete cccp to ☭ , at least at first :p (I think that's how compose keys on Linux allow to use this character?)
pep.nephele, sure
pep.As Ge0rG said above, last-spoke and/or last-mentioned
pulkomandyyes, empty buffer should allow to cycle through all nicknames. Preferrably the ones that are not reachable on the local keyboard first?
pulkomandyand yes, that too
nepheleCompose key on linux (on X11) is based on a big list somewhere deep in X11 configs, I rewrote that config for myself once to make bindings that make sense
oxtypedhas left
lovetox-has joined
moparisthebestSomeone in another MUC changed their name to start with something that looked like an A but was not, so only people using conversations could address them, no one using tab-complete clients could