jdev - 2022-03-12


  1. al has left

  2. Millesimus has left

  3. jgart has joined

  4. goffi has left

  5. emus has left

  6. Millesimus has joined

  7. dezant has joined

  8. Millesimus has left

  9. oxtyped has joined

  10. larma has left

  11. oxtyped has left

  12. Millesimus has joined

  13. Millesimus has left

  14. Millesimus has joined

  15. marc0s has left

  16. marc0s has joined

  17. dezant has left

  18. dezant has joined

  19. marc0s has left

  20. marc0s has joined

  21. PapaTutuWawa has joined

  22. debacle has left

  23. pasdesushi has left

  24. xnamed has left

  25. Millesimus has left

  26. PapaTutuWawa has left

  27. Millesimus has joined

  28. Millesimus has left

  29. larma has joined

  30. marc0s has left

  31. marc0s has joined

  32. Millesimus has joined

  33. Millesimus has left

  34. larma has left

  35. marc0s has left

  36. marc0s has joined

  37. Millesimus has joined

  38. Millesimus has left

  39. Millesimus has joined

  40. Millesimus has left

  41. dezant has left

  42. marc0s has left

  43. marc0s has joined

  44. Millesimus has joined

  45. Yagizа has joined

  46. nephele has joined

  47. paul has joined

  48. nephele has left

  49. Millesimus has left

  50. nephele has joined

  51. Millesimus has joined

  52. nephele has left

  53. nephele has joined

  54. nephele has left

  55. nephele has joined

  56. dezant has joined

  57. Millesimus has left

  58. Millesimus has joined

  59. nephele has left

  60. al has joined

  61. Sam has left

  62. Sam has joined

  63. FireFly has left

  64. nephele has joined

  65. nephele has left

  66. nephele has joined

  67. mirux has joined

  68. nephele has left

  69. nephele has joined

  70. Ingolf has joined

  71. FireFly has joined

  72. nephele has left

  73. nephele has joined

  74. nephele has left

  75. nephele has joined

  76. emus has joined

  77. nephele has left

  78. Alex has joined

  79. Alex has left

  80. Alex has joined

  81. Alex has left

  82. Alex has joined

  83. jgart has left

  84. Millesimus has left

  85. nephele has joined

  86. Millesimus has joined

  87. nephele has left

  88. nephele has joined

  89. nephele has left

  90. jgart has joined

  91. nephele has joined

  92. nephele has left

  93. nephele has joined

  94. nephele has left

  95. nephele has joined

  96. FireFly has left

  97. wurstsalat has joined

  98. nephele has left

  99. nephele has joined

  100. nephele has left

  101. nephele has joined

  102. nephele has left

  103. moparisthebest has left

  104. J Marinaro has left

  105. FireFly has joined

  106. nephele has joined

  107. dezant has left

  108. nephele has left

  109. nephele has joined

  110. nephele has left

  111. nephele has joined

  112. dezant has joined

  113. nephele has left

  114. nephele has joined

  115. nephele has left

  116. nephele has joined

  117. jgart has left

  118. FireFly has left

  119. marmistrz has joined

  120. moparisthebest has joined

  121. emus has left

  122. nephele has left

  123. FireFly has joined

  124. wurstsalat has left

  125. wurstsalat has joined

  126. emus has joined

  127. kfv has left

  128. kfv has joined

  129. nephele has joined

  130. nephele has left

  131. nephele has joined

  132. J Marinaro has joined

  133. nephele has left

  134. nephele has joined

  135. al has left

  136. nephele has left

  137. nephele has joined

  138. xnamed has joined

  139. nephele has left

  140. nephele has joined

  141. nephele has left

  142. nephele has joined

  143. nephele has left

  144. nephele has joined

  145. nephele has left

  146. nephele has joined

  147. nephele has left

  148. nephele has joined

  149. Apollo has left

  150. nephele has left

  151. nephele has joined

  152. nephele has left

  153. nephele has joined

  154. goffi has joined

  155. goffi has left

  156. goffi has joined

  157. nephele has left

  158. moparisthebest has left

  159. pasdesushi has joined

  160. msavoritias has joined

  161. nephele has joined

  162. nephele has left

  163. nephele has joined

  164. debacle has joined

  165. Yagizа has left

  166. Yagizа has joined

  167. nephele has left

  168. nephele has joined

  169. xecks has left

  170. nephele has left

  171. nephele has joined

  172. nephele has left

  173. nephele has joined

  174. nephele has left

  175. nephele has joined

  176. nephele has left

  177. nephele has joined

  178. nephele has left

  179. Yagizа has left

  180. Yagizа has joined

  181. Laura has left

  182. nephele has joined

  183. nephele has left

  184. J Marinaro has left

  185. J Marinaro has joined

  186. nephele has joined

  187. nephele has left

  188. kfv has left

  189. kfv has joined

  190. nephele has joined

  191. kfv has left

  192. kfv has joined

  193. nephele has left

  194. nephele has joined

  195. Laura has joined

  196. nephele has left

  197. sonny has left

  198. goffi has left

  199. nephele has joined

  200. nephele has left

  201. nephele has joined

  202. nephele has left

  203. goffi has joined

  204. nephele has joined

  205. Yagizа has left

  206. marc0s has left

  207. marc0s has joined

  208. Yagizа has joined

  209. marc0s has left

  210. marc0s has joined

  211. sonny has joined

  212. marc0s has left

  213. Mjolnir Archon has left

  214. Maranda has left

  215. marc0s has joined

  216. Mjolnir Archon has joined

  217. Maranda has joined

  218. nephele has left

  219. Laura has left

  220. Laura has joined

  221. msavoritias has left

  222. msavoritias has joined

  223. rubi has left

  224. rubi has joined

  225. Yagizа has left

  226. Yagizа has joined

  227. sonny has left

  228. sonny has joined

  229. goffi has left

  230. Kev has joined

  231. Kev has left

  232. goffi has joined

  233. Kev has joined

  234. Kev has left

  235. Kev has joined

  236. Laura has left

  237. sonny has left

  238. debacle has left

  239. xecks has joined

  240. lovetox- has joined

  241. lovetox

    im interested in the size of the tcp read buffer people use

  242. lovetox

    currently is use 8192 bytes

  243. lovetox

    i wonder if this is too high, seems like a lot of stanzas can fit into this

  244. lovetox

    which means on every read i would have to process potentially a lot of UI updates

  245. Ge0rG

    that sounds like the opposite of a problem

  246. homebeach has left

  247. Matrix Traveler (bot) has left

  248. Matrix Traveler (bot) has joined

  249. homebeach has joined

  250. lovetox

    why?

  251. Ge0rG

    lovetox: you'll probably only hit this limit if you get flooded by some entity, like right after authenticating to the server or when joining a huge MUC

  252. lovetox

    i always get flooded from servers

  253. Ge0rG

    aggregting actions is much better than doing one thing at a time in the event loop

  254. lovetox

    thats the default if you join mucs

  255. Laura has joined

  256. Ge0rG

    if you have trouble coping with that in the UI, optimize the UI code ;)

  257. Ge0rG

    maybe you have some O(n²) algorithm hidden in there?

  258. J Marinaro has left

  259. jonas’

    what has the tcp read buffer to do with anything?

  260. jonas’

    isn't that more of a choice whether you process the actions caused by stanzas synchronously or asynchronously?

  261. J Marinaro has joined

  262. lovetox

    thats not much of a choice

  263. Yagizа has left

  264. jonas’

    if you say so

  265. J Marinaro has left

  266. lovetox

    the buffer defines the max amout you can receive with a read()

  267. lovetox

    hence limits the UI operations done afterwards

  268. Kev has left

  269. Kev has joined

  270. lovetox

    the situation is, you can pull more faster data from the network than you have capacity to process in the UI

  271. lovetox

    the situation is, you can pull faster data from the network than you have capacity to process in the UI

  272. xnamed has left

  273. Ge0rG

    lovetox: have you thought about not updating the ui in small increments, and instead perform all changes, then doing a single redraw() when the read buffers are empty?

  274. rom1dep has left

  275. rom1dep has joined

  276. lovetox

    sounds insanly complicated

  277. lovetox

    like i receive a presence, and then wait an amount X maybe i receive an updated presence, and then draw only the last?

  278. Ge0rG

    depends on your UI framework and on how you interact with it, I suppose

  279. Ge0rG

    lovetox: well, some programs will just delay the redraw by .2s, others will only issue the redraw after the read buffer is empty

  280. lovetox

    im not sure what you mean by empty, i call read(8192)

  281. Ge0rG

    is it a blocking or a non-blocking call?

  282. lovetox

    non blocking

  283. Ge0rG

    so it will return 0 if you are done, and you can issue a redraw then

  284. lovetox

    but it will not return 0 for like a minute

  285. lovetox

    if you join100 mucs on connect

  286. lovetox

    thats what i mean this is essentially a ddos scenario

  287. Ge0rG

    well, you could have two threads, one doing the stanza processing, the other doing UI redraws as fast as it can

  288. lovetox

    the server sends me a potential endless stream of data

  289. lovetox

    Ge0rG, so you build up a unbounded queue of UI draw events, because the one thread pulls data in faster then the other thread can process

  290. xnamed has joined

  291. lovetox

    thats what i say, async does not help here

  292. lovetox

    you need to limit the input

  293. lovetox

    to a level what your computer can process

  294. lovetox

    not stuff it into a queue which you process sometime later, maybe

  295. lovetox

    i had 2 ideas, one i limit the read size, meaning i need more time to pull in the data, meaning the ui has more time to update

  296. lovetox

    the second was to just limit the priority of when i call read(), meaning if there any UI events outstanding, i simply not call read() again

  297. Ge0rG

    lovetox: if you don't do redraws every 8KB, you won't do redraws every 4KB or 2KB

  298. lovetox

    pulling 4x 2 kb from the network takes more time, then pulling 8kb once

  299. xnamed has left

  300. Ge0rG

    some microseconds, yes

  301. Ge0rG

    it's already there in your network stack, waiting for you to fetch it

  302. xnamed has joined

  303. lovetox

    yes nevertheless, its an asnyc call, meaning if i call read(), i will not receive anything (even if its already there) in the mainloop iteration where i issued the call

  304. lovetox

    meaning thats a full iteration where other code can do other things

  305. mathieui has left

  306. lovetox

    but actually i wanted to know what other people use

  307. lovetox

    so Ge0rG what do you use?

  308. Ge0rG

    lovetox: I have a dedicated thread doing data processing

  309. Ge0rG

    and I send events to the UI to redraw.

  310. lovetox

    so when your UI freezes because it needs to process a lot of events, you pull in just more and more data in the background

  311. Ge0rG

    no, I always process data on a different thread, not blocking the UI

  312. lovetox

    i really having trouble understanding how that technically works, so you receive 200 stanzas, and then you issue 200 events to the ui thread, but it can only process say 10 events / second, so where do the other 190 events go? into some kind of UI event queue? and while that queue builds up you just pull more data in the other thread?

  313. Ge0rG

    I think I'm pooling things like all presence from a room

  314. goffi has left

  315. lovetox

    presence was an example, great if you are pooling them, but my question was about the how that would work if you getting spammed by a server

  316. Ge0rG

    lovetox: it's complicated ;) my backend service is storing data in an SQLite DB, and the content provider notifies the UI then

  317. lovetox- has left

  318. goffi has joined

  319. Millesimus has left

  320. lovetox- has joined

  321. Millesimus has joined

  322. lovetox- has left

  323. debacle has joined

  324. nephele has joined

  325. nephele has left

  326. nephele has joined

  327. nephele has left

  328. nephele has joined

  329. nephele has left

  330. nephele has joined

  331. oxtyped has joined

  332. abdullah has left

  333. nephele has left

  334. lovetox

    one problem i often come across is, when i need to export data from a jid to hard disk, and i need to name the file like the jid

  335. lovetox

    is there some save jid to allowed chars for harddrive convert thingy

  336. Ge0rG

    on linux, you can have anything but / and NUL, so bare JIDs go.

  337. Ge0rG

    on windows, you are in hell

  338. lovetox

    really anything? because full jids also allow stuff like emoji codepoints

  339. Ge0rG

    lovetox: depends on the fs of course, but on ext4 everything goes.

  340. Ge0rG

    you don't need to have valid utf8 or somesuch

  341. abdullah has joined

  342. lovetox

    hm so i replace / with "-" or something, and look when that breaks

  343. qy

    > lovetox wrote: > presence was an example, great if you are pooling them, but my question was about the how that would work if you getting spammed by a server How frequent or realistic a scenario is it that you'd be spammed by a server so fast that the UI could not keep up? (admittedly in weechat I use the same model, but only begrudgingly)

  344. oxtyped has left

  345. lovetox

    pretty high, because in xmpp you can pretty easily amplify the amount of data you get, i can send 100 join presences, resulting in 100.000 presence stanzas

  346. lovetox

    this all depends of course on how fast your server is, how fast your connection is, how fast your computer is

  347. qy

    But surely by that logic, when i applied MR677, my gajim should have become unusable and constantly be playing catchup?

  348. lovetox

    Not sure about what you are talking right now, but that is not the Gajim chat and i was not talking about a issue you had

  349. qy

    Huh

  350. atomicwatch has joined

  351. oxtyped has joined

  352. qy

    Still relevant to me, cause i am technically halfway through rewriting my client to use an unbounded queue instead of direct processing, and best i can tell, even with all my mucs open, there's no real risk of not being able to keep up except at initial connect, so unless the UI redraw takes a very long time, i don't quite follow where the risk is, since the alternative is just freezing anyway

  353. pasdesushi has left

  354. lovetox

    that just does not sound like a sound architecture

  355. lovetox

    introducing an unbounded queue

  356. lovetox

    like instead of ui freezing (one problem) you probably have now much more propblems

  357. lovetox

    like instead of ui freezing (one problem) you probably have now much more problems

  358. lovetox

    data loss, memory management problems etc

  359. Ge0rG

    lovetox: what about joining fewer MUCs in parallel? ;)

  360. lovetox

    also a solution, it falls into the category "reducing input"

  361. lovetox

    in my expierience its very easy to write inefficient UI code

  362. jonas’

    lovetox, re JIDs on the FS: the problem is not what's allowed, the problem is that a JID may be much longer than what a filename may be on ext4 (255 bytes) or even a full pathname may be on linux (it depends™)

  363. jonas’

    lovetox, re JIDs on the FS: on linux, the problem is not what's allowed, the problem is that a JID may be much longer than what a filename may be on ext4 (255 bytes) or even a full pathname may be on linux (it depends™)

  364. lovetox

    jonas’, i know there are multiple problems all not pretty, i just have no better idea, if i offer the user a export off all conversations, and want to put them in plain text, then one file per conversation makes sense

  365. jonas’

    only way to stay sane is sha256 | base32 on the JID if you need to truly key it on the jid and hope that that's hard enough to collide.

  366. lovetox

    is there a better solution to this?

  367. lovetox

    jonas’, but its plaintext export, people should see the filename and know about what it is

  368. lovetox

    not some sha256 string

  369. jonas’

    alternatively you could truncate the individual JID parts (thankfully, DNS also only allows 255 chars), so you could e.g. localpart[:128] + '@' + domainpart[:100] + '/' + resourcepart[:10] + sha256sum(jid).to_hex()[:5] or so

  370. jonas’

    or you might separate domainpart, localpart, resourcepart into a directory structure, which may also be much more navigatable anyway

  371. larma has joined

  372. lovetox

    hm i like that idea on first look

  373. pasdesushi has joined

  374. jonas’

    (that still requires truncating on long local- and resourceparts tho)

  375. lovetox

    im fine with a 99% solution here

  376. nephele has joined

  377. lovetox

    thanks, i think i will try that with the folders

  378. mathieui has joined

  379. oxtyped has left

  380. nephele has left

  381. mathieui has left

  382. mathieui has joined

  383. sonny has joined

  384. xecks has left

  385. moparisthebest has joined

  386. xecks has joined

  387. Alex has left

  388. Alex has joined

  389. sonny has left

  390. lovetox

    uh resource can also contain /

  391. cedar has left

  392. cedar has joined

  393. dezant has left

  394. lovetox- has joined

  395. debacle has left

  396. lovetox- has left

  397. xecks has left

  398. dezant has joined

  399. rom1dep has left

  400. raghavgururajan has left

  401. raghavgururajan has joined

  402. abdullah has left

  403. xecks has joined

  404. abdullah has joined

  405. marmistrz has left

  406. atomicwatch has left

  407. xecks has left

  408. atomicwatch has joined

  409. xecks has joined

  410. sonny has joined

  411. rom1dep has joined

  412. mister has joined

  413. dezant has left

  414. pulkomandy

    hello, do you have any hints, comments, things to avoid, etc about nickname tab completion? for example: - any interesting algorithm to detect what substring of the input to try to complete? normally I would split the string on spaces, but xmpp nicknames can have spaces in them - how do you select which completion to use first when there are multiple matches? depending on who spoke last in the chat? depending on which tab completion was used previously for the same input? any other things to take into account? - really any feedback on how you did it and what makes oyu happy or unhappy about your current implementation

  415. Ge0rG

    pulkomandy: splitting on whitespace is generally fine, you'd only end up in conflict if you have multiple users with the same first word

  416. Ge0rG

    pulkomandy: it makes sense to order tab-completion by last-spoke _and_ last-mentioned

  417. Ge0rG

    poezio only does last-spoke, and I'm frustrated every time when messaging the same person multiple times back-to-back

  418. Ge0rG

    not sure if last-mentioned-by-you or -by-anybody though

  419. pep.

    Someone(tm) should push for a wire format to be used for this.

  420. pep.

    Converse is already using 372, dunno if that's the one, but something is badly needed

  421. Ge0rG

    pep.: to attach an occupant reference to a message?

  422. Ge0rG

    that's quite orthogonal

  423. pep.

    I think that's pretty much the deal. To me it's the same issue as 0071/0393. Randomly matching stuff in body

  424. pep.

    And not allowing the sender to express intentions

  425. pulkomandy

    yes, 372 can do that, I will add it to the message when tab completion is used to complete a nickname

  426. moparisthebest

    Last spoke is super annoying, if you've ever spent significant time in IRC you'll notice people constantly addressing the wrong person because that person spoke right before tab+enter

  427. Ge0rG

    moparisthebest: only if you have nothing to say

  428. Ge0rG

    I always do tab, then type a message, then press enter

  429. rom1dep has left

  430. pulkomandy

    also, do you use any fancy data structure / algorithms for this (possibly a trie or something like that), or do you consider that the number of people in a room is low enough that it isn't worth it, and just compare strings one by one to find all the matches?

  431. pulkomandy

    also, case sensitive or not?

  432. Ge0rG

    case insensitive. good luck getting that right with unicode and each user's locales.

  433. nephele has joined

  434. pulkomandy

    well I don't care about other user's locale, I can just convert everything to lowercase for comparisons locally

  435. nephele has left

  436. nephele has joined

  437. pulkomandy

    ah yes and there's also the case where someone has a nickname starting with [ or ` or some other silly char and I may want to ignore it for completion (or maybe I don't)

  438. oxtyped has joined

  439. pep.

    In poezio there's a user named ☭ sometimes. Looks like they've changed to another symbol today :)

  440. pep.

    In poezio@ there's a user named ☭ sometimes. Looks like they've changed to another symbol today :)

  441. nephele

    You don't need tab completion if you have only one char in your name :)

  442. nephele

    The only real vector I've had problems with is people namibg themselves /command something in group chats, depending on the input that can be an attack vector

  443. pep.

    Depends, let's say symbols are not often mapped on keyboards, and not everybody is into clicking

  444. rom1dep has joined

  445. pep.

    And being able to match that the nick has been mentioned is also necessary

  446. marc has left

  447. nephele

    It's not a completion if you have nothing to base it on, do you do tab conpletions with an empty buffer?

  448. pulkomandy

    probably won't have a special case to tab-complete cccp to ☭ , at least at first :p (I think that's how compose keys on Linux allow to use this character?)

  449. pep.

    nephele, sure

  450. pep.

    As Ge0rG said above, last-spoke and/or last-mentioned

  451. pulkomandy

    yes, empty buffer should allow to cycle through all nicknames. Preferrably the ones that are not reachable on the local keyboard first?

  452. pulkomandy

    and yes, that too

  453. nephele

    Compose key on linux (on X11) is based on a big list somewhere deep in X11 configs, I rewrote that config for myself once to make bindings that make sense

  454. oxtyped has left

  455. lovetox- has joined

  456. moparisthebest

    Someone in another MUC changed their name to start with something that looked like an A but was not, so only people using conversations could address them, no one using tab-complete clients could

  457. lovetox- has left

  458. nephele has left

  459. nephele has joined

  460. pasdesushi has left

  461. pasdesushi has joined

  462. nephele has left

  463. nephele has joined

  464. nephele has left

  465. nephele has joined

  466. nephele has left

  467. nephele has joined

  468. nephele has left

  469. larma has left

  470. nephele has joined

  471. marc has joined

  472. nephele has left

  473. nephele has joined

  474. nephele has left

  475. nephele has joined

  476. goffi has left

  477. nephele has left

  478. nephele has joined

  479. lovetox

    7

  480. lovetox

    .

  481. nephele has left

  482. nephele has joined

  483. larma has joined

  484. nephele has left

  485. oxtyped has joined

  486. inky has left

  487. oxtyped has left

  488. mathieui

    pep., to be fair that user is usually called "ux" and gets annoyed anytime people mention them

  489. nephele has joined

  490. inky has joined

  491. nephele has left

  492. nephele has joined

  493. nephele has left

  494. nephele has joined

  495. nephele has left

  496. oxtyped has joined

  497. goffi has joined

  498. oxtyped has left

  499. marc has left

  500. nephele has joined

  501. nephele has left

  502. nephele has joined

  503. nephele has left

  504. marc has joined

  505. debacle has joined

  506. emus has left

  507. me9 has joined

  508. nephele has joined

  509. larma has left

  510. FireFly has left

  511. nephele has left

  512. FireFly has joined

  513. FireFly has left

  514. oxtyped has joined

  515. selurvedu has joined

  516. oxtyped has left

  517. lovetox- has joined

  518. nephele has joined

  519. antranigv has left

  520. nephele has left

  521. cedar has left

  522. FireFly has joined

  523. kfv has left

  524. kfv has joined

  525. PapaTutuWawa has joined

  526. FireFly has left

  527. larma has joined

  528. nephele has joined

  529. nephele has left

  530. emus has joined

  531. mirux has left

  532. J Marinaro has joined

  533. atomicwatch has left

  534. oxtyped has joined

  535. mirux has joined

  536. oxtyped has left

  537. jgart has joined

  538. disgyze has joined

  539. disgyze has left

  540. disgyze has joined

  541. disgyze has left

  542. disgyze has joined

  543. disgyze has left

  544. disgyze has joined

  545. disgyze has left

  546. disgyze has joined

  547. disgyze has left

  548. disgyze has joined

  549. disgyze has left

  550. disgyze has joined

  551. disgyze has left

  552. disgyze has joined

  553. disgyze has left

  554. disgyze has joined

  555. mirux has left

  556. emus has left

  557. msavoritias has left

  558. me9 has left

  559. larma has left

  560. FireFly has joined

  561. antranigv has joined

  562. homebeach has left

  563. Matrix Traveler (bot) has left

  564. Matrix Traveler (bot) has joined

  565. homebeach has joined

  566. Vaulor has left

  567. PapaTutuWawa has left

  568. marc0s has left

  569. marc0s has joined

  570. kfv has left

  571. FireFly has left

  572. oxtyped has joined

  573. kfv has joined

  574. oxtyped has left

  575. Vaulor has joined

  576. dezant has joined

  577. lovetox- has left

  578. marc0s has left

  579. marc0s has joined

  580. FireFly has joined

  581. homebeach has left

  582. Matrix Traveler (bot) has left

  583. Matrix Traveler (bot) has joined

  584. homebeach has joined

  585. atomicwatch has joined

  586. emus has joined

  587. oxtyped has joined

  588. oxtyped has left

  589. marc0s has left

  590. marc0s has joined

  591. larma has joined