jdev - 2020-02-09

  1. goffi has left

  2. sonny has joined

  3. bhaveshsgupta has left

  4. bhaveshsgupta has joined

  5. strar has left

  6. strar has joined

  7. Marc has left

  8. bhaveshsgupta has left

  9. kikuchiyo has left

  10. kikuchiyo has joined

  11. bhaveshsgupta has joined

  12. kikuchiyo has left

  13. kikuchiyo has joined

  14. Extarv has left

  15. Extarv has joined

  16. kikuchiyo has left

  17. debacle has left

  18. kikuchiyo has joined

  19. sonny has left

  20. sonny has joined

  21. moparisthebest has left

  22. kikuchiyo has left

  23. kikuchiyo has joined

  24. moparisthebest has joined

  25. kikuchiyo has left

  26. kikuchiyo has joined

  27. bhaveshsgupta has left

  28. bhaveshsgupta has joined

  29. kikuchiyo has left

  30. bhaveshsgupta has left

  31. bhaveshsgupta has joined

  32. bhaveshsgupta has left

  33. bhaveshsgupta has joined

  34. bhaveshsgupta has left

  35. bhaveshsgupta has joined

  36. bhaveshsgupta has left

  37. lovetox has joined

  38. bhaveshsgupta has joined

  39. lovetox has left

  40. bhaveshsgupta has left

  41. bhaveshsgupta has joined

  42. bhaveshsgupta has left

  43. bhaveshsgupta has joined

  44. bhaveshsgupta has left

  45. bhaveshsgupta has joined

  46. lovetox has joined

  47. Smokey99 has joined

  48. paul has joined

  49. Smokey99 has left

  50. bhaveshsgupta has left

  51. bhaveshsgupta has joined

  52. bhaveshsgupta has left

  53. asterix has joined

  54. bhaveshsgupta has joined

  55. lovetox

    hm, if i lose internet connection, say disable wlan, my socket does not really care, i can still send endless data on it

  56. lovetox

    i thought that he would throw some error, but nothing

  57. lovetox

    now i read a bit about it and seems this works as intended

  58. lovetox

    so the only way to know if i lost connection is application layer pings

  59. lovetox

    can someone confirm this

  60. lovetox


  61. lovetox

    and there is some other functionality SO_KEEPALIVE, where the system tests if a socket is still connected

  62. lovetox

    but according to the internet the timeout is 2 hours on linux

  63. lovetox

    so not really useable as a client

  64. bhaveshsgupta has left

  65. bhaveshsgupta has joined

  66. bhaveshsgupta has left

  67. MattJ

    Yes, this is all correct

  68. MattJ

    There are platform-specific ways to reduce the timeout

  69. lovetox

    damn, brilliant design

  70. defanor

    Well, the send buffer is finite. And according to tcp(7) (assuming it's about TCP), keep-alive probes would lead to timing out in about 11 minutes on Linux by default, while retransmission attempts would last 13 to 30 minutes. And then there's TCP_USER_TIMEOUT, and I think I've observed it timing out in just a few minutes (not sure why).

  71. kikuchiyo has joined

  72. goffi has joined

  73. arashi has left

  74. lovetox

    yeah i was looking for something in the range of seconds

  75. bhaveshsgupta has joined

  76. kikuchiyo has left

  77. bhaveshsgupta has left

  78. goffi has left

  79. bhaveshsgupta has joined

  80. bhaveshsgupta has left

  81. bhaveshsgupta has joined

  82. pulkomandy has left

  83. pulkomandy has joined

  84. Extarv has left

  85. Extarv has joined

  86. jonas’

    lovetox, anything in the range of seconds would make things fail on EDGE or GPRS type connections quickly

  87. Extarv has left

  88. Extarv has joined

  89. pulkomandy has left

  90. pulkomandy has joined

  91. goffi has joined

  92. arashi has joined

  93. pulkomandy

    Or even wifi if you're a bit far from the router

  94. MattJ

    And consume battery/bandwidth

  95. pulkomandy

    Or ethernet if you just unplug the cable for a few seconds to untangle it :]

  96. bhaveshsgupta has left

  97. bhaveshsgupta has joined

  98. jonas’

    in the case of EDGE/GPRS, you can easily get into a fun loop with packets from your closed connection cluttering the bandwidth, making your new connection choke right away.

  99. bhaveshsgupta has left

  100. bhaveshsgupta has joined

  101. lovetox has left

  102. lovetox has joined

  103. lovetox

    hm, so what timeout would you suggest?

  104. lovetox

    i think its useful for the client to know if the connection broke down

  105. jonas’

    timeouts are always wrong

  106. debacle has joined

  107. lovetox

    problem is if the connection broke down, and client doesnt notice it, stuff stops working

  108. bhaveshsgupta has left

  109. jonas’


  110. jonas’

    I suggest to have two timeouts

  111. lovetox

    for sending a message i can just show some icon that indicates the message didnt yet reach the server

  112. jonas’

    one I call the `soft_timeout`, which is the maximum time you spend without receiving any data from the server

  113. jonas’

    (be it whitespace or a stanza)

  114. jonas’

    if the soft timeout elapses, you send a ping

  115. jonas’

    and you wait for at most `round_trip_time` (the second timeout) to receive something (doesn’t have to be a reply to your ping, anything will do)

  116. jonas’

    the good thing about this is that with high-delay links, you’ll still most of the time receive things and thus know that your stream is alive

  117. lovetox

    yeah i implemented this just now

  118. jonas’

    you also save on sending pings when you get stanzas from the server (= saved energy)

  119. lovetox

    but what is the max roundtrip tie

  120. lovetox

    but what is the max roundtrip time

  121. lovetox

    is 10 seconds good?

  122. jonas’

    in case of gajim, I’d probably set something like 5 minutes for the soft timeout, and auto-calibrate the round-trip time based on IQ round trips

  123. lovetox

    60 seconds?

  124. jonas’

    something between 60s and 120s is probably ok

  125. lovetox

    5 minutes is way to high, as i said stuff stops working, if user opens his profile it just shows a endless loading screen, if he wants to join a MUC, shows a loading screen, etc etc

  126. jonas’

    lovetox, you can make it more interactive on the application layet

  127. lovetox

    and at this time i want to give the user an indication, hey your connection may be broke down

  128. jonas’

    e.g. if after 15s you have no reply, you could do othre things

  129. jonas’

    such as pinging google or so

  130. lovetox

    but if i have a 5 minute soft timeout, i cant give the user any indication

  131. jonas’

    and if you can’t ping google, suggest that the internte may be down

  132. jonas’

    oh you can

  133. bhaveshsgupta has joined

  134. jonas’

    have an even softer 15s timeout on your requests (IQs, MUC joins etc.) which triggers some additional action to make the UX smooth

  135. jonas’

    "This is taking longer than expected. Is your internet working?"

  136. jonas’

    and also listen for OS events about connectivity changes

  137. lovetox

    so 15 seconds :)

  138. jonas’

    if you get a "reconnected to network X" event from the OS, discard your stream and resume using '198

  139. lovetox

    thats the answer i wanted

  140. jonas’

    but 15 seconds isn’t the time after which you should throw away the stream

  141. jonas’

    that’s just when you start talking to your user

  142. jonas’

    that’s also not the time after which you start sending pings

  143. jonas’

    because you just sent a "ping"

  144. jonas’

    (which is your request)

  145. lovetox

    yeah jonas’ i get it :)

  146. jonas’

    uh, although, what could be fun is trying to establish a new stream

  147. lovetox

    i was always talking about the time i wait for the poing

  148. jonas’

    in the background

  149. jonas’

    and if you notice that this works very quickly, but your request is slow, you send a ping to the server. if you don’t get a reply within the time it took to get the new stream past TLS, you know that your stream is borked and you should '198 resume on the new stream and discard the old one

  150. jonas’

    that’s neat actually

  151. jonas’

    establishing a new TCP connection and doing TLS is not that expensive in number of packets and as good as a test as anything else

  152. jonas’

    (it’ll also not clutter your own send buffers further, you can simply discard the new connection (along with its buffers) easily if it doesn’t work out)

  153. lovetox

    jonas’, sorry sounds way to complicated and over-engineered

  154. lovetox

    i wait X seconds, and throw the stream away, much easier

  155. lovetox

    thats what we have stream management for, for fast reconnects

  156. jonas’

    lovetox, that’ll really hurt you

  157. jonas’

    on a bad mobile link

  158. lovetox

    on bad mobile link, stuff will be bad, no suprise for the user :)

  159. Extarv has left

  160. lovetox

    but the whole topic is a nightmare, we have many actions on xmpp that need a response

  161. lovetox

    so many action will look to the user like, he clicks something but nothing happens

  162. Zash

    Show a spinner if it takes more than a few seconds

  163. Extarv has joined

  164. lovetox

    for some things thats a fine approach

  165. lovetox

    for others its not so nice

  166. lovetox

    deleting a contact from your rosterfor example

  167. lovetox

    dont want to show a dialog with a spinner for that

  168. Zash

    Show it on the contact then. Like, replace the avatar or status icon or something

  169. lovetox

    you really have to design the whole application for this case

  170. lovetox

    what i think i will be doing, let the user send message when we lost connection

  171. lovetox

    but disable every other action

  172. Extarv has left

  173. Extarv has joined

  174. pulkomandy has left

  175. pulkomandy has joined

  176. bhaveshsgupta has left

  177. bhaveshsgupta has joined

  178. Extarv has left

  179. Extarv has joined

  180. pulkomandy has left

  181. Marc has joined

  182. bhaveshsgupta has left

  183. bhaveshsgupta has joined

  184. Extarv has left

  185. Extarv has joined

  186. lovetox has left

  187. pulkomandy has joined

  188. flow

    > lovetox> damn, brilliant design when is a connection "lost"? I don't think there is a better design without loosing some other (desirable) properties

  189. Extarv has left

  190. Extarv has joined

  191. Zash

    That's what you get when you emulate a circuit over a packet-switched network. All sorts of fun!

  192. kikuchiyo has joined

  193. Extarv has left

  194. Extarv has joined

  195. kikuchiyo has left

  196. bhaveshsgupta has left

  197. bhaveshsgupta has joined

  198. bhaveshsgupta has left

  199. Extarv has left

  200. jonas’

    lovetox, or just pretend that it has happened for things where you’re rather certain that you can enforce it to happen eventually

  201. jonas’

    for example, contact deletion

  202. jonas’

    queue the operation up and do it when you have connectivity and hide the element from the UI

  203. bhaveshsgupta has joined

  204. Extarv has joined

  205. serge90 has left

  206. Alex has left

  207. serge90 has joined

  208. bhaveshsgupta has left

  209. bhaveshsgupta has joined

  210. Extarv has left

  211. pep.

    And then users get confused when they use another client that has connectivity :/

  212. Extarv has joined

  213. pep.

    I see the appeal in what you say but I'm not entirely sure I like it

  214. Extarv has left

  215. Extarv has joined

  216. larma has left

  217. goffi has left

  218. larma has joined

  219. pulkomandy has left

  220. pulkomandy has joined

  221. pulkomandy has left

  222. bhaveshsgupta has left

  223. Alex has joined

  224. pulkomandy has joined

  225. bhaveshsgupta has joined

  226. bhaveshsgupta has left

  227. bhaveshsgupta has joined

  228. bhaveshsgupta has left

  229. bhaveshsgupta has joined

  230. bhaveshsgupta has left

  231. sonny has left

  232. sonny has joined

  233. bhaveshsgupta has joined

  234. bhaveshsgupta has left

  235. bhaveshsgupta has joined

  236. bhaveshsgupta has left

  237. bhaveshsgupta has joined

  238. bhaveshsgupta has left

  239. bhaveshsgupta has joined

  240. debacle has left

  241. pulkomandy has left

  242. pulkomandy has joined

  243. kikuchiyo has joined

  244. lovetox has joined

  245. bhaveshsgupta has left

  246. pulkomandy has left

  247. pulkomandy has joined

  248. wurstsalat has left

  249. wurstsalat has joined

  250. goffi has joined

  251. debacle has joined

  252. sonny has left

  253. kikuchiyo has left

  254. sonny has joined

  255. pulkomandy has left

  256. pulkomandy has joined

  257. sonny has left

  258. Extarv has left

  259. Extarv has joined

  260. sonny has joined

  261. kikuchiyo has joined

  262. debacle has left

  263. asterix has left

  264. asterix has joined

  265. bhaveshsgupta has joined

  266. pulkomandy has left

  267. pulkomandy has joined

  268. bhaveshsgupta has left

  269. debacle has joined

  270. lovetox

    jonas’, sounds easy, but reality is much harder

  271. lovetox

    you can invest thousands of lines of code into this, and it still will be buggy

  272. lovetox

    or you just tell the user, hey right now you cant delete, wait until you have a proper connection to the server

  273. wurstsalat has left

  274. wurstsalat has joined

  275. kikuchiyo has left

  276. lovetox

    which is totally fine in my book, i dont think anyone expects to do server operations while offline

  277. bhaveshsgupta has joined

  278. pulkomandy has left

  279. pulkomandy has joined

  280. moparisthebest

    lovetox: you can change the SO_KEEPALIVE interval on Linux, it's settable if you are root

  281. bhaveshsgupta has left

  282. pulkomandy has left

  283. pep.

    yay gajim as root :P

  284. Zash

    Just run it in a container.

  285. pep.

    yay gajim as root in a container!

  286. jonas’

    a.k.a. flatpak?

  287. jonas’

    or was it snap?

  288. bhaveshsgupta has joined

  289. Zash

    why not both?

  290. pep.

    Is the process really root? Isn't bubblewrap used just to spawn the thing?

  291. jonas’

    pep., I was making fun of those

  292. jonas’

    probably not root ;)

  293. moparisthebest

    Well no gajim wouldn't change it, the machine administrator would

  294. pulkomandy has joined

  295. moparisthebest

    With sysctl

  296. Zash

    Isn't there a per-process/-socket option you can poke if your socket library doesn't hide it from you?

  297. moparisthebest

    Not for keepalive iirc

  298. moparisthebest

    Don't worry TCP's replacement is all in userspace not kernel so you can do whatever

  299. Zash


  300. Zash

    But I was excited about TLS in kernel space

  301. pulkomandy has left

  302. pulkomandy has joined

  303. kikuchiyo has joined

  304. wurstsalat has left

  305. wurstsalat has joined

  306. wurstsalat has left

  307. wurstsalat has joined

  308. pulkomandy has left

  309. pulkomandy has joined

  310. Marc has left

  311. Marc has joined

  312. bhaveshsgupta has left

  313. bhaveshsgupta has joined

  314. pulkomandy has left

  315. pulkomandy has joined

  316. bhaveshsgupta has left

  317. bhaveshsgupta has joined

  318. kikuchiyo has left

  319. bhaveshsgupta has left

  320. bhaveshsgupta has joined

  321. bhaveshsgupta has left

  322. bhaveshsgupta has joined

  323. pulkomandy has left

  324. pulkomandy has joined

  325. bhaveshsgupta has left

  326. bhaveshsgupta has joined

  327. Link Mauve

    I still want to play with that.

  328. Link Mauve

    I should have a look in rustls.

  329. pulkomandy has left

  330. pulkomandy has joined

  331. lovetox

    so if i do the xmpp pings, i guess i dont need the send every X seconds a whitespace

  332. lovetox

    or isthis useful for something else

  333. pulkomandy has left

  334. Link Mauve

    The whitespace is a keepalive.

  335. bhaveshsgupta has left

  336. bhaveshsgupta has joined

  337. pulkomandy has joined

  338. lovetox

    yeah though i just learned a socket does not break down, even if i cut the internet, so keeping it alive is not really the issue here

  339. bhaveshsgupta has left

  340. MattJ


  341. MattJ

    Some (too many) routers like to silently terminate idle connections

  342. asterix has left

  343. lovetox

    ok but sending a xmpp ping on a idle connection every 15 seconds should keep it alive

  344. MattJ


  345. lovetox

    im asking if i need both

  346. MattJ

    15 seconds is kinda frequent :/

  347. MattJ

    You could at least use 198 <r>/<a> for pings, which are lighter

  348. Zash

    How about 14 minutes

  349. Link Mauve

    lovetox, that sounds like a battery drainer.

  350. Link Mauve

    Pidgin does that AFAIK, and it’s terrible.

  351. Link Mauve

    Don’t be like Pidgin.

  352. lovetox

    hm or i depend on a network manager, to tell me if the internet is down

  353. bhaveshsgupta has joined

  354. Link Mauve

    Aren’t you already doing that with nm?

  355. Link Mauve

    But jonas’’s approach of using XEP-0198 stream resumption looks quite elegant to me.

  356. Link Mauve

    That’s what I’d do if I were to start something from scratch again.

  357. lovetox

    what approach is that?

  358. Link Mauve

    The one from 10:24:16 CET.

  359. Zash

    Just trying a new TCP connection to the server would at least tell you something about whether it's the existing connection that's broken or all connectivity

  360. Link Mauve

    or all connectivity to the server*

  361. lovetox

    but thats what a network manager is for

  362. lovetox


  363. Zash

    It won't detect if someone trips over a cable in the building switch room

  364. Link Mauve

    Seems like it would require running one though.

  365. Zash

    or the server suddenly explodes

  366. lovetox

    i think we are mixing 2 things here

  367. lovetox

    detecting if my remote is gone

  368. lovetox

    and detecting if my internet is gone

  369. lovetox

    or is this the same thing?

  370. lovetox

    i hope not

  371. Link Mauve

    What is your end goal?

  372. lovetox

    either way, network manager tells me when there is no route to the internet, reading the manual

  373. lovetox

    so this includes tripping over my cable

  374. lovetox

    Link Mauve, detecting when i have no connection to the server

  375. Link Mauve

    lovetox, why do you need to know about the Internet then?

  376. Link Mauve

    And also, what do you want to do in such a case?

  377. bhaveshsgupta has left

  378. marc0s has left

  379. marc0s has joined

  380. Link Mauve

    What about if the kernel eventually flushes its buffers and messages you sent now got sent?

  381. lovetox

    i show the user that there is no connection and limit what he can do

  382. pulkomandy has left

  383. pulkomandy has joined

  384. Link Mauve

    That sounds like bad UX when you can instead remember what the user did and then figure out whether it got done or not on stream resumption.

  385. Link Mauve

    Meh, stupid Freenode sending me tons of presences from IRC…

  386. Zash

    There was just a notice about them doing some maintenence

  387. larma has left

  388. Link Mauve

    poezio is still spending all of my CPU time handling those presences…

  389. lovetox

    so just a simple example

  390. lovetox

    its easy to just store the stanzas that are going out and on failed resumption replay them

  391. lovetox

    but if you think about that the user could close the application while not connected

  392. lovetox

    now you suddenly have to save your not acked stanza cache to harddisk, and on resume you have to judge which of them make sense to send

  393. MattJ

    Yes, it's hard

  394. lovetox

    you have to take into account how much time passed

  395. lovetox

    probably offer UI to ask the user etc etc

  396. kikuchiyo has joined

  397. lovetox

    a lot of magic that can fail, and end in scenarios where the user just cant understand what and why something happend or didnt happen

  398. lovetox

    compare this with a very simple "You are not connected, so you cant do that right now"

  399. bhaveshsgupta has joined

  400. lovetox

    but as it seems, i cant even determine if im connected :D

  401. pep.

    I agree with lovetox that it's a lot of magic that can go wrong, (buffering and resending when connectivity gets back) :/

  402. bhaveshsgupta has left

  403. lovetox

    for messages only i can imagine it, if i would have proper UI where i notify the user of messages that have not been sent last time, and he can choose to resend

  404. lovetox

    for everything else i would say not worth it

  405. larma has joined

  406. wurstsalat

    sending messages while being offline would also satisfy a feature request :)

  407. lovetox

    yeah not a problem, but we need UI first that shows that a message has not reached the server, and after X minutes, dont send it anymore instead user has to manually issue a command to resend them

  408. bhaveshsgupta has joined

  409. wurstsalat

    we could use that info bar at the top of each chat control to inform + offer to try again (and mark unsent messages in the chat)

  410. bhaveshsgupta has left

  411. lovetox

    yeah that sentence is a week worth of work :D

  412. bhaveshsgupta has joined

  413. larma has left

  414. bhaveshsgupta has left

  415. wurstsalat

    sorry, just trying to point out a possible UI^^

  416. wurstsalat

    I always wanted to use that infobar for more :)

  417. larma has joined

  418. larma has left

  419. larma has joined

  420. wurstsalat has left

  421. larma has left

  422. larma has joined

  423. strar has left

  424. goffi has left

  425. strar has joined

  426. larma has left

  427. kikuchiyo has left

  428. gav has joined

  429. larma has joined

  430. paul has left

  431. larma has left

  432. debacle has left