XSF Discussion - 2019-07-16

  1. wojtek has left

  2. pdurbin has joined

  3. pdurbin has left

  4. igoose has left

  5. igoose has joined

  6. Douglas Terabyte has joined

  7. zach has left

  8. zach has joined

  9. lskdjf has left

  10. igoose has left

  11. igoose has joined

  12. igoose has left

  13. igoose has joined

  14. igoose has left

  15. igoose has joined

  16. david has joined

  17. Lance has left

  18. igoose has left

  19. igoose has joined

  20. Lance has joined

  21. Lance has left

  22. neshtaxmpp has left

  23. neshtaxmpp has joined

  24. igoose has left

  25. igoose has joined

  26. igoose has left

  27. igoose has joined

  28. Daniel has left

  29. remko has joined

  30. Daniel has joined

  31. Lance has joined

  32. Lance has left

  33. alacer has joined

  34. remko has left

  35. igoose has left

  36. igoose has joined

  37. UsL has left

  38. Daniel has left

  39. david has left

  40. david has joined

  41. Daniel has joined

  42. pdurbin has joined

  43. patrick has left

  44. alacer has left

  45. alacer has joined

  46. patrick has joined

  47. david has left

  48. igoose has left

  49. igoose has joined

  50. patrick has left

  51. patrick has joined

  52. david has joined

  53. remko has joined

  54. david has left

  55. Andrew Nenakhov has joined

  56. david has joined

  57. pdurbin has left

  58. pdurbin has joined

  59. patrick has left

  60. remko has left

  61. igoose has left

  62. igoose has joined

  63. Nekit has joined

  64. kokonoe has left

  65. kokonoe has joined

  66. adityaborikar has left

  67. wurstsalat has joined

  68. pdurbin has left

  69. adityaborikar has joined

  70. valo has left

  71. karoshi has joined

  72. valo has joined

  73. valo has left

  74. valo has joined

  75. goffi has joined

  76. waqas has left

  77. waqas has joined

  78. Douglas Terabyte has left

  79. Douglas Terabyte has joined

  80. waqas has left

  81. lnj has joined

  82. valo has left

  83. valo has joined

  84. larma has left

  85. larma has joined

  86. goffi has left

  87. goffi has joined

  88. madhur.garg has joined

  89. madhur.garg has left

  90. sezuan has joined

  91. pdurbin has joined

  92. igoose has left

  93. igoose has joined

  94. debacle has joined

  95. kokonoe has left

  96. pdurbin has left

  97. remko has joined

  98. lnj has left

  99. Mikaela has joined

  100. alacer has left

  101. lskdjf has joined

  102. kokonoe has joined

  103. Kev has joined

  104. Nekit has left

  105. pdurbin has joined

  106. lnj has joined

  107. madhur.garg has joined

  108. alacer has joined

  109. alacer has left

  110. alacer has joined

  111. madhur.garg

    hi, I am implementing Infinite scroll using MAM in Poezio for that, I have to query messages in reverse order. I am sending the query like this: https://xmpp.org/extensions/xep-0313.html#example-9 with the timestamp of the current 1st message in the chat as the end timestamp, I have one doubt with this: What value should I take for start timestamp here? (should it be some fixed timestamp like '2010-08-07T00:00:00Z' or taking it relative to end timestamp like 'end timestamp - 360 days')

  112. lovetox_ has joined

  113. lovetox_


  114. lovetox_

    start is not a mandatory field, If omitted, the server SHOULD assume the value of 'start' to be equal to the date/time of the earliest message stored in the archive.

  115. pep.

    The idea here is not to get the earliest message then

  116. pep.

    It's to get messages before a what's already in the buffer for the room

  117. lovetox_

    you do that via RSM page request

  118. lovetox_

    not via mam search limit

  119. pep.

    It's to get messages before what's already in the buffer for the room

  120. lovetox_

    first you decide in what timespan the server should prepare messages

  121. lovetox_

    these messages are not sent ! then you decide what page of that request you want

  122. lovetox_

    so yes i want everything from first message in my chat to first message ever in the archive to be prepared by the server

  123. lovetox_

    then i want the last page of that result

  124. ralphm

    Then again syncing all of history can be pretty expensive (time, bandwidth, size). What we did at first start is get the last 7 days, and then on individual chats infinite scroll back if the user wanted to go further back.

  125. lovetox_

    ralphm, this has nothing to do with syncing full history

  126. lovetox_

    only because i tell the server operate within these limits, does not mean i request all the pages until i reach the limit

  127. lovetox_

    but yeah one could specify something like (end - 2 weeks ) or something

  128. ralphm

    Oh, then I misinterpreted what you wrote.

  129. lovetox_

    the problem with dates is you dont know how many messages are within a date span

  130. ralphm

    Yes, definitely

  131. lovetox_

    i can request backwars 1 year, and there are 0 messages

  132. lovetox_

    so i request backwards until the first message in the archive, then request in 30 page junks

  133. ralphm

    The date range was for the very first sync, for all of history across contacts.

  134. lovetox_

    and abort after 100 messages

  135. ralphm

    We used size limits (I think 50 or 100) for infinite scroll.

  136. lovetox_

    yes ralph, for forwards sync

  137. lovetox_

    date range is good

  138. lovetox_

    on first start

  139. lovetox_

    i do also 7 days i think

  140. ralphm

    Had some with non-chat messages, which ties into handling of reactions, edits.

  141. ralphm

    Some issues

  142. lovetox_

    everything but mam full sync from a stanza-id forward

  143. lovetox_

    is messy

  144. ralphm

    Things like deliver/read markers, CDRs (for calls).

  145. frainz has left

  146. lovetox_

    thats why i still didnt implement backwards scrolling, just to many issues that can arise from it

  147. lovetox_

    and im not sure full sync is so bad

  148. lovetox_

    because most servers have a 3 months limit on storage anyway

  149. pep.

    Tell that to my CPU :p

  150. madhur.garg

    By default, does it takes the start timestamp of the first message? (I am not sure if that is true, but will check it again) So, what should be good to go with nothing as a start date or with a date range of say 360 days?

  151. lovetox_

    i made tests with decent servers, and 3000 messages are downloaded in a matter of seconds

  152. frainz has joined

  153. pep.

    lovetox_, I store as much as I can on my server tbh, and I won't change that for anything

  154. lovetox_

    madhur.garg, set nothing as start, and abort after receiving X pages

  155. pep.

    lovetox_, we need something as start no? We don't start with an empty buffer

  156. lovetox_

    pep. this is no problem means on first start you have to wait 10 seconds, instead of 5

  157. pep.

    I remember dino downloading a year of private messages each time I'd reset it (testing stuff)

  158. lovetox_

    yeah so? if i present you with a loading progress i dont see a problem here

  159. lovetox_

    10.000 messages are probably 10 MB or something 😃

  160. pep.

    yeah, sure

  161. lovetox_

    pep. with backwards scrolling you want to define the end date, not start

  162. lovetox_

    you can reverse a mam request

  163. pep.

    So the end date means "From this date"?

  164. pep.

    and then backwards

  165. lovetox_

    see the mam requst like the DB query the server makes

  166. lovetox_

    start = first message of archive, end = whatever is first in your buffer

  167. lovetox_

    and start = emtpy, means first message of the archive

  168. lovetox_

    at least thats what the XEP mandates

  169. lovetox_

    thats only the result set then

  170. lovetox_

    after that you set with RSM which page from that result you want

  171. lovetox_

    obviously not the first

  172. lovetox_

    you want the last page of that db query

  173. lovetox_

    see 0059 for how to request the last page of a result set

  174. ralphm

    I'd never want to use a server that limits how much history is kept.

  175. pep.

    lovetox_, I want the last page of that query because.. it's in reverse order?

  176. Nekit has joined

  177. frainz has left

  178. frainz has joined

  179. lovetox_

    pep. you cant reverse a query

  180. lovetox_

    a query is always from start to end within all the messages of the archive

  181. lovetox_

    and in the order first to last message

  182. lovetox_

    if your first message in the buffer is number 10

  183. UsL has joined

  184. lovetox_

    then you request from the server messages 1-9, then limit that result with RSM so the server gives you the last page of that result

  185. lovetox_

    and if your page size=1

  186. lovetox_

    this means he sends you number 9

  187. lovetox_

    you successfully backscrolled !

  188. debacle has left

  189. pep.

    right, ok

  190. frainz has left

  191. frainz has joined

  192. debacle has joined

  193. lovetox_ has left

  194. Nekit has left

  195. Nekit has joined

  196. Guus has left

  197. Guus has joined

  198. alacer has left

  199. alacer has joined

  200. pdurbin has left

  201. lumi has joined

  202. UsL has left

  203. UsL has joined

  204. patrick has joined

  205. alacer has left

  206. Daniel has left

  207. Daniel has joined

  208. Daniel has left

  209. Daniel has joined

  210. mr.fister has left

  211. ziggys has joined

  212. Daniel has left

  213. Daniel has joined

  214. alacer has joined

  215. alacer has left

  216. alacer has joined

  217. kokonoe has left

  218. kokonoe has joined

  219. alacer has left

  220. alacer has joined

  221. karoshi has left

  222. karoshi has joined

  223. alacer has left

  224. alacer has joined

  225. pdurbin has joined

  226. pdurbin has left

  227. COM8 has joined

  228. COM8 has left

  229. Nekit has left

  230. Nekit has joined

  231. ziggys has left

  232. ziggys has joined

  233. COM8 has joined

  234. COM8 has left

  235. adityaborikar has left

  236. adityaborikar has joined

  237. COM8 has joined

  238. COM8 has left

  239. dele2 has joined

  240. APach has left

  241. alacer has left

  242. alacer has joined

  243. APach has joined

  244. Lance has joined

  245. alacer has left

  246. alacer has joined

  247. valo has left

  248. valo has joined

  249. alacer has left

  250. APach has left

  251. Lance has left

  252. APach has joined

  253. pdurbin has joined

  254. pdurbin has left

  255. Nekit has left

  256. Lance has joined

  257. Andrew Nenakhov has left

  258. Andrew Nenakhov has joined

  259. adityaborikar has left

  260. adityaborikar has joined

  261. wojtek has joined

  262. sezuan has left

  263. Lance has left

  264. Lance has joined

  265. lovetox has joined

  266. dele2 has left

  267. lskdjf has left

  268. lskdjf has joined

  269. Yagiza has joined

  270. adityaborikar has left

  271. adityaborikar has joined

  272. pdurbin has joined

  273. Kev has left

  274. pdurbin has left

  275. curen has left

  276. Guus has left

  277. Guus has joined

  278. flow has left

  279. waqas has joined

  280. waqas has left

  281. valo has left

  282. valo has joined

  283. wojtek has left

  284. wojtek has joined

  285. flow has joined

  286. andrey.g has left

  287. Holger has left

  288. Nekit has joined

  289. andrey.g has joined

  290. frainz has left

  291. frainz has joined

  292. frainz has left

  293. frainz has joined

  294. frainz has left

  295. rion has left

  296. rion has joined

  297. frainz has joined

  298. Lance has left

  299. wojtek has left

  300. Holger has joined

  301. pdurbin has joined

  302. debacle has left

  303. pdurbin has left

  304. Kev has joined

  305. Lance has joined

  306. Yagiza has left

  307. Kev has left

  308. lskdjf has left

  309. lskdjf has joined

  310. lskdjf has left

  311. lskdjf has joined

  312. DebXWoody has left

  313. lskdjf has left

  314. lskdjf has joined

  315. lskdjf has left

  316. lskdjf has joined

  317. DebXWoody has joined

  318. lskdjf has left

  319. lskdjf has joined

  320. goffi has left

  321. goffi has joined

  322. mr.fister has joined

  323. ziggys has left

  324. debacle has joined

  325. goffi has left

  326. remko has left

  327. pep.

    TIL https://xmpp.org/extensions/xep-0142.html

  328. lovetox


  329. pdurbin has joined

  330. pdurbin has left

  331. kokonoe has left

  332. kokonoe has joined

  333. Nekit has left

  334. frainz has left

  335. remko has joined

  336. ziggys has joined

  337. murabito has left

  338. murabito has joined

  339. Mikaela has left

  340. Mikaela has joined

  341. wurstsalat has left

  342. frainz has joined

  343. igoose has left

  344. lovetox has left

  345. igoose has joined

  346. Mikaela has left

  347. remko has left

  348. ziggys has left

  349. ziggys has joined

  350. lnj has left

  351. lskdjf has left

  352. lskdjf has joined

  353. moparisthebest has left

  354. moparisthebest has joined

  355. lskdjf has left

  356. Daniel has left

  357. lskdjf has joined

  358. Neustradamus has left

  359. lskdjf has left

  360. lskdjf has joined

  361. pdurbin has joined

  362. karoshi has left

  363. lskdjf has left

  364. lskdjf has joined

  365. Daniel has joined

  366. lskdjf has left

  367. lskdjf has joined

  368. lskdjf has left

  369. lskdjf has joined

  370. waqas has joined

  371. lskdjf has left

  372. lskdjf has joined

  373. Daniel has left

  374. lskdjf has left

  375. lskdjf has joined

  376. debacle has left

  377. lskdjf has left

  378. lskdjf has joined

  379. lskdjf has left

  380. lskdjf has joined

  381. lskdjf has left

  382. lskdjf has joined

  383. remko has joined

  384. pdurbin has left

  385. curen has joined

  386. lskdjf has left

  387. lskdjf has joined

  388. lskdjf has left

  389. lskdjf has joined

  390. UsL has left

  391. UsL has joined

  392. Daniel has joined

  393. lskdjf has left

  394. lskdjf has joined

  395. lskdjf has left

  396. lskdjf has joined

  397. remko has left

  398. lskdjf has left

  399. lskdjf has joined

  400. ziggys has left

  401. intosi has left

  402. intosi has joined

  403. ziggys has joined

  404. lskdjf has left

  405. lskdjf has joined