jdev - 2021-08-23


  1. paul has left
  2. me9 has left
  3. mikeye has joined
  4. xecks has left
  5. Duncan Hart has left
  6. Duncan Hart has joined
  7. Duncan Hart has left
  8. Duncan Hart has joined
  9. Sam Talking about how to query MAM earlier someone pointed out that if you're using start/end you can get rid of the duplicae last message by also adding a RSM before or after. This effectively makes your query the same as if you'd used the mam#extended "before-id" and "after-id". This made me wonder why not just always query for all history and filter using before/after?
  10. mikeye has left
  11. mac has joined
  12. laylolamb has left
  13. Alex has left
  14. dezant has left
  15. wurstsalat has left
  16. laylolamb has joined
  17. Duncan Hart has left
  18. moparisthebest TCP head-of-line blocking ?
  19. Sam I don't understand what you mean; how would that be related?
  20. mac has left
  21. laylolamb has left
  22. moparisthebest has left
  23. Duncan Hart has joined
  24. dezant has joined
  25. 9lakes has left
  26. Yagizа has joined
  27. moparisthebest has joined
  28. moparisthebest has left
  29. moparisthebest has joined
  30. laylolamb has joined
  31. emus has left
  32. paul has joined
  33. mac has joined
  34. antranigv has left
  35. dezant has left
  36. dezant has joined
  37. Sam has left
  38. Duncan Hart has left
  39. Sam has joined
  40. Duncan Hart has joined
  41. DebXWoody has joined
  42. mac has left
  43. mac has joined
  44. Duncan Hart has left
  45. scorch has left
  46. _Liveware Problem_ has joined
  47. scorch has joined
  48. Alex has joined
  49. jonas’ Sam, because you cannot filter with before/after. you'll only get N results there, limited to the page size.
  50. jonas’ and using before *and* after is underspecified IIRC
  51. jonas’ and using before *and* after (in the same query) is underspecified IIRC
  52. lovetox has left
  53. Sam has left
  54. goffi has joined
  55. pulkomandy has left
  56. pulkomandy has joined
  57. Duncan Hart has joined
  58. mikeye has joined
  59. dezant has left
  60. marc has left
  61. marc has joined
  62. Sam has joined
  63. wurstsalat has joined
  64. pulkomandy has left
  65. pulkomandy has joined
  66. antranigv has joined
  67. debacle has joined
  68. scorch has left
  69. scorch has joined
  70. aram has left
  71. mikeye has left
  72. me9 has joined
  73. flow some even believing that using both, 'before' and 'after', should be disallowed
  74. Zash Hence the mam#extended `before-id` and `after-id`
  75. Duncan Hart has left
  76. Duncan Hart has joined
  77. antranigv has left
  78. Alex has left
  79. Duncan Hart has left
  80. Alex has joined
  81. mikeye has joined
  82. antranigv has joined
  83. antranigv has left
  84. antranigv has joined
  85. Duncan Hart has joined
  86. antranigv has left
  87. antranigv has joined
  88. antranigv has left
  89. antranigv has joined
  90. sth has joined
  91. mikeye has left
  92. me9 has left
  93. Duncan Hart has left
  94. Duncan Hart has joined
  95. Duncan Hart has left
  96. Duncan Hart has joined
  97. Duncan Hart has left
  98. Duncan Hart has joined
  99. Duncan Hart has left
  100. Duncan Hart has joined
  101. debacle has left
  102. Duncan Hart has left
  103. Duncan Hart has joined
  104. me9 has joined
  105. Matrix Traveler (bot) has left
  106. Server Stats Discoverer (traveler bot) has left
  107. homebeach has left
  108. Matrix Traveler (bot) has joined
  109. homebeach has joined
  110. Server Stats Discoverer (traveler bot) has joined
  111. Duncan Hart has left
  112. Duncan Hart has joined
  113. Duncan Hart has left
  114. Duncan Hart has joined
  115. Squeaky Latex Folf has left
  116. Duncan Hart has left
  117. Duncan Hart has joined
  118. Squeaky Latex Folf has joined
  119. Wojtek has joined
  120. Duncan Hart has left
  121. Duncan Hart has joined
  122. xecks has joined
  123. 9lakes has joined
  124. Duncan Hart has left
  125. emus has joined
  126. Duncan Hart has joined
  127. debacle has joined
  128. sth has left
  129. Duncan Hart has left
  130. Duncan Hart has joined
  131. mac has left
  132. Duncan Hart has left
  133. Duncan Hart has joined
  134. paul has left
  135. Duncan Hart has left
  136. Duncan Hart has joined
  137. 9lakes has left
  138. Duncan Hart has left
  139. Duncan Hart has joined
  140. 9lakes has joined
  141. Duncan Hart has left
  142. Duncan Hart has joined
  143. Duncan Hart has left
  144. Duncan Hart has joined
  145. sth has joined
  146. paul has joined
  147. lovetox has joined
  148. Duncan Hart has left
  149. scorch has left
  150. sth has left
  151. Alex has left
  152. Alex has joined
  153. scorch has joined
  154. antranigv has left
  155. antranigv has joined
  156. scorch has left
  157. Wojtek has left
  158. antranigv has left
  159. Sam Makes sense if you need both before and after, but I don't
  160. Sam I don't get the thing about paging though, it would start the first page after the one you want, then you'd just page like normal, no? Just like after-id
  161. Sam So I don't get why you'd only get N results
  162. scorch has joined
  163. jonas’ feature request: all MAM implementatinos should encrypt their before/after tags symmetrically so that implementatoins learn that they are opaque strings and only by coincidence match the archive IDs.
  164. Zash Huuh
  165. MattJ Ah but XEP-0313 requires that they *are* the archive IDs, sorry :)
  166. jonas’ MattJ, shush!
  167. Zash But whyyyyyy
  168. jonas’ (that requirement should've been removed with the introduction of after-id/before-id
  169. MattJ But I agree, that would have been a possible change for #extended :)
  170. jonas’ (that requirement should've been removed with the introduction of after-id/before-id)
  171. Zash We could have had query cursors!
  172. Sam Yah, archive IDs are opaque strings and those are archive IDs, right? I really don't understand the difference apparently
  173. jonas’ Sam, no
  174. jonas’ archive IDs are opaque strings you feed into before-id / after-id
  175. Zash mam#extended2
  176. Sam If they're not what before/after are for what goes there? It's the id attribute which has to match the archive ID, no?
  177. jonas’ the things you feed into <before/> and <after/> are opaque strings you get from the RSM result returned by the server (<last/>, <first/> in the <rsm/> tag)
  178. Sam So you can't start a query with before/after?
  179. jonas’ no
  180. jonas’ (or: "exactly")
  181. Sam Hmm, that's what was recommended to me the other day and it seems to work
  182. Zash That depends on the RSM ... combination with other protocol
  183. MattJ In normal RSM, you can't. It works because XEP-0313 forces implementations to use archive IDs in RSM.
  184. Sam So it seems fine to do in 0313 then
  185. jonas’ but why would you rely on this specificity of 313 instead of doing it generically?
  186. jonas’ (especially if it might go away…)
  187. Zash Can you take it away tho, without going mam:3
  188. Zash (we're at mam:2 still, right? RIGHT?)
  189. MattJ It's unlikely to be going anywhere... :)
  190. Sam Because I am a normal user who knows nothing about XEP development and doesn't know it will go away and has started using it, making it harder to take away, just like all XEP development
  191. jonas’ Zash, > Implementation of the protocol described herein is encouraged in exploratory implementations, but production systems are advised to carefully consider whether it is appropriate to deploy implementations of this protocol before it advances to a status of Draft.
  192. Zash jonas’, you know as well as me that nobody cares about that 🙂
  193. jonas’ Zash, it's monday, shush ;P
  194. Zash Supply and demand
  195. Zash Maybe Draft should be forced once it's discovered that the XEP is used in production
  196. jonas’ haha
  197. Sam I mean, if it's bad somehow fine, I won't use it, but this seems like a perfectly reasonable thing to do after reading the XEP. Also none of this is made clear anywhere, and other users are apparently doing it (it was Gajim that recommended using it to me), so if you don't want users doing it, seems like the XEP should have made it clear what the differences are, not just made two after's that are exactly the same ("you" being "people who don't want after/before to be used" not the original authors of MAM)
  198. jonas’ to me, it's not clear why people think it's a fine thing to do after reading RSM, I'm still trying to figure out (for years) how to make it clear to folks what the difference between query arguments to the backing protocol (e.g. MAM, disco, jabber search) and <before/> and <after/> is.
  199. Sam Is there a place that says "after is something different, here's what it is" in MAM?
  200. MattJ I really don't think it's important, since it's a per-protocol thing anyway
  201. MattJ and MAM explicitly went down this route. I know it doesn't help RSM in the general case, but I think MAM was the first thing to truly use RSM in most implementations.
  202. MattJ Which is sad, because the next thing will mean everyone will either reimplement it or need to refactor it to be generic.
  203. Sam Hmm, okay, the one place that *might* indicate how after is used is the paragraph that starts "Having previously made a query that returned results limited by the server"
  204. Sam That seems to indicate that after can't be used in the first query. However, servers seem to support it, so maybe it's just a case of the XEP not covering edge cases well and servers doing the wrong thing
  205. Sam In which case, we either need to outright forbid it and go into that fight with implementations, or just accept it and codify it (which could possibly result in getting rid of after-id since no one uses it anyways yet and this would be identical at that point)
  206. MattJ It is codified in the XEP, it's too late for that
  207. MattJ "For the purposes of this protocol, the UIDs used by RSM correspond with the UIDs of the stanzas stored in the archive."
  208. MattJ and "Note: There is no concept of an "open query", and servers MUST be prepared to receive arbitrary page requests at any time."
  209. Sam So I don't see why people care that others are using it then or saying it's wrong, seems fine and removes the need to support archive-id which is just a duplicate ¯\_(ツ)_/¯
  210. Sam after-id, I mean
  211. MattJ Sam, because people have opinions. Mine isn't strong on this one :)
  212. MattJ MAM uses RSM because it needed a paging mechanism, and RSM was exactly that.
  213. Sam I mean, yah, sure, I'm just trying to understand if those opinions are grounded in some actual problem I'm going to run into
  214. jonas’ Sam, unless you need to select a range of messages, because you can't do that with RSM alone.
  215. MattJ Later people requested arbitrary range requests, which RSM was not
  216. Sam jonas’: sure, that part makes sense, but I don't need to do that.
  217. Sam Also, I kind of wonder if that works just fine too, but I haven't tested it
  218. emus has left
  219. jonas’ it's at least undefinde behaviour as per the spec
  220. MattJ In glorious hindsight we'd have supported that from the beginning, and possibly would have decided not to use RSM (or not use it in the same way)
  221. emus has joined
  222. jonas’ so you might end up with different server behaviour across implementations and releases
  223. Sam Right, but I'm not using before and after, so that's fine. As far as I can tell, just querying for after *or* before is well specified and means they are exact duplicates of after-id and before-id (at least, when used on the first page). So that seems like a good strategy to have MAM that works *today* and not have to wait for after-id.
  224. MattJ Yes
  225. Zash > querying for after *or* before is well specified in RSM, right > and means they are exact duplicates of after-id and before-id sans RSM related semantics, like how using before means paging in the other direction
  226. moparisthebest has left
  227. moparisthebest has joined
  228. Sam oh yah, good point; also doesn't matter for my use case thankfully (in fact, it's desirable), but that is a difference that could matter to some people.
  229. Sam Thanks for the help all. The only thing still nagging me is whether there's any benefit to continuing to use start/end to limit the initial query. At least from the client side I don't think there is, and if you're using a relational database I don't think there is on the server (assuming you have an index on the archive ID). With other non-relational databases I don't really know.
  230. Sam MattJ: would you be opposed to a summary of this discussion being added to the implementation notes in MAM, or possibly on that page on modernxmpp? I think I finally understand the picture here enough to write it down
  231. MattJ I was just thinking that (that it probably warrants an implementation note at this point)
  232. Sam *nods* I'll draft some text and send it your way shortly.
  233. moparisthebest has left
  234. MattJ Sounds good :)
  235. flow Sam, feel free to announce the PR for modernxmpp here as well, I'd be interested to read your text. Just to see if it matches my understanding too
  236. Wojtek has joined
  237. Sam flow: will do
  238. Alex has left
  239. antranigv has joined
  240. flow thanks :)
  241. Matrix Traveler (bot) has left
  242. homebeach has left
  243. Server Stats Discoverer (traveler bot) has left
  244. Matrix Traveler (bot) has joined
  245. homebeach has joined
  246. Server Stats Discoverer (traveler bot) has joined
  247. moparisthebest has joined
  248. Sam flow, MattJ: done: https://github.com/xsf/xeps/pull/1099
  249. kikuchiyo has left
  250. kikuchiyo has joined
  251. mikeye has joined
  252. mikeye has left
  253. paul has left
  254. me9 has left
  255. Alex has joined
  256. Alex has left
  257. Alex has joined
  258. mikeye has joined
  259. norayr has left
  260. norayr has joined
  261. antranigv has left
  262. stuart.j.mackintosh has left
  263. paul has joined
  264. antranigv has joined
  265. spectrum has left
  266. spectrum has joined
  267. dezant has joined
  268. nephele has joined
  269. mikeye has left
  270. Alex has left
  271. tsk has left
  272. tsk has joined
  273. Alex has joined
  274. stuart.j.mackintosh has joined
  275. _Liveware Problem_ has left
  276. paul has left
  277. jgart has joined
  278. antranigv has left
  279. antranigv has joined
  280. dezant has left
  281. dezant has joined
  282. pulkomandy has left
  283. pulkomandy has joined
  284. antranigv has left
  285. Alex has left
  286. Alex has joined
  287. pulkomandy has left
  288. pulkomandy has joined
  289. lovetox has left
  290. lovetox has joined
  291. mac has joined
  292. paul has joined
  293. Alex has left
  294. norayr has left
  295. antranigv has joined
  296. emus has left
  297. emus has joined
  298. antranigv has left
  299. antranigv has joined
  300. Alex has joined
  301. dezant has left
  302. dezant has joined
  303. sonny has left
  304. sonny has joined
  305. Alex has left
  306. Alex has joined
  307. Alex has left
  308. _Liveware Problem_ has joined
  309. Alex has joined
  310. antranigv has left
  311. antranigv has joined
  312. Alex has left
  313. me9 has joined
  314. Alex has joined
  315. antranigv has left
  316. dezant has left
  317. kikuchiyo has left
  318. kikuchiyo has joined
  319. rom1dep has left
  320. kikuchiyo has left
  321. scorch has left
  322. debacle has left
  323. dezant has joined
  324. kikuchiyo has joined
  325. lovetox has left
  326. kikuchiyo has left
  327. scorch has joined
  328. debacle has joined
  329. rom1dep has joined
  330. nephele has left
  331. nephele has joined
  332. georgeorwell has left
  333. nephele has left
  334. georgeorwell has joined
  335. lovetox has joined
  336. nephele has joined
  337. Neustradamus has left
  338. nephele has left
  339. antranigv has joined
  340. DebXWoody has left
  341. georgeorwell has left
  342. Wojtek has left
  343. Yagizа has left
  344. sonny has left
  345. sonny has joined
  346. Syndace has left
  347. Syndace has joined
  348. dezant has left
  349. dezant has joined
  350. edhelas has left
  351. edhelas has joined
  352. dezant has left
  353. dezant has joined
  354. sonny has left
  355. sonny has joined
  356. sonny has left
  357. sonny has joined
  358. hiran has joined
  359. mac has left
  360. goffi has left
  361. norayr has joined
  362. Duncan Hart has joined
  363. sonny has left
  364. sonny has joined
  365. sonny has left
  366. sonny has joined
  367. debacle has left
  368. stpeter has joined
  369. me9 has left
  370. 9lakes has left
  371. emus has left
  372. selurvedu has joined
  373. dezant has left
  374. dezant has joined
  375. hiran has left
  376. _Liveware Problem_ has left