XSF Discussion - 2020-03-31

  1. sonny has joined

  2. j.r has left

  3. j.r has joined

  4. calvin has joined

  5. j.r has left

  6. j.r has joined

  7. j.r has left

  8. mukt2 has left

  9. Shell has left

  10. Shell has joined

  11. waqas has left

  12. waqas has joined

  13. j.r has joined

  14. arc has left

  15. arc has joined

  16. emus has left

  17. karoshi has left

  18. sonny has left

  19. sonny has joined

  20. winfried has left

  21. arc has left

  22. winfried has joined

  23. arc has joined

  24. winfried has left

  25. winfried has joined

  26. winfried has left

  27. winfried has joined

  28. neshtaxmpp has left

  29. Max has left

  30. Max has joined

  31. calvin has left

  32. adiaholic_ has joined

  33. neshtaxmpp has joined

  34. calvin has joined

  35. Roberto has left

  36. lskdjf has left

  37. adiaholic_ has left

  38. adiaholic_ has joined

  39. waqas has left

  40. pdurbin has joined

  41. waqas has joined

  42. calvin has left

  43. lovetox has left

  44. mukt2 has joined

  45. lovetox has joined

  46. APach has joined

  47. Yagiza has joined

  48. LNJ has left

  49. mukt2 has left

  50. andy has joined

  51. DebXWoody has joined

  52. Shell has left

  53. pdurbin has left

  54. Seve has joined

  55. eevvoor has joined

  56. Roberto has joined

  57. mukt2 has joined

  58. moparisthebest has left

  59. mukt2 has left

  60. Tobias has joined

  61. Guus has left

  62. Guus has joined

  63. mukt2 has joined

  64. eevvoor has left

  65. lorddavidiii has joined

  66. mukt2 has left

  67. Marc has left

  68. Marc has joined

  69. waqas has left

  70. bear has left

  71. goffi has joined

  72. DebXWoody has left

  73. mukt2 has joined

  74. mukt2 has left

  75. bear has joined

  76. LNJ has joined

  77. wurstsalat has left

  78. wurstsalat has joined

  79. emus has joined

  80. bear has left

  81. eevvoor has joined

  82. bear has joined

  83. karoshi has joined

  84. pdurbin has joined

  85. Nekit has joined

  86. pdurbin has left

  87. bear has left

  88. xsf has left

  89. lovetox has left

  90. DebXWoody has joined

  91. xsf has joined

  92. robertooo has joined

  93. mukt2 has joined

  94. Steve Kille has left

  95. Steve Kille has joined

  96. mukt2 has left

  97. bear has joined

  98. lovetox has joined

  99. debacle has left

  100. larma has left

  101. lskdjf has joined

  102. larma has joined

  103. adiaholic_ has left

  104. adiaholic_ has joined

  105. bear has left

  106. Guus

    Assume a scenario in which a user is online with a single resource, stream management enabled, then gets disconnected abruptly, leaving the SM session in a resumable state, and a stanza arrives. What now is the expected behavior if the same user establishes a new stream, but does not resume the session but performs resource binding instead?

  107. larma has left

  108. larma has joined

  109. Daniel

    The sm session 'times out' all stanzas get bounced (or what ever your server is doing)

  110. Daniel

    What would the alternative be?

  111. Zash

    Like, they closed their laptop, woke up their phone and left the office?

  112. Guus

    Well, for stanzas that were addressed to a bare JID, it might be desirable to re-evaluate the routing logic, delivering it to the newly established session.

  113. Zash

    I suspect this is a full week topic for a summit

  114. Guus

    and in case the new session rebinds using the same resource... maybe deliver all stanzas to that session?

  115. Daniel

    > and in case the new session rebinds using the same resource... maybe deliver all stanzas to that session? That would maybe work for some messages. But nothing else

  116. Zash

    And messages usually go to the bare jid now, and you can get it from MAM

  117. Daniel

    Yes. So anything that is delivered to mam you might not want to actually bounce. I was just saying do what ever you do when you time out

  118. Guus

    ejabberd has an option that can be used to have all queued messages to be resend after SM times out (resend_on_timeout)

  119. Ge0rG

    Guus: that's bad if you have multiple clients.

  120. Ge0rG

    Guus: the existing implementations all suck

  121. flow

    and potentially breaks the stable order guarantee of XMPP (depending on your PoV)

  122. Ge0rG

    the right thing to do would be to do server-side tracking of individual clients, based on their hopefully constant-over-time resource string

  123. Max has left

  124. Max has joined

  125. Guus

    That's basically what I suggested, right? When resource binding on the same resource... do magic.

  126. Ge0rG

    Guus: indeed, yes. But you can't know how much of the old session actually already arrived at that device, unless it attempts to resume

  127. Ge0rG

    my client won't honor the resume timeout value and just attempt to resume always, if it has a previous session

  128. Guus

    Ge0rG That's very true.

  129. Guus

    problem here is that this client doesn't keep track of the old session metadata if it crashes unexpectedly.

  130. Guus

    (unsure if that's fixable on their end - that seems to be the preferred approach to me)

  131. Ge0rG

    Guus: that's exactly what remains unsolved. Or you use MAM

  132. jonas’

    use MAM

  133. jonas’

    better than guessing

  134. Daniel

    I use MAM

  135. jonas’

    or churning the disk by persisting the entire stream state to disk on every. single. stanza.

  136. Ge0rG

    Guus: with MAM, as Daniel said, the server should drop all message stanzas from the old session when destroying it, and not bounce them

  137. Daniel

    > or churning the disk by persisting the entire stream state to disk on every. single. stanza. I always wanted to write an iOS client that does that

  138. Ge0rG

    (unfortunately, not all servers do that yet)

  139. Daniel

    I don't think that's too unreasonable if you plan it from the beginning and design the app around it

  140. Ge0rG

    Daniel: yes

  141. Ge0rG

    Daniel: all we need is a full redesign of the complete xmpp client stack

  142. Ge0rG

    it needs to be serializable in an efficient way, ideally with some kind of journal

  143. Ge0rG

    Android could also profit from that.

  144. mukt2 has joined

  145. Guus

    flow I'm getting this from the client developers. Do you know if that's true? > Another possible solution to the problem is to persist the stream id, session id and the number of acknowledged messages from the server and the client. This is not supported by SMACK by default,

  146. Guus

    ah, wait, I read that as "smack can't resume" but maybe they're saying "smack doesn't persist to disk"

  147. Daniel

    Well it's not as easy as storing those three values

  148. bear has joined

  149. Ge0rG

    Guus: yes, smack can resume, but not from disk

  150. jonas’

    you also need to store all presence state

  151. Ge0rG

    Guus: you'd need to serialize the XMPPTCPConnection and everything related to it to disk as well

  152. flow

    depends, no?

  153. flow

    but yes, you usually want to resurrect the XMPPTCPConnection with the same listeners setup as before it was suspended to persistent storage

  154. mukt2 has left

  155. flow

    but that does not necessarily require to serialize XMPPTCPConnection

  156. Ge0rG

    no... just most of its members.

  157. Daniel

    Well all pending stanza, all pending callbacks to iqs

  158. Daniel

    All state

  159. Daniel

    Like presence, mucs etc

  160. Ge0rG

    Guus: so you can't easily hibernate smack to disk and reload afterwards.

  161. Ge0rG

    Guus: but you can hope that your app won't be killed

  162. Guus

    and if you could, you'd have to do that preemptively on ever.single.stanza (or how did Jonas' put it?)

  163. flow

    Guus, what's the problem you are trying to solve (couldn't find it in the backlog)

  164. Ge0rG

    Guus: yes, after each change of the internal state (e.g. receiving or sending a stanza) you'd have to persist everything

  165. Guus

    flow just a question I received over email: client gets killed, the restart it, but won't get messages that have been delivered while offline (unless they do MAM).

  166. Daniel

    Ge0rG: well you have to persist the delta

  167. Ge0rG

    Daniel: but with smack, you need to manually extract the delta from the state of dozens of weakly linked objects

  168. Daniel

    Yeah don't use smack for that

  169. flow

    Guus, use MAM ;)

  170. Guus

    Yeah, that's been my conclusion too. Still, I'm wondering if it makes sense to have some kind of option like ejabberd's 'resend' - but instead, I think I'd prefer to send certain messages to a new session with the same resource - something like that.

  171. Guus

    probably bad for MUC though

  172. jonas’

    bad for a lot of things assuming state

  173. flow

    I also think it would violate the stable order requirement of the RFC (but that may not be an issue for your use-case)

  174. Guus

    I'd at the very least not enable this by default.

  175. Guus

    just trying to think of a solution that'd be helpful for certain use-cases.

  176. Ge0rG

    Guus: you can't do that generally because you would end up re-sending all the stanzas that already were delivered but not acked

  177. Ge0rG

    Guus: use MAM.

  178. Ge0rG

    Guus: or, use eternal 0198 sessions ;)

  179. Ge0rG

    (no, that doesn't solve the client crash problem)

  180. Guus

    You're right - but not every user is as obsessed with being 100.0% right all of the time as we are. 🙂

  181. Guus

    some duplication might be acceptable.

  182. Guus

    (to them)

  183. Guus

    And yes, MAM is the proper way to go - but that would probably make them spend more development resources than what they have available.

  184. bear has left

  185. Guus

    (afk - feeding the offspring)

  186. jonas’

    Guus, some SM implementations will also feed the unacked messages to offline store when the SM session gets terminated.

  187. jonas’

    that may be a viable alternative?

  188. Ge0rG

    yes, if you are guaranteed to only have one client per user, you can use something like prosody's mod_smacks_offline

  189. adiaholic_ has left

  190. adiaholic_ has joined

  191. !XSF_Martin has left

  192. !XSF_Martin has joined

  193. !XSF_Martin has left

  194. !XSF_Martin has joined

  195. andrey.g has left

  196. andrey.g has joined

  197. !XSF_Martin has left

  198. !XSF_Martin has joined

  199. paul has left

  200. paul has joined

  201. !XSF_Martin has left

  202. !XSF_Martin has joined

  203. adiaholic_ has left

  204. adiaholic_ has joined

  205. !XSF_Martin has left

  206. !XSF_Martin has joined

  207. mukt2 has joined

  208. adiaholic_ has left

  209. adiaholic_ has joined

  210. bear has joined

  211. !XSF_Martin has left

  212. !XSF_Martin has joined

  213. mukt2 has left

  214. !XSF_Martin has left

  215. !XSF_Martin has joined

  216. !XSF_Martin has left

  217. !XSF_Martin has joined

  218. !XSF_Martin has left

  219. !XSF_Martin has joined

  220. !XSF_Martin has left

  221. !XSF_Martin has joined

  222. !XSF_Martin has left

  223. !XSF_Martin has joined

  224. bear has left

  225. Nekit has left

  226. !XSF_Martin has left

  227. !XSF_Martin has joined

  228. !XSF_Martin has left

  229. !XSF_Martin has joined

  230. Maranda has left

  231. Maranda has joined

  232. !XSF_Martin has left

  233. !XSF_Martin has joined

  234. Douglas Terabyte has left

  235. Douglas Terabyte has joined

  236. !XSF_Martin has left

  237. !XSF_Martin has joined

  238. mukt2 has joined

  239. typikol has joined

  240. Douglas Terabyte has left

  241. Douglas Terabyte has joined

  242. !XSF_Martin has left

  243. pdurbin has joined

  244. bear has joined

  245. !XSF_Martin has joined

  246. typikol has left

  247. !XSF_Martin has left

  248. !XSF_Martin has joined

  249. adiaholic_ has left

  250. adiaholic_ has joined

  251. j.r has left

  252. pdurbin has left

  253. bear has left

  254. waqas has joined

  255. waqas has left

  256. waqas has joined

  257. Shell has joined

  258. j.r has joined

  259. karoshi has left

  260. karoshi has joined

  261. Shell has left

  262. calvin has joined

  263. bear has joined

  264. moparisthebest has joined

  265. eevvoor has left

  266. bear has left

  267. mukt2 has left

  268. neshtaxmpp has left

  269. adiaholic_ has left

  270. adiaholic_ has joined

  271. edhelas


  272. jonas’

    ah that

  273. jonas’

    it’s obvious from the UI when you know what to look out for

  274. jonas’

    (also, I don’t know of any end-to-end encrypted video conferencing software)

  275. adiaholic_ has left

  276. adiaholic_ has joined

  277. neshtaxmpp has joined

  278. pdurbin has joined

  279. bear has joined

  280. pdurbin has left

  281. Shell has joined

  282. edhelas

    when you're calling yourself locally it's end to end encryted 🤔 ?

  283. bear has left

  284. Ge0rG

    jonas’: that would be... interesting.

  285. pep.

    I'd be interested to know how expensive that is resource-wise

  286. pep.

    jitsi-meet is already getting complaints that they're not doing e2ee fwiw..

  287. jonas’

    essentially you’d degrade the videobridge to a stupid TURN server

  288. jonas’

    then you’re back to E2EE again

  289. jonas’

    if your webrtc session does it

  290. Ge0rG

    you could do homomorphic encryption!

  291. Ge0rG

    but I suppose the more practical approach would be to encrypt the stream payloads and to let videobridge route at the stream level.

  292. Ge0rG

    and to have multi-recipient keys for each stream

  293. Ge0rG

    you are typically using symmetric encryption for the data stream anyway, so just send the decryption key to all parties

  294. Ge0rG

    it's not complex, just requires effort.

  295. jonas’

    right, recipient keys

  296. Ge0rG

    also probably won't work with WebRTC.

  297. jonas’

    it should be possible on the protocol level, but I wouldn’t be surprised if javascript APIs don’t expose this

  298. Ge0rG

    jonas’: it depends on how you derive the session key

  299. mukt2 has joined

  300. Max has left

  301. Max has joined

  302. karoshi has left

  303. karoshi has joined

  304. stpeter has joined

  305. stpeter has left

  306. mukt2 has left

  307. Wojtek has joined

  308. adiaholic_ has left

  309. adiaholic_ has joined

  310. bear has joined

  311. flow

    also downscaling become

  312. flow

    also downscaling by the bridge becomes impossible with e2ee

  313. Ge0rG

    which jitsi meet doesn't do AFAIK

  314. Ge0rG

    the right thing would be to create multiple progressive streams, where you get the lowest res with stream 0, the medium res with stream 0+1 and the highest res with 0+1+2

  315. Ge0rG

    but that would require a progressive video codec of sorts.

  316. flow

    do those exists in practice?

  317. jonas’

    jitsi-videobridge (the software) does not do recoding, but there’s allegedly a hardware thing which does

  318. bear has left

  319. Daniel has left

  320. rion has left

  321. rion has joined

  322. Daniel has joined

  323. Student has joined

  324. adiaholic_ has left

  325. adiaholic_ has joined

  326. paul has left

  327. werdan has joined

  328. paul has joined

  329. Shell has left

  330. adiaholic_ has left

  331. Shell has joined

  332. Neustradamus

    What is the solution about XMPP TLS S2S problems for XMPP clients? Example: When an user@xmpp1.tld tries to join a mucroom@conference.xmpp2.tld, it is not possible and the XMPP client has not an error.

  333. pdurbin has joined

  334. bear has joined

  335. MattJ

    Neustradamus, the solution is to switch to Matrix

  336. Neustradamus


  337. Neustradamus

    Prosody will be a Matrix server?

  338. MattJ

    Who knows

  339. Holger


  340. Holger

    Conversations says "remote server not found" if I try to join mucroom@conference.xmpp2.tld.

  341. MattJ

    Holger, in case you missed the previous discussion on this topic, you probably don't want to reopen this can of worms

  342. Holger


  343. Holger shuts up.

  344. adiaholic_ has joined

  345. paul has left

  346. paul has joined

  347. Student has left

  348. pdurbin has left

  349. Neustradamus

    Holger: The log on muc.xmpp.org: - User has joined - User has left Client has not error and the client always tries but it has already failed.

  350. Ge0rG


  351. Neustradamus

    Ge0rG: example jabber.org (not only) -> xmpp.org is broken.

  352. bear has left

  353. bear has joined

  354. MattJ

    New draft MAM revision currently here: https://github.com/mwild1/xeps/tree/mam-r7

  355. MattJ

    I'll post to the list about it soon (probably tomorrow)

  356. MattJ

    I have a few things to do before I PR (such as finishing off the other documents I split stuff out to)

  357. mukt2 has joined

  358. Zash


  359. sonny has left

  360. mukt2 has left

  361. lorddavidiii has left

  362. lorddavidiii has joined

  363. lorddavidiii has left

  364. lorddavidiii has joined

  365. lovetox

    nice MattJ

  366. lovetox

    does it make sense to flip the rsm page, and not the whole mam filter result?

  367. lovetox

    notice this is not the same outcome

  368. lorddavidiii has left

  369. lovetox

    i fail to see the use case for flipping the page

  370. lorddavidiii has joined

  371. lovetox

    i does not get you the filter result in the reverse order

  372. lovetox

    it does only get you the first page of the filter result, which you dont actually want, we want the last page, in reverse order

  373. jonas’

    MattJ, the "Client uses two discovered query fields in a query" example differs from the previous example in the value of the @var attributes, is that intentional?

  374. LNJ has left

  375. bear has left

  376. calvin has left

  377. calvin has joined

  378. alexis has left

  379. pep.

    MattJ, does that mean pubsub mam is going to be defined somewhere else?

  380. Zash

    Maybe randomize the subdomain in `<iq to='pubsub.shakespeare.lit' type='set' id='juliet1'>` a bit more?

  381. flow

    pep does it matter where it will potentially be defined?

  382. pep.

    flow, it matters that it's defined, as with this PR it doesn't seem to be defined anywhere anymore

  383. sonny has joined

  384. Zash

    > I have a few things to do before I PR (such as finishing off the other documents I split stuff out to) pep. ↑

  385. calvin has left

  386. calvin has joined

  387. pep.


  388. pep.

    waiting for updates then

  389. flow

    pep., was it defined before in xep313? IIRC it was just something like "ya'll can do MAM on PubSub, but I don't tell you exactly how<period>"

  390. Zash

    There's some precedent in splitting a XEP since at least MIX

  391. Zash

    flow, didn't say a lot about it, but there was a few lines

  392. pep.

    flow, not saying it was perfect but at least it was a thing :P

  393. pep.

    I'm sure it could be defined a lot better

  394. flow

    I think I would prefer if xep313 simply stated that MAM on PubSub is a possiblity but the exact specification is outside the scope if this XEP

  395. flow

    hmm, not sure about that <flip-page/> thing…

  396. flow

    hope it's optional

  397. lovetox

    i think i understand now the flip page thing, you are supposed to request the last page with rsm via </before> then flip that page

  398. lovetox

    and why do you hope flow its optional? are you a server developer that needs to implement that ?

  399. lovetox

    MattJ, what is the reason instead of namespace bumping this to mam:3, making it a disco feature?

  400. flow

    lovetox, I consider a minimal mandatory-to-implement feature set to be benefical, as it allows to ramp up a new implementation fast

  401. flow

    sure there are counterarguments, and one has to carefully select which feature to make optional

  402. flow

    but <flip-page/> appears to be a good candidate

  403. Zash

    So that's mam:2, the new stuff is mam:2#extended

  404. pep.

    Zash, but it's essentially the same

  405. pep.

    mam:3 or mam:2 + mam:2#extended

  406. flow

    +A client that wishes to use flipped pages MUST ensure that the server +advertises the \'urn:xmpp:mam:2\#extended\' feature.

  407. pep.

    mam:2#extended is just the new mam:3

  408. Zash

    pep., adding support doesn't break all existing clients

  409. flow

    pep I don't think it is

  410. Zash

    to the server, that is

  411. flow

    #extended sounds like an optional extension to mam:2

  412. lovetox

    Zash adding the extended things does not brak clients

  413. lovetox

    as it says it EXTENDS the spec

  414. pep.

    It wouldn't break all clients indeed, just continue supporting mam:2 just like you're going to do anyway

  415. lovetox

    there is no change to existing functionality

  416. Zash

    lovetox, that's what I said. bumping the namespace to :3 breaks everything, unless you do both

  417. !XSF_Martin has left

  418. Zash

    If you have a mam:2 implementation, adding mam:2#extended doesn't affect anyone

  419. pep.

    Zash, because you're still advertizing for mam:2. Which you can continue doing

  420. lovetox

    it also affects no one if you just announce mam:3

  421. !XSF_Martin has joined

  422. flow

    well, it hopefully affects the counterparty that could talk #extended with you

  423. MattJ

    Doing both has turned out to be a pain i n pretty much every implementation I've seen (when we bumped other namespaces)

  424. flow

    lovetox, as long as you still support mam:2

  425. pep.

    MattJ, I genuinely don't get the difference

  426. lovetox

    flow its just a namespace there is nothing the server needs to do different

  427. pep.

    Just that it's not explicit it's a new thing, you're just adding a new feature and now I've got to check both

  428. lovetox

    its just, if i get a IQ with mam:3 just treat it like mam:2

  429. lovetox


  430. MattJ

    pep.: you don't have to check both

  431. MattJ

    Just one

  432. pep.

    Yeah so it's just mam:3

  433. MattJ


  434. flow

    honestly, and I haven't read that diff completely so I may not seeing the whole picture, but I am surprised that the #extended thing causes any discussion

  435. pep.

    Or undercovered mam:3 :P

  436. pep.

    flow, to me it's not about mam, it's about how things are done in general

  437. lovetox

    let me guess flow you never implemented mam?

  438. flow

    If the #extended thing is that I think it is, an optional extension to the mam we have, than that appears very sensible to me

  439. flow

    instead of doing mam:3

  440. flow

    lovetox, I did the MAM implementation for Smack

  441. pep.

    Feels like people are just afraid to increase a number

  442. flow

    and wrote the unit and integration tests

  443. flow

    pep., of course we are afraid

  444. lovetox

    then i really dont get how you dont understand how this is bad

  445. pep.

    Why? It's the exact same thing

  446. lovetox

    but let me tell you why

  447. flow

    namespace bumps are not for free

  448. flow

    smack is still on mam:1 fwiw

  449. pep.

    flow, breaking changes are not free

  450. lovetox

    this is not a simple syntax change in the xep, this extended functionality allows clients to do mam in a totally different way

  451. flow

    pep., right that is why they have to be avoided when sensible

  452. lovetox

    its a change how archiving is done on client side in fundamental way

  453. pep.

    namespace bump doesn't equal breaking changes

  454. lovetox

    to say its optional on server side

  455. lovetox

    means that server dont have to adopt it

  456. lovetox

    so that means i have to support mam:2 without extended

  457. pep.

    Right that also..

  458. lovetox

    which is now not just a simple syntax change

  459. lovetox

    no i have to maintain a totally different way of doing mam

  460. Zash

    Has everyone forgotten the pain of mam:0 vs mam:1 vs mam:2 so soon?

  461. lovetox

    what for?, these changes are trivial for servers

  462. pdurbin has joined

  463. lovetox

    this is just some Mysql querys added differently

  464. Zash

    lovetox: haha no

  465. flow

    pep, I am sorry I fail to see how a namespace bump von mam:2 to mam:3 is the exact same thing as an optional extension to mam:2

  466. pep.

    one could say mam:3 is an optional extension over mam:2 :)

  467. flow

    mam:3 would be by definition a breaking change, as it requires a namespace bump

  468. lovetox

    flow the changes here proposed are long wished things of client developers

  469. lovetox

    this is not some optional thing that is nice

  470. lovetox

    these are changes the community deems necessary for an archiving xep

  471. lovetox

    there is really no reason to make this optional

  472. lovetox

    especially for a experimental xep

  473. flow

    there is also no reason to make it not optional?

  474. lovetox

    the only reason something like this would be optional, if there are reasons like we believe there are implementations who simply cant implement the changes because of tecnical problems

  475. MattJ

    Only that it is good features that people want to use

  476. flow

    lovetox, I mean if what you and pep want is that servers are going to announce mam:2 *and* mam:3 simultanously, then I still think that this is far more effort to support than annoucning mam:2 and mam:2#extensions

  477. flow

    because some protocol operations now need to be available under them mam:2 and mam:3 namespace

  478. lovetox

    no flow same operations can be available under both

  479. lovetox

    its no difference

  480. lovetox

    its a subset please

  481. flow

    that's what I said, some operations need to be avaialble under both namespace

  482. lovetox

    its the same operation

  483. lovetox

    do you really think they duplicate the code because of a mam:3 string in an iq

  484. flow

    no, but that does mean that it will become less ugly

  485. flow

    no, but that doesn't mean that it will become less ugly

  486. Zash

    lovetox, yes, bunch of duplication and more complex code

  487. !XSF_Martin has left

  488. lovetox

    ok but thats of no concern, this is the XMPP Archving XEP this is not some sideshow where we make concessions, these are essential changes

  489. lovetox

    every client needs that

  490. flow

    then every client will implement that

  491. !XSF_Martin has joined

  492. lovetox

    its nothing that some weird use cases need and so we make it optional

  493. Zash

    seems to me you're making very broad assumptions

  494. lovetox

    if you tell someone XMPP has a archiving xep where you cant even download a single specific message

  495. Shell has left

  496. lovetox

    and thats an OPTIONAL feature, some servers implement

  497. lovetox

    thats insane

  498. Shell has joined

  499. MattJ

    We're still in a world where some servers don't support it at all :)

  500. lovetox

    we dont need to get more fragmented, we need to move faster and decisive

  501. MattJ

    As far as I'm concerned, servers would be stupid not to implement the #extended stuff, but we can't force them to any more than we can force them to implement any XEP

  502. pdurbin has left

  503. flow

    lovetox, specifying something as mandatory does not mean that implementations will suddenly implement it. In fact, it will cause implementations that only implement a subset of the mandatory to implement features, which causes much more main. Features get implemented because someone decides that it is worth the effort.

  504. flow

    lovetox, specifying something as mandatory does not mean that implementations will suddenly implement it. In fact, it will cause implementations that only implement a subset of the mandatory to implement features, which causes much more pain. Features get implemented because someone decides that it is worth the effort.

  505. MattJ

    lovetox: btw, I added page flipping for you... if you don't want it I can take it back out ;)

  506. MattJ

    You and a couple of people who complained anyway

  507. lovetox

    no its good, i just wondered if its not to complicated to do the <before> and <flip-page> together

  508. flow

    I'd always assume that simply requesting a small page size would be sufficient

  509. lovetox

    instead of just reverse the filter query

  510. !XSF_Martin has left

  511. flow

    but I am also always in favor of experimenting with new features

  512. MattJ

    There was a reason I did it on the page rather the results, I can't remember why... might be in my notes from last year somewhere

  513. !XSF_Martin has joined

  514. MattJ

    There was a reason I did it on the page rather than the results, I can't remember why... might be in my notes from last year somewhere

  515. lovetox

    probably because of </before> in the rsm spec

  516. lovetox

    you can already request the last page of a query

  517. MattJ


  518. lovetox

    so you just have to flip that page to finish it

  519. !XSF_Martin has left

  520. MattJ

    I think that is logically simpler anyway

  521. lovetox

    but i guess we should mention that in the XEP

  522. lovetox

    because nobody will get this on their own

  523. Yagiza has left

  524. !XSF_Martin has joined

  525. lovetox

    it would be simpler if people would now the </before> thing in RSM

  526. MattJ

    There is an example for that

  527. lovetox

    in your mam change?

  528. lovetox

    ok thats nice then

  529. MattJ

    Yes, unless it got lost

  530. lovetox

    too much changes, i probably missed it

  531. MattJ

    I'm planning a separate "How to MAM" document anyway

  532. bear has joined

  533. lovetox

    what i want to tell is, that this is a "additional" thing on the server, you just add some different sql querys if you encounter different filters

  534. lovetox

    but for clients its not just additional

  535. lovetox

    its either you write your impl with these futures, or without it

  536. MattJ

    I agree, the sensible thing is for clients to depend on #extended

  537. lovetox

    at least i believe that, have to do it :)

  538. lovetox

    ok correct MattJ so from a client perspective this is the same as depending on mam:3

  539. lovetox

    so from a client perspective this is the same, and optional not useful here

  540. lovetox

    so the question remains, is it useful for the server?

  541. MattJ

    Not really, no

  542. MattJ

    I don't see any reason for people to only implement 80% of the XEP

  543. !XSF_Martin has left

  544. Zash

    We already implement 80% of the XEP, because those 80% was 100% of the previous version of the XEP.

  545. krauq has left

  546. MattJ


  547. !XSF_Martin has joined

  548. MattJ

    But whether it is called #extended or :3 it makes no difference - people need to upgrade

  549. krauq has joined

  550. jonas’

    the difference is that you can re-use your code from :2

  551. MattJ

    I mean it makes no difference to whether people implement it or not

  552. !XSF_Martin has left

  553. lovetox

    i dont know the process but is that code discussion really something the XSF should care about?

  554. Guus

    Skipping over a lot of the conversation here: but is the flip-page directive something that adds real value? To me, it seems like a convenience for clients to not have to revert element orders if they want to. I wonder if that warrants added complexity to the protocol.

  555. j.r has left

  556. !XSF_Martin has joined

  557. j.r has joined

  558. lovetox

    Guus you cant invert the order

  559. MattJ

    Guus: that's a debate to have with lovetox and some others... you can bring it up on the list when I post it I guess

  560. jonas’

    you can, you just need to buffer the (MAM) stanzas first.

  561. jonas’

    but on-list is a better place for this indeed

  562. !XSF_Martin has left

  563. lovetox

    thats a fundamental change to how clients work jonas’

  564. lovetox

    we dont do that for literally anything

  565. Zash

    Do you know how <before/> is implemented in Prosody? ... by buffering the results and then flipping them.

  566. MattJ

    To how your client works :)

  567. lovetox

    stanzas are processed when they arrive

  568. Guus

    flip-page is only about order of elements in one RSM result, right?

  569. !XSF_Martin has joined

  570. MattJ

    Guus: yes

  571. lovetox

    Guus a RSM page has usually 100 stanzas

  572. flow

    Hmm, then shouldn't something like flip-page be part of RSM instead?

  573. Guus

    ... not seeing the problem - unless the RSM pages are ungodly huge.

  574. Guus

    so - reduce the RSM size?

  575. lovetox

    then mam catchup is much slower ?

  576. flow

    Guus, I think something like flip-page really adds value over low latency connections

  577. MattJ

    flow: I considered that, but I don't want to bump RSM ;)

  578. flow

    MattJ, optional extension?

  579. Zash

    If you're doing infinite scroll and scroll up?

  580. Guus

    Okay, if it has real world benefits I'm happy to include it. Seems far-fetched to me, but then again, I'm no client builder 🙂

  581. lovetox

    also this means i have to download a 100 stanzas, before i can show the user the first

  582. lovetox

    seems like just not a good protocol

  583. jonas’

    slow connections is a valid point

  584. Guus

    Nah, just limit the max result set page size to 20 or somesuch?

  585. lovetox

    then you wait for 20 ..

  586. jonas’

    in other RSM cases it’s not as relevant (cc @ MattJ) because normally you get an RSM result page in a single stanza

  587. flow

    err, I meant high latency connections obviously

  588. Guus

    The entire point of having a page is that it offers a manageable bite size of data.

  589. jonas’

    Guus, then you have expensive round-trips. Flipping the result makes a lot of sense here.

  590. MattJ

    jonas’: yeah

  591. jonas’

    MAM is a splendid bulk operation from the client side

  592. flow

    but thinking about it a little more, I am actually not that sure if it adds value

  593. !XSF_Martin has left

  594. flow

    those 20 stanzas are all bundled

  595. jonas’

    Guus, especially on long-fat-links (high latency, lots of throughput) you want few roundtrips and lots of data. quite a bit of mobile links these days are like that, especially when you’re on the road: your packets are buffered somewhere and then you get a bunch of them back. roundtrips are expensive.

  596. flow

    and what's typical stanzas size? how long does it really take to receive 20 stanzas?

  597. jonas’

    flow, they don’t fit in a single TCP segment, so it can take arbitrarily long for the last (first) stanza to be delayed while you already have 19 others

  598. !XSF_Martin has joined

  599. flow

    true, if you not only have high latency but also packet loss

  600. jonas’

    or irregular high latency

  601. jonas’

    as you have on mobile links

  602. Guus

    I'm not dead-set against this - but how is this now a problem, and not in the past 10 years? Connectivity has improved, generally...

  603. jonas’

    you sometimes see sudden RTTs of 60s!

  604. jonas’

    Guus, it’s always been a problem

  605. flow

    as I said, I think <flip-page/> is a prime candidate of an optional feature. Not everyone may want or needs to implement it

  606. jonas’

    clients are just "crap" and sync from oldest messages first and not from newest first

  607. jonas’

    probably also because of stuff like this

  608. flow

    Guus, I'd argue bad connectiviy is probably more widespread

  609. jonas’

    when you sync from oldest, it doesn’t matter, but it’s also terrible UX

  610. flow

    if you think about all those smartphones in africe

  611. flow

    if you think about all those smartphones in africa

  612. jonas’

    or germany

  613. Guus

    oh well. I'll take your word for it 🙂

  614. jonas’

    once you’re in an intercity train

  615. Guus


  616. Guus

    (problem solved)

  617. Guus

    no wait, my parents' wifi.

  618. MattJ

    As far as I'm concerned it adds minimal value, but it makes lives for some people easier, so I'm fine with including it

  619. lovetox

    Sorry this is a messaging protocol and we discuss if its too much to ask from a archive to get the messages in the order you want them

  620. lovetox

    seriously lol

  621. lovetox

    download the archive and reverse them yourself

  622. lovetox


  623. MattJ

    lovetox: you just want SQL replication over XMPP ;)

  624. Yagiza has joined

  625. Guus

    no, we're discussing if adding a feature that I thought was trivial to implement in code warrants added complexity in the protocol

  626. jonas’

    MattJ, don’t confuse lovetox with zinid

  627. bear has left

  628. Guus

    that is a relevant discussion, especially in a standards body.

  629. Zash

    MattJ: You mean XEP-0136

  630. lovetox

    Guus, its the simple use case, of loading messages while you scroll upwards in your messenger

  631. lovetox

    just think from a client point of view how you want to implement that

  632. jonas’

    MattJ, is the auto-pull of xeps working again?

  633. lovetox

    this is not someting exotic, ANY user expect the client to do this

  634. lovetox

    in a efficient fashion

  635. Guus

    lovetox as I already said: I'm happy to accept that this is less trivial as what I thought it was, going into this discussion.

  636. !XSF_Martin has left

  637. MattJ

    Most XMPP alternatives will be using HTTP and JSON. Trust me, they don't start displaying results until they receive the full result set from the server

  638. !XSF_Martin has joined

  639. jonas’

    MattJ, but that also means that they can handle latency spikes much better, since they have regular TCP restarts instead of having to deal with a TCP recovering from a minute of packet loss

  640. Zash

    Did someone say BOSH?

  641. !XSF_Martin has left

  642. !XSF_Martin has joined

  643. MattJ

    jonas’: I'd be surprised to see anyone using a TCP connection per request these days

  644. jonas’

    MattJ, I wouldn’t be surprised for them restarting a fresh TCP connection when the other one runs in a timeout

  645. jonas’

    when you do that with XMPP, you’re in a world of massive pain

  646. jonas’

    unless you’re very clever, but nobody wants to be that these das

  647. jonas’

    unless you’re very clever, but nobody wants to be that these days

  648. jonas’

    (start a new session, SM-resume your other, suppesedly timed-out stream with your current local counters, discard the other stream once you got the resumption ACK)

  649. flow

    ahh muc-presence-versioning now in html, nice

  650. flow

    hmm, or not?

  651. jonas’

    I pushed the docker image, no idea if the pull is working

  652. jonas’

    hence 18:36:31 jonas’> MattJ, is the auto-pull of xeps working again?

  653. MattJ

    It's not

  654. pep.

    flow, "specifying something as mandatory does not mean that implementations will suddenly implement it" I totally agree. And I don't think this applies "we shall then make features optional". At least making something mandatory would tell implementors that these features are important

  655. jonas’

    MattJ, can you issue a manual pull then please?

  656. jonas’

    or, has board already decided about my iteam membership? ;)

  657. MattJ

    When I get back to my laptop, in an hour or however long it takes children to go to bed

  658. jonas’

    pep., I’m more in favour of having documents which give a rationale for the importance of a feature, instead of just relying on the MUST hammer

  659. jonas’

    MattJ, kthx

  660. jonas’

    say hello from the strange internet parrot people

  661. MattJ


  662. flow

    pep., why not simply tell the implementors that these features are important instead?

  663. pep.

    Well that's how you tell them it's important

  664. pep.

    "Ah btw that's what 90% of clients are going to use nowadays, but if you don't implement it it's fine don't worry"

  665. pep.

    Is how I see the optional thing

  666. flow

    pep., again, then why not simply tell the implementors that these feature are important to preven this?

  667. flow

    pep., again, then why not simply tell the implementors that these feature are important to prevent them from getting that impression?

  668. pep.

    Suggestion for this year's CS, add namespaces to be supported. So that people don't claim they support mam:0 (or whatever NS that nobody uses anymore) and be done with it

  669. lovetox

    and then add it to the compliance suite :D

  670. krauq has left

  671. lovetox

    if i think about thats actually what i care for

  672. krauq has joined

  673. lovetox

    i want the server to know what they should implement

  674. lovetox

    and thats my fear that optional things get only implemented if i personally bug every server dev

  675. pep.


  676. lovetox

    so yes if we have a document that tells the server dev what he should actually implement to have a relevant impl, then i guess im fine with the XEP not mandating it

  677. jonas’

    in contrast to that, I bet you’ll find every mandatory thing of '45 or '60 implemented in all servers

  678. pep.


  679. flow

    otherwhise your fear would be that that big monohilitc everything-is-mandatory spec does not get implemented at all, or only partly (which would cause a lot of "fun")

  680. pep.

    Well quite a lot of MUSTs or SHOULDs are behind optional features

  681. jonas’

    discoverable partial support is better than only finding out about lack of support the hard way

  682. pep.

    flow, it's fine if the thing is still usable

  683. pep.

    But it's useless if it isn't

  684. jonas’

    which can either be a <service-unavailable/> / <feature-not-implemented/> (if you’re lucky), or different behaviour because an element was ignored.

  685. flow

    pep., I am sorry pep, but I can't follow

  686. pep.

    "I implemented all the mandatory bits! No clients can use my implementation!"

  687. pep.

    (all and only*)

  688. stpeter has joined

  689. stpeter has left

  690. werdan has left

  691. Yagiza has left

  692. Nekit has joined

  693. werdan has joined

  694. eevvoor has joined

  695. werdan has left

  696. bear has joined

  697. bear has left

  698. Douglas Terabyte has left

  699. Douglas Terabyte has joined

  700. adiaholic_ has left

  701. adiaholic_ has joined

  702. j.r has left

  703. DebXWoody has left

  704. j.r has joined

  705. pdurbin has joined

  706. rion has left

  707. LNJ has joined

  708. pdurbin has left

  709. rion has joined

  710. bear has joined

  711. andrey.g has left

  712. bear has left

  713. jonas’

    MattJ, poke

  714. andrey.g has joined

  715. adiaholic_ has left

  716. adiaholic_ has joined

  717. mukt2 has joined

  718. Marc has left

  719. Marc has joined

  720. krauq has left

  721. krauq has joined

  722. mukt2 has left

  723. Neustradamus

    Two "Version 1.0.0 of XEP-0363" in next Newsletter

  724. bear has joined

  725. alexis has joined

  726. adiaholic_ has left

  727. alexis has left

  728. Tobias has left

  729. calvin has left

  730. alexis has joined

  731. MattJ

    jonas’, done

  732. alexis has left

  733. mukt2 has joined

  734. alexis has joined

  735. alexis has left

  736. alexis has joined

  737. robertooo has left

  738. robertooo has joined

  739. pdurbin has joined

  740. !XSF_Martin


  741. alexis has left

  742. alexis has joined

  743. robertooo has left

  744. pdurbin has left

  745. Shell has left

  746. Shell has joined

  747. emus has left

  748. goffi has left

  749. eevvoor has left

  750. Nekit has left

  751. mukt2 has left

  752. andy has left

  753. wurstsalat has left

  754. bear has left

  755. lorddavidiii has left

  756. lskdjf has left

  757. lskdjf has joined

  758. Shell has left

  759. LNJ has left

  760. Wojtek has left

  761. karoshi has left

  762. xelxebar has left

  763. bear has joined

  764. LNJ has joined