XSF Discussion - 2020-01-22


  1. larma has left

  2. larma has joined

  3. mukt2 has left

  4. Daniel has joined

  5. Dele (Mobile) has left

  6. pep.

    Is there a way to do with pubsub (or else?) many publishers many subscribers, but only subscribers see everything. publishers see their own items

  7. karoshi has left

  8. karoshi has joined

  9. larma has left

  10. larma has joined

  11. larma has left

  12. larma has joined

  13. Shell has joined

  14. mukt2 has joined

  15. aj has left

  16. rion has left

  17. rion has joined

  18. Daniel has left

  19. !XSF_Martin has left

  20. larma has left

  21. larma has joined

  22. !XSF_Martin has joined

  23. typikol has joined

  24. typikol has left

  25. Daniel has joined

  26. david has left

  27. david has joined

  28. aj has joined

  29. Daniel has left

  30. mukt2 has left

  31. pdurbin has joined

  32. mukt2 has joined

  33. Shell has left

  34. aj has left

  35. Shell has joined

  36. pdurbin has left

  37. david has left

  38. david has joined

  39. stpeter has left

  40. dwd

    pep., Defining fulltext search fully would mean servers would have to implement a full-text search engine entirely - it wouldn't handle, for example, stemming in a homogeneous manner, so we'd presumably have to ban that, which feels undesirable. AIUI, MattJ's suggestion is a strict substring field as well as a "magic" field. I think the threat of beer-buying is sufficient to prevent outright silliness (it also prevents anyone being silly and still claiming full conformance, BTW).

  41. emus has left

  42. sonny has joined

  43. debacle has left

  44. Daniel has joined

  45. mukt2 has left

  46. mukt2 has joined

  47. Daniel has left

  48. stpeter has joined

  49. mukt2 has left

  50. mukt2 has joined

  51. karoshi has left

  52. Daniel has joined

  53. mukt2 has left

  54. Daniel has left

  55. Daniel has joined

  56. Douglas Terabyte has left

  57. mukt2 has joined

  58. Daniel has left

  59. mukt2 has left

  60. mukt2 has joined

  61. sonny has left

  62. mukt2 has left

  63. mukt2 has joined

  64. lskdjf has left

  65. !XSF_Martin has left

  66. jubalh has left

  67. fippo has left

  68. mukt2 has left

  69. sonny has joined

  70. !XSF_Martin has joined

  71. mukt2 has joined

  72. Dele (Mobile) has joined

  73. Daniel has joined

  74. pdurbin has joined

  75. stpeter has left

  76. Douglas Terabyte has joined

  77. pdurbin has left

  78. pdurbin has joined

  79. Daniel has left

  80. Daniel has joined

  81. mukt2 has left

  82. mukt2 has joined

  83. Shell has left

  84. mukt2 has left

  85. adiaholic has left

  86. adiaholic has joined

  87. andy has joined

  88. Tobias has joined

  89. mukt2 has joined

  90. Yagiza has joined

  91. neshtaxmpp has left

  92. Daniel has left

  93. mukt2 has left

  94. stpeter has joined

  95. !XSF_Martin has left

  96. lorddavidiii has joined

  97. pdurbin has left

  98. Daniel has joined

  99. stpeter has left

  100. Daniel has left

  101. sonny has left

  102. sonny has joined

  103. waqas has left

  104. mukt2 has joined

  105. paul has joined

  106. Daniel has joined

  107. !XSF_Martin has joined

  108. mukt2 has left

  109. wurstsalat has joined

  110. Daniel has left

  111. karoshi has joined

  112. sonny has left

  113. sonny has joined

  114. Daniel has joined

  115. Daniel has left

  116. j.r has left

  117. j.r has joined

  118. sonny has left

  119. pdurbin has joined

  120. Daniel has joined

  121. !XSF_Martin has left

  122. !XSF_Martin has joined

  123. karoshi has left

  124. jubalh has joined

  125. karoshi has joined

  126. MattJ

    dwd, I don't think we have to rule out stemming

  127. MattJ

    nor mandate it

  128. MattJ

    (for the "plain" search)

  129. MattJ

    But most FTS engines provide an advanced query language, and that's mainly what I want to avoid exposing

  130. lorddavidiii has left

  131. MattJ

    But e.g. it sounded like that's what Guus was doing, it's usually the (slightly) easier option

  132. lorddavidiii has joined

  133. dwd

    Right, indeed. Your suggestion is a dumb substring search, plus magic. I'd aim for magic first, going that any query language is close to nothing. I'm thinking in terms of tsvector in pgsql, for example.

  134. dwd

    Unless I misunderstand your suggestion here.

  135. MattJ

    I'm just saying there should be two fields, plain and implementation-specific

  136. MattJ

    running with the postgres example, the plain one would use plainto_tsquery() for example

  137. sonny has joined

  138. mathijs has left

  139. mathijs has joined

  140. dwd

    But then your plain one would do stemming for example. Surely?

  141. MattJ

    Yes

  142. MattJ

    I don't see that as a problem

  143. MattJ

    It defines the semantics of the user input, not what the implementation does with that info

  144. MattJ

    "This is a query from the user with no special operators or syntax"

  145. MattJ

    Now find some messages

  146. MattJ

    Which is different to: > <Guus>  simple keywords will work, but more elaborate lucene queries too (although you'd need to know the index fields)

  147. Nekit has joined

  148. MattJ

    I'm saying we should have a way to expose the elaborate queries (if that's what deployments/implementations really want), but we should also have a safe option

  149. MattJ

    Safe in the sense that you can just throw some text in there and have a reasonable expectation it will return something useful

  150. MattJ

    This is from last year, but I just remembered it: https://opensourceconnections.com/blog/2019/05/29/falsehoods-programmers-believe-about-search/

  151. MattJ

    > When you find the boolean operator ‘OR’, you always know it doesn’t mean Oregon

  152. MattJ

    Though I think my favourite from there is: > A customer using the same query twice expects the same results for both searches

  153. jubalh has left

  154. mukt2 has joined

  155. sonny has left

  156. eevvoor has joined

  157. dwd

    Ah, I see.

  158. eevvoor has left

  159. ralphm

    Yeah, search is not trivial.

  160. lovetox has joined

  161. goffi has joined

  162. mukt2 has left

  163. sonny has joined

  164. dwd

    So if I understand correctly, MattJ is arguing that the second sentence of 3.2 of my protoxep should be in effect reversed, and servers MUST interpret any words or characters as search terms, and not treat them as directives or operators.

  165. dwd

    I can certainly go that route.

  166. MattJ

    Depends. You only specified one field, and it depends whether you specified the plain or the non-plain one :)

  167. MattJ

    Guus wants the non-plain one, and this draft was primarily for him at this point, right? :)

  168. Kev

    We already implement MAM search, FWIW, and have for years.

  169. MattJ

    But I wanted to add in a way for the server to convey some help text for the non-plain one

  170. MattJ

    Also we need to deal with localization in the various parts of this

  171. dwd

    Yay.

  172. MattJ

    and that's not easy - it's extremely likely that the server is only going to have FTS for a single locale

  173. MattJ

    But multiple is of course possible with the right setup, I just don't see many people crazy enough to dedicate the resources to that

  174. dwd

    Yeah, fine with extending my extension to introduce extensions. I was mostly in my XEP project for inbox and figured I'd knock it this once so as to have something.

  175. MattJ

    I can contribute the missing parts, I've had this in my head for quite a while, but I'm way too busy this side of FOSDEM

  176. dwd

    I'm not wed to anything here except the beers.

  177. MattJ

    "OR an orange juice"?

  178. dwd

    You don't have to drink the beer. They just have to buy it.

  179. Ge0rG

    this reminds me of how the formulas in Excel/LibreCalc are language-dependent, so if you work with multiple locales, you always get it wrong

  180. dwd

    It makes it impossible to claim they meet the standard by syntax alone.

  181. mathijs has left

  182. mathijs has joined

  183. emus has joined

  184. Guus

    Kev what fields do you use, and what functionality is behind it? My primary motivation was to re-use existing field names if possible, to have overlap.

  185. Kev

    You expect me to remember things? :o

  186. neshtaxmpp has joined

  187. jubalh has joined

  188. jubalh has left

  189. Kev

    { "search", sizeof("search")-1, XEP313_FILTER_TYPE_STRING, "If specified, only return messages that contain each of these words, in any order" }, Seems to be what we're using.

  190. jubalh has joined

  191. MattJ

    I think that's a sensible implementation of the "plain" variant

  192. jubalh has left

  193. Daniel

    that's what Conversations' local search does as well; and what I would expect a server side search to do as well

  194. jubalh has joined

  195. !XSF_Martin has left

  196. jubalh has left

  197. jubalh has joined

  198. cedric has joined

  199. !XSF_Martin has joined

  200. jonas’

    define "word"

  201. jonas’

    if I search for arc, will it return messages which contain "search"?

  202. jonas’

    will it return messages which contain "The word 'arc' may be contained or not"?

  203. jonas’

    </rambling-about-how-fts-is-not-trivial>

  204. emus has left

  205. emus has joined

  206. lorddavidiii has left

  207. dwd

    jonas’, Does it matter?

  208. jonas’

    probably not

  209. lorddavidiii has joined

  210. dwd

    jonas’, I mean, does anyone care about replicability of search results between servers? (or between the client's local archive and the server?)

  211. dwd

    Although there's an argument that it might be useful to have a MAM switch that emits only the ids and not the entire messages in case you already have the data. But that feels like an optimisation for another day.

  212. Zash

    If you already have the data, can't you just search there directly?

  213. lorddavidiii has left

  214. jonas’

    Zash, look at how slow conversations is since it got a FTS index ;)

  215. dwd

    Zash, Well, depends on how much of the data you have.

  216. lorddavidiii has joined

  217. Zash

    Also there's the thing where, given an id, it's tricky to retrieve that message

  218. dwd

    Zash, It is?

  219. Zash

    You can request messages before, or after, but not a specific message by id.

  220. dwd

    Zash, Oh. Well, that's stupid then.

  221. dwd

    Zash, So you'd have to ask for one after, then one before the result you get.

  222. Zash

    Yeah

  223. dwd

    Yeah, that's daft.

  224. Zash

    Or one before, then the one after that.

  225. Zash

    Either may or may not work depending on how many messages you have

  226. Zash

    Inb4 inventing SQL over XMPP to solve this

  227. dwd

    Zash, Ask for one before and one after concurrently.

  228. dwd

    Zash, Then it's only 2RTT to get the message you actually wanted.

  229. lorddavidiii has left

  230. mukt2 has joined

  231. lorddavidiii has joined

  232. Guus

    Unrelated question: with RSM, the direction in which you page through the resultset doesn't affect what's defined as the 'first' and 'last' element, right?

  233. Guus

    iow the order of elements on a page does not differ based on the direction that you page through the result set?

  234. Zash

    Correct

  235. Guus

    👍 thanks

  236. jubalh has left

  237. lorddavidiii has left

  238. Zash

    Which makes it funky to ORDER BY backwards, get the results from SQL backwards, then flip them and send them to the client.

  239. lorddavidiii has joined

  240. jubalh has joined

  241. jubalh has left

  242. sonny has left

  243. sonny has joined

  244. mukt2 has left

  245. lorddavidiii has left

  246. lorddavidiii has joined

  247. lorddavidiii has left

  248. lorddavidiii has joined

  249. paul has left

  250. lorddavidiii has left

  251. lorddavidiii has joined

  252. lorddavidiii has left

  253. lorddavidiii has joined

  254. lorddavidiii has left

  255. lorddavidiii has joined

  256. lorddavidiii has left

  257. lorddavidiii has joined

  258. lorddavidiii has left

  259. lorddavidiii has joined

  260. lorddavidiii has left

  261. lorddavidiii has joined

  262. lorddavidiii has left

  263. lorddavidiii has joined

  264. lorddavidiii has left

  265. lorddavidiii has joined

  266. lorddavidiii has left

  267. lorddavidiii has joined

  268. jubalh has joined

  269. flow

    Zash, Guus, I'd love to see this written down in xep313 (if it's not already).

  270. eevvoor has joined

  271. lovetox has left

  272. Zash

    Doesn't it say in RSM?

  273. flow

    ahh, if so, then i guess that is fine too

  274. lovetox has joined

  275. sonny has left

  276. sonny has joined

  277. lorddavidiii has left

  278. lorddavidiii has joined

  279. paul has joined

  280. murabito has left

  281. jubalh has left

  282. murabito has joined

  283. winfried has left

  284. winfried has joined

  285. wurstsalat has left

  286. wurstsalat has joined

  287. goffi has left

  288. goffi has joined

  289. jubalh has joined

  290. jubalh has left

  291. winfried has left

  292. matkor has joined

  293. winfried has joined

  294. winfried has left

  295. winfried has joined

  296. jubalh has joined

  297. winfried has left

  298. winfried has joined

  299. jubalh has left

  300. jubalh has joined

  301. winfried has left

  302. winfried has joined

  303. jubalh has left

  304. jubalh has joined

  305. jubalh has left

  306. winfried has left

  307. winfried has joined

  308. debacle has joined

  309. winfried has left

  310. debacle has left

  311. winfried has joined

  312. debacle has joined

  313. jubalh has joined

  314. jubalh has left

  315. jubalh has joined

  316. jubalh has left

  317. Wojtek has joined

  318. Zash

    I suppose it doesn't hurt adding some implementation note about it. Feel free to PR 😉

  319. pdurbin has left

  320. sonny has left

  321. dwd

    https://twitter.com/wire/status/1219745367475933185

  322. lovetox has left

  323. jubalh has joined

  324. winfried has left

  325. winfried has joined

  326. winfried has left

  327. winfried has joined

  328. !XSF_Martin has left

  329. !XSF_Martin has joined

  330. sonny has joined

  331. marc has left

  332. marc has joined

  333. lovetox has joined

  334. sonny has left

  335. lskdjf has joined

  336. sonny has joined

  337. govanify has joined

  338. eevvoor has left

  339. pep.

    > dwd> You don't have to drink the beer. They just have to buy it. > It makes it impossible to claim they meet the standard by syntax alone. I claim encumbrance. You don't know how easy it is for them to obtain beer :p

  340. jonas’

    I claim encumbrance. I reject supporting the beer production.

  341. pep.

    dwd: interoperability still mandates common wire format doesn't it

  342. pep.

    re MLS/wire

  343. dwd

    pep., Ah, yes. I thought it interesting primarily because Wire were pushing MLS as primary marketing. It's more or less finished, but it's got all the heavyweight cryptanalysis to go - roughly at the same stage where some early experimental deployment of TLSv1.3 was happening while till in Draft, about a year or so befroe the RFC was published.

  344. flow

    pep., MLS-interoperability across federated messaging protocols? I'd expect that to require even more than just a common wire format

  345. mukt2 has joined

  346. Zash

    A common data model at least, so you can map into whatever format

  347. sonny has left

  348. dwd

    flow, You could do text message bridging, though. Depends what the goals are.

  349. jonas’

    depends on where you draw the line around "wire" in "wire format"

  350. jonas’

    or, what Zash says

  351. flow

    I wouldn't be suprised if MLS needs to be tightly-coupled with the underlying groupchat mechanism

  352. dwd

    flow, Prepare to be surprised, then.

  353. flow

    I am prepared, can I be suprised now?

  354. sonny has joined

  355. dwd

    flow, In principle, if two members of the group attempt to commit at once it could get weird, and the DS is supposed to impose a strict ordering, but XMPP does that anyway so I don't think anything special would be needed.

  356. flow

    dwd, DS?

  357. dwd

    flow, Also, "Commit?". Easiest to skim the architecture drafts and get a feel for it.

  358. flow

    will do

  359. dwd

    I can probbaly knock together a lightingish talk at the Summit on MLS if there's interest. Not that I'm any kind of cryptographer of course.

  360. jonas’

    I’d be interested

  361. jonas’

    reminds me to put me on the list of remote attendants

  362. jonas’

    and reminds me to allocate a day off

  363. Max has left

  364. Max has joined

  365. mathijs has left

  366. mathijs has joined

  367. j.r has left

  368. j.r has joined

  369. mukt2 has left

  370. mathijs has left

  371. mathijs has joined

  372. rion has left

  373. rion has joined

  374. sonny has left

  375. mukt2 has joined

  376. sonny has joined

  377. jubalh has left

  378. jubalh has joined

  379. mukt2 has left

  380. pdurbin has joined

  381. Kev

    I think Andrew's right, we should use what's already in the most popular XMPP server (although it's 2014 it was added, not 2016) and use MAM search the way M-Link does :)

  382. pdurbin has left

  383. adiaholic has left

  384. lorddavidiii has left

  385. Zash

    Popularity contest? I object!

  386. lorddavidiii has joined

  387. dwd

    Kev, No idea what you're on about, Openfire's only just adding the feature.

  388. Guus

    I was contemplating how to put that to words, dwd .

  389. jubalh has left

  390. j.r has left

  391. mukt2 has joined

  392. j.r has joined

  393. Zash

    Excuse me, that's the weirdest spelling of Prosody I've seen yet

  394. Zash

    https://cerdale.zash.se/upload/dHpA6ZKtKtstwlTJ/bild.png

  395. winfried has left

  396. winfried has joined

  397. jonas’

    lol

  398. Guus

    to be honest, I have no clue how many instances of Openfire are running

  399. Guus

    We have download stats, and update check stats, which give some indication, but that's about it.

  400. dwd

    Lots in locked-down enterprise networks connecting to Active Directory, though.

  401. jonas’

    in the federated world, not many, I think

  402. Guus

    probably true

  403. dwd

    jonas’, Still plenty there; I think most of those doing update checks are likely to be federated.

  404. jonas’

    at least not many seen by s.j.n

  405. jonas’

    so not many hosting MUC services

  406. mukt2 has left

  407. adiaholic has joined

  408. winfried has left

  409. winfried has joined

  410. adiaholic has left

  411. !XSF_Martin has left

  412. !XSF_Martin has joined

  413. adiaholic has joined

  414. j.r has left

  415. j.r has joined

  416. j.r has left

  417. j.r has joined

  418. j.r has left

  419. calvin has joined

  420. stpeter has joined

  421. matkor has left

  422. SubPub has joined

  423. mukt2 has joined

  424. moparisthebest

    Daniel, larma, lovetox, any thoughts on a swap over to finally sending 12-byte IVs ? context: https://github.com/siacs/Conversations/issues/2578

  425. calvin has left

  426. calvin has joined

  427. MattJ

    Relevant: https://github.com/siacs/Conversations/commit/e38a9cd729bfa44d06beb44859516a1eebbb3c92

  428. MattJ

    (and https://github.com/siacs/Conversations/commit/9af056bb16d7294e427dce2d92944c4d12bd8d0f )

  429. Daniel

    it will probbaly happen with the next minor release (not bugfix)

  430. Daniel

    Siskin and profanity are 'fixed' in master

  431. Daniel

    and we will wait for them to release

  432. moparisthebest

    aw awesome, going to go ahead and comment on that issue

  433. Wojtek

    BeagleIM as well (same library as Siskin), should be released soon-ish (depends a bit on Apple)

  434. mukt2 has left

  435. !XSF_Martin has left

  436. j.r has joined

  437. j.r has left

  438. j.r has joined

  439. calvin has left

  440. !XSF_Martin has joined

  441. sonny has left

  442. govanify has left

  443. govanify has joined

  444. winfried has left

  445. winfried has joined

  446. sonny has joined

  447. winfried has left

  448. winfried has joined

  449. jonas’

    cc @ Syndace

  450. mukt2 has joined

  451. stpeter has left

  452. winfried has left

  453. winfried has joined

  454. winfried has left

  455. winfried has joined

  456. winfried has left

  457. winfried has joined

  458. jubalh has joined

  459. Syndace

    thanks jonas’, was involved in that decision

  460. jonas’

    ok

  461. mukt2 has left

  462. calvin has joined

  463. lorddavidiii has left

  464. lorddavidiii has joined

  465. jubalh has left

  466. krauq has left

  467. krauq has joined

  468. eevvoor has joined

  469. eevvoor has left

  470. stpeter has joined

  471. !XSF_Martin has left

  472. j.r has left

  473. j.r has joined

  474. !XSF_Martin has joined

  475. j.r has left

  476. j.r has joined

  477. j.r has left

  478. j.r has joined

  479. j.r has left

  480. j.r has joined

  481. lorddavidiii has left

  482. mukt2 has joined

  483. lorddavidiii has joined

  484. lorddavidiii has left

  485. lorddavidiii has joined

  486. j.r has left

  487. j.r has joined

  488. j.r has left

  489. j.r has joined

  490. sonny has left

  491. mathijs has left

  492. mathijs has joined

  493. sonny has joined

  494. alameyo has left

  495. alameyo has joined

  496. mathijs has left

  497. mathijs has joined

  498. pdurbin has joined

  499. mukt2 has left

  500. mukt2 has joined

  501. pdurbin has left

  502. vanitasvitae has left

  503. vanitasvitae has joined

  504. sonny has left

  505. mukt2 has left

  506. sonny has joined

  507. mukt2 has joined

  508. Daniel

    what's the implementation status of bookmarks 2?

  509. pep.

    After what's been done in the sprint?

  510. Daniel

    yeah probably not much

  511. pep.

    the prosody module should be working now

  512. pep.

    converts between all 3 iirc

  513. Link Mauve

    Converts from both forms of XEP-0048 to XEP-0402 format, and then lets the old form of XEP-0048 read from the same store.

  514. mukt2 has left

  515. Link Mauve

    The PEP form of XEP-0048 is only considered for migration, after which it is left unusable.

  516. Link Mauve

    This should work fine since clients can’t rely on this PEP form working when XEP-0411 isn’t advertised.

  517. mukt2 has joined

  518. Daniel

    Yes I actually think that's fine

  519. Daniel

    I know I was super eager on having migration between old pep and new pep working as well. But I don't really understand why anymore

  520. Link Mauve

    It is now working anyway. :)

  521. Link Mauve

    Migration, not concurrent usage.

  522. Daniel

    Yeah. I meant concurrent usage. But yeah it should be fine.

  523. Daniel

    You can unload the old module and then load the new and everything should be ok

  524. Link Mauve

    Yes.

  525. sonny has left

  526. Link Mauve

    The new module will refuse to get loaded if the first one is in the configuration file.

  527. Link Mauve

    (Or loaded.)

  528. Daniel

    Yeah that's cool. Yeah I would like to see a last call on that. Get some more feedback from a wider community and then deploy it.

  529. Daniel

    So for once we could actually do it properly and have a LC before deployment

  530. pep.

    What about the extensions proposal from Link Mauve btw? did that progress a bit? Maybe awaiting for a PR?

  531. Daniel

    The what now?

  532. Daniel

    The changes to the xep went through

  533. pep.

    let me grep in the list

  534. Link Mauve

    pep., which extensions proposal?

  535. pep.

    yours, to bookmarks2

  536. pep.

    For stuff like password etc., or else

  537. Link Mauve

    Ah, the have clients not throw away extensions?

  538. pep.

    yeah

  539. Link Mauve

    dwd said he was going to add that to the spec.

  540. Link Mauve

    IIRC.

  541. pep.

    ok

  542. eevvoor has joined

  543. Daniel

    I would actually be cool if we could make Draft before Berlin

  544. Link Mauve

    +1

  545. Daniel

    Then we can put the final touches on the implementations in Berlin

  546. marc0s has joined

  547. eevvoor

    at the sprint you mean Daniel?

  548. Daniel

    eevvoor: yes

  549. Ge0rG

    dwd: is Inbox a sophisticated attempt at testing how many levels deep you can nest a <message> without getting your computer taken away? ;)

  550. dwd

    That's a cruel and accurate suggestion.

  551. dwd

    Really, it's a matter of trying to reuse the result from MAM such that things like MAMFC plug into it neatly.

  552. dwd

    But it did feel a bit nesty. Might be a better way of constructing it by injecting an inbox bit inside the result, perhaps.

  553. Ge0rG

    maybe I'm just fed up with trying to read nested messages from one-liner XML dumps from my client and server logs

  554. Ge0rG

    dwd: I don't have a good idea ATM

  555. Kev

    xmllint --format became my friend years ago, and has remained so since.

  556. Kev

    Because yes, reading one-line XMPP stanzas gets worse the deeper they go.

  557. Ge0rG

    Kev: I suppose I need to add a key binding for it to my vim

  558. lovetox has left

  559. Ge0rG

    Kev: it's double nasty in clients that just dump the raw stream instead of individual stanzas, so that your grep dumps a screenful of XML and you need to find the beginning and end of things to be able to xmllint

  560. eevvoor has left

  561. pep.

    I'm not sure I understand why <entry> contains the latest message

  562. pep.

    I mean the whole message

  563. Ge0rG

    pep.: so that you can show the last message in your chat list

  564. pep.

    Are you not going to do MAM anyway right after?

  565. Kev

    No reason to.

  566. pep.

    To get more than 1 message yes

  567. Ge0rG

    pep.: you could implement a thin client that only MAMs when you open a tab

  568. Kev

    ^

  569. pep.

    Ge0rG, sure, and then I just need to do MAM when you open the tab

  570. pep.

    Because I will do MAM

  571. pep.

    What I'm interested in inbox is really just the list, because then I know what to fetch via MAM

  572. Kev

    It's fairly common when rendering an inbox (both in chat clients and elsewhere) to want to show a preview of the most recent message, so including the most recent message would achieve that (without doing 100/200/howevermany individual MAM queriest to get the latest message for each inbox entry).

  573. Kev

    So it seems useful to me.

  574. lovetox has joined

  575. lovetox has left

  576. lovetox has joined

  577. pep.

    yeah maybe.. probably something I'll have to ignore then

  578. Ge0rG

    pep.: yes.

  579. Ge0rG

    I still think that poezio should be a fat client, though ;)

  580. pep.

    Ge0rG, in any case that message is useless to me in poezio

  581. pep.

    I'll do MAM to sync up with the last known id

  582. Kev

    The whole fat client/thin client thing I think is only going to be 'resolved' by allowing for both.

  583. Ge0rG

    Kev: I agre

  584. Ge0rG

    Kev: I agree

  585. Kev

    In cases where allowing for both is going to mean lots of data being sent that one or the other doesn't want, potentially shoving a bool on a query to exclude the noise might make sense.

  586. Ge0rG

    I actually have a use-case for both. I want a "fat" poezio on my colo server, with full local logging, and a "thin" MAM-backed one on my laptop when I'm on the go

  587. Zash

    https://modules.prosody.im/mod_map.html

  588. Kev

    I don't know if that would add any value to inbox or not, but it's a possibility in general.

  589. Zash

    dwd, had you seen ↑ ?

  590. Kev

    Zash: Is that also similar to the unread stuff in bind2?

  591. pep.

    Ge0rG, both can use MAM

  592. Ge0rG

    pep.: sure, but in different ways

  593. Zash

    Kev, yes, it's inspired by that example in bind2

  594. Ge0rG

    pep.: I want my fat client to do a full MAM sync on startup, and then no more MAM

  595. Kev

    Where inbox is also related to the unread stuff in bind2 (but none of them quite the same)

  596. pep.

    Ge0rG, when joining a new channel

  597. Ge0rG

    startup = new session

  598. Ge0rG

    pep.: history fetch is often good enough, but yeah, okay

  599. Kev

    Zash: I wonder if there's a race there, by not doing it during bind, but it looks useful.

  600. Kev

    Zash: Submit a protoxep?

  601. sonny has joined

  602. Kev

    I do think that server-side tracking of unread per-contact is practically needed, which that doesn't quite do, so it's not a whole solution, I think, but is moving in that direction.

  603. Zash

    Kev: It's mostly done like that to allow easy testing since I don't have bind2 yet.

  604. Kev

    Yeah, that one's a bit of an issue :)

  605. krauq has left

  606. Ge0rG

    as is IM2?

  607. edhelas has left

  608. Zash

    Everything2

  609. edhelas has joined

  610. Zash

    XMPP 2.0

  611. waqas has joined

  612. krauq has joined

  613. winfried has left

  614. winfried has joined

  615. winfried has left

  616. winfried has joined

  617. sonny has left

  618. sonny has joined

  619. dwd

    Zash, I had seen it, but then forgotten about it.

  620. dwd

    pep., And yes, you might not always want the entire message, and instead just know there is one with a particular id. Or you might not need inbox at all if you're going to pull the entire MAM archive across anyway.

  621. dwd

    pep., But lots of existing clients like to list out the conversations, and show a previewish thing of the last message. It's why, for example, Instagram's direct message inbox works in exactly this way.

  622. neshtaxmpp has left

  623. dwd

    pep., We have bigger challenges because we have lots of different styles of client in the XMPP world, and need to cater for them all efficiently without precluding any. I'm not trying to claim this is a finished design suitable for all cases.

  624. Ge0rG

    but it's a very good start

  625. pdurbin has joined

  626. Ge0rG

    dwd: I think it's missing a notion of "open conversations", which is a good thing to keep around in just this place

  627. sonny has left

  628. pep.

    dwd, my goal is not to pull the entire MAM history

  629. pep.

    At least not at first

  630. pep.

    My goal for the inbox thing is really just to get a list of JIDs to fetch MAM for. If I don't have that then I have to fetch then entire history to know who talked to me as there might be JIDs I don't know of (not MUCs nor roster)

  631. pdurbin has left

  632. Ge0rG

    pep.: how is having the last message in the response harmful to that?

  633. pep.

    Ok it may seem I'm still ranting about that, I'm not

  634. Zash

    Timestamp and body of last message per contact gets you most of the data you'd need to show a list of recent conversations and can be done with simple MAM. Read status needs more tracking than what at least Prosody has

  635. pep.

    Zash, that's the thing, you might not be talking only to contacts

  636. Zash

    s/contact/"with" in MAM terms/

  637. pep.

    yeah but you need to know who, which is why I like inbox

  638. Zash

    That MAP thing did that iirc. Wanna be convinced to convert it into mod_inbox? :)

  639. SubPub has left

  640. reedhhw has left

  641. andrey.g has left

  642. sonny has joined

  643. sonny has left

  644. matkor has joined

  645. jubalh has joined

  646. Zash

    Do we need some XPath-ish MAM search thing like the other example of extended search forms?

  647. pep.

    would it be possible to make that message optional maybe?

  648. pep.

    dwd, ^

  649. eevvoor has joined

  650. dwd

    Sure.

  651. !XSF_Martin has left

  652. dwd

    Zash, XPath-based MAM search? Yuck.

  653. Ge0rG

    pep.: what's your goal with that?

  654. pep.

    Ge0rG, why are you fighting it that much? That message is not needed in there all the time :x

  655. Ge0rG

    pep.: I'm not fighting, I'm curious. Every boolean options doubles the number of states you create and have to debug

  656. pep.

    we're at the protocol level still, I think we can live with one or two more options. We're not doing client UX

  657. sonny has joined

  658. Ge0rG

    pep.: please tell me why that Carbon message isn't displayed on my desktop client.

  659. Ge0rG

    (yes, this is a protocol question. More than UX at least)

  660. pep.

    even if that makes things more complex I'm of the opinion that I should be able to choose. if we do one-size-fits-all nobody is going to be happy, or rather, only the golden use case is going to be happy and that's annoying for everybody else

  661. sonny has left

  662. krauq has left

  663. krauq has joined

  664. winfried has left

  665. winfried has joined

  666. Daniel

    So you want to make it optional to request or optional to generate?

  667. Daniel

    Because making it optional to generate would be bad

  668. pep.

    how bad?

  669. pep.

    I was mostly thinking "I don't need it, the server doesn't need to send it". Whether it generates it or not (or stores it as is) it's not my problem

  670. !XSF_Martin has joined

  671. jubalh has left

  672. jubalh has joined

  673. Wojtek has left

  674. MattJ

    What about deployments without MAM? (e.g. for privacy or resource constraint reasons)

  675. waqas has left

  676. mukt2 has left

  677. pep.

    with offline messages?

  678. waqas has joined

  679. pep.

    In any case if the server doesn't keep messages, then it doesn't make sense indeed to force it to return the last one

  680. andrey.g has joined

  681. mukt2 has joined

  682. MattJ

    My point is mainly that you may want to support inbox on a server that doesn't store messages. It seems to me it would be easier for client devs to deal with no message than no inbox

  683. MattJ

    Er, I think I'd be fine with "if the server/user has a MAM archive enabled, you must do this"

  684. emus has left

  685. MattJ

    Just not with making a hard dependency from Inbox to MAM for deployments that don't want that

  686. emus has joined

  687. MattJ

    For something that is ultimately a convenience/optimisation feature

  688. mukt2 has left

  689. mukt2 has joined

  690. winfried has left

  691. Ge0rG

    So such a server would always return an empty list?

  692. winfried has joined

  693. MattJ

    Ge0rG: I think <entry/> would just be empty

  694. sonny has joined

  695. Ge0rG

    MattJ: what JIDs would you list?

  696. MattJ

    (which is already valid per the XEP)

  697. MattJ

    I guess the XEP doesn't really specify what JIDs are in the list

  698. mukt2 has left

  699. MattJ

    With previous PEP-based proposals that was trivial, and the list is a list of open tabs/chats

  700. pep.

    yeah I also liked that

  701. MattJ

    Opening/closing a chat cleanly mapped to adding/removing from the list

  702. MattJ

    Now it's a bit ambiguous

  703. pdurbin has joined

  704. mukt2 has joined

  705. Ge0rG

    May be it can be resolved by adding another flag to the inbox, one that reflects an open chat and is sticky even when there are no unread messages?

  706. debacle has left

  707. pep.

    MattJ, I'd want a per-client list though :x (or profiles or whatever. I guess per-client is already good)

  708. j.r has left

  709. Ge0rG

    pep.: if you have it per client, you don't need to sync it to the server

  710. j.r has joined

  711. pep.

    Ge0rG, maybe it's not just a dumb list in PEP that I want. I also do want to know if MAM/offline stuff that somebody that's not in my roster talked to me and that I need to do MAM with it

  712. pep.

    if I don't have that information right away, I need to fetch the world and I want to avoid that

  713. pdurbin has left

  714. Ge0rG

    pep.: do you want that for all remote JIDs or just the ones that your client hasn't heard from or the ones that are new since the last MAM fetch from any of your clients?

  715. Ge0rG

    I'm trying to determine which sets of information we need for the different use cases and how they overlap

  716. sonny has left

  717. mukt2 has left

  718. Nekit has left

  719. mukt2 has joined

  720. jubalh has left

  721. karoshi has left

  722. MattJ

    Why not a special PEP node plus an iq that performs a query basically equivalent to what Dave's proposal has

  723. MattJ

    For the set of JIDs currently stored

  724. MattJ

    ......plus unreads??

  725. arc has joined

  726. Ge0rG

    Why not have both in the same IQ?

  727. a has left

  728. a has joined

  729. cedric has left

  730. waqas has left

  731. MattJ

    Both what?

  732. neshtaxmpp has joined

  733. Ge0rG

    Both the open tabs and the inbox

  734. mukt2 has left

  735. Yagiza has left

  736. eevvoor has left

  737. Tobias has left

  738. mathijs has left

  739. mathijs has joined

  740. calvin has left

  741. mukt2 has joined

  742. arc has left

  743. arc has joined

  744. arc has left

  745. arc has joined

  746. wurstsalat has left

  747. j.r has left

  748. j.r has joined

  749. lorddavidiii has left

  750. MattJ

    That's basically what I'm proposing, yes

  751. jubalh has joined

  752. waqas has joined

  753. adiaholic has left

  754. adiaholic has joined

  755. arc has left

  756. arc has joined

  757. Ge0rG

    So we were misunderstanding each other all the time? Because that's what I wanted all along as well

  758. Syndace has left

  759. Syndace has joined

  760. MattJ

    Dave's current proposal appears to me to not define any logic around which JIDs should be included in the result

  761. goffi has left

  762. MattJ

    I'm suggesting we merge the old PEP inbox proposal, and use that as the list of JIDs, plus include any others that have pending unread messages

  763. MattJ

    So you have a single query for all "open" chats and unread messages

  764. MattJ

    I think it's similar to what you/someone suggested earlier about a sticky bit on the JIDs, it's just not clear to me how that would get set, how notifications would get broadcast to other clients on update, etc.

  765. MattJ

    I think PEP is a good mechanism for that part

  766. sonny has joined

  767. MattJ

    And that solves my issue too... clients/servers without MAM can still "implement" the open chats part (PEP) without needing to implement the magic query

  768. Daniel has left

  769. Daniel has joined

  770. !XSF_Martin has left

  771. mukt2 has left

  772. gav has joined

  773. !XSF_Martin has joined

  774. mukt2 has joined

  775. emus has left

  776. pdurbin has joined

  777. emus has joined

  778. pdurbin has left

  779. pep.

    Ge0rG, I still want a list of jids (1:1/muc/whatever), but since I'll most likely want a different one per client I can indeed implement it locally, and also I want to know who I have to fetch when I was offline, without having to sync the world

  780. sonny has left

  781. pep.

    Ge0rG, I still want a list of open tabs (1:1/muc/whatever), but since I'll most likely want a different one per client I can indeed implement it locally, and also I want to know who I have to fetch when I was offline, without having to sync the world

  782. debacle has joined

  783. pep.

    Ge0rG, I want a list of open tabs (1:1/muc/whatever), but since I'll most likely want a different one per client I can indeed implement it locally, and also I want to know who I have to fetch when I was offline, without having to sync the world

  784. Daniel has left

  785. krauq has left

  786. calvin has joined

  787. andy has left

  788. andy has joined

  789. Daniel has joined

  790. Daniel has left

  791. pep.

    Also.. the last message is probably not useful for some e2ee mechanisms (PFS).

  792. pep.

    Ah nevermind.

  793. pep.

    That would be an unread message :-°

  794. jubalh has left

  795. emus has left

  796. mukt2 has left

  797. mukt2 has joined

  798. lovetox has left

  799. paul has left

  800. mukt2 has left

  801. mukt2 has joined

  802. winfried has left

  803. winfried has joined

  804. winfried has left

  805. winfried has joined

  806. marc0s has left

  807. marc0s has joined

  808. winfried has left

  809. winfried has joined

  810. winfried has left

  811. winfried has joined

  812. mukt2 has left

  813. !XSF_Martin has left

  814. Daniel has joined