XSF Discussion - 2019-07-16


  1. madhur.garg

    hi, I am implementing Infinite scroll using MAM in Poezio for that, I have to query messages in reverse order. I am sending the query like this: https://xmpp.org/extensions/xep-0313.html#example-9 with the timestamp of the current 1st message in the chat as the end timestamp, I have one doubt with this: What value should I take for start timestamp here? (should it be some fixed timestamp like '2010-08-07T00:00:00Z' or taking it relative to end timestamp like 'end timestamp - 360 days')

  2. lovetox_

    nothing

  3. lovetox_

    start is not a mandatory field, If omitted, the server SHOULD assume the value of 'start' to be equal to the date/time of the earliest message stored in the archive.

  4. pep.

    The idea here is not to get the earliest message then

  5. pep.

    It's to get messages before a what's already in the buffer for the room

  6. lovetox_

    you do that via RSM page request

  7. lovetox_

    not via mam search limit

  8. pep.

    It's to get messages before what's already in the buffer for the room

  9. lovetox_

    first you decide in what timespan the server should prepare messages

  10. lovetox_

    these messages are not sent ! then you decide what page of that request you want

  11. lovetox_

    so yes i want everything from first message in my chat to first message ever in the archive to be prepared by the server

  12. lovetox_

    then i want the last page of that result

  13. ralphm

    Then again syncing all of history can be pretty expensive (time, bandwidth, size). What we did at first start is get the last 7 days, and then on individual chats infinite scroll back if the user wanted to go further back.

  14. lovetox_

    ralphm, this has nothing to do with syncing full history

  15. lovetox_

    only because i tell the server operate within these limits, does not mean i request all the pages until i reach the limit

  16. lovetox_

    but yeah one could specify something like (end - 2 weeks ) or something

  17. ralphm

    Oh, then I misinterpreted what you wrote.

  18. lovetox_

    the problem with dates is you dont know how many messages are within a date span

  19. ralphm

    Yes, definitely

  20. lovetox_

    i can request backwars 1 year, and there are 0 messages

  21. lovetox_

    so i request backwards until the first message in the archive, then request in 30 page junks

  22. ralphm

    The date range was for the very first sync, for all of history across contacts.

  23. lovetox_

    and abort after 100 messages

  24. ralphm

    We used size limits (I think 50 or 100) for infinite scroll.

  25. lovetox_

    yes ralph, for forwards sync

  26. lovetox_

    date range is good

  27. lovetox_

    on first start

  28. lovetox_

    i do also 7 days i think

  29. ralphm

    Had some with non-chat messages, which ties into handling of reactions, edits.

  30. ralphm

    Some issues

  31. lovetox_

    everything but mam full sync from a stanza-id forward

  32. lovetox_

    is messy

  33. ralphm

    Things like deliver/read markers, CDRs (for calls).

  34. lovetox_

    thats why i still didnt implement backwards scrolling, just to many issues that can arise from it

  35. lovetox_

    and im not sure full sync is so bad

  36. lovetox_

    because most servers have a 3 months limit on storage anyway

  37. pep.

    Tell that to my CPU :p

  38. madhur.garg

    By default, does it takes the start timestamp of the first message? (I am not sure if that is true, but will check it again) So, what should be good to go with nothing as a start date or with a date range of say 360 days?

  39. lovetox_

    i made tests with decent servers, and 3000 messages are downloaded in a matter of seconds

  40. pep.

    lovetox_, I store as much as I can on my server tbh, and I won't change that for anything

  41. lovetox_

    madhur.garg, set nothing as start, and abort after receiving X pages

  42. pep.

    lovetox_, we need something as start no? We don't start with an empty buffer

  43. lovetox_

    pep. this is no problem means on first start you have to wait 10 seconds, instead of 5

  44. pep.

    I remember dino downloading a year of private messages each time I'd reset it (testing stuff)

  45. lovetox_

    yeah so? if i present you with a loading progress i dont see a problem here

  46. lovetox_

    10.000 messages are probably 10 MB or something 😃

  47. pep.

    yeah, sure

  48. lovetox_

    pep. with backwards scrolling you want to define the end date, not start

  49. lovetox_

    you can reverse a mam request

  50. pep.

    So the end date means "From this date"?

  51. pep.

    and then backwards

  52. lovetox_

    see the mam requst like the DB query the server makes

  53. lovetox_

    start = first message of archive, end = whatever is first in your buffer

  54. lovetox_

    and start = emtpy, means first message of the archive

  55. lovetox_

    at least thats what the XEP mandates

  56. lovetox_

    thats only the result set then

  57. lovetox_

    after that you set with RSM which page from that result you want

  58. lovetox_

    obviously not the first

  59. lovetox_

    you want the last page of that db query

  60. lovetox_

    see 0059 for how to request the last page of a result set

  61. ralphm

    I'd never want to use a server that limits how much history is kept.

  62. pep.

    lovetox_, I want the last page of that query because.. it's in reverse order?

  63. lovetox_

    pep. you cant reverse a query

  64. lovetox_

    a query is always from start to end within all the messages of the archive

  65. lovetox_

    and in the order first to last message

  66. lovetox_

    if your first message in the buffer is number 10

  67. lovetox_

    then you request from the server messages 1-9, then limit that result with RSM so the server gives you the last page of that result

  68. lovetox_

    and if your page size=1

  69. lovetox_

    this means he sends you number 9

  70. lovetox_

    you successfully backscrolled !

  71. pep.

    right, ok

  72. pep.

    TIL https://xmpp.org/extensions/xep-0142.html

  73. lovetox

    wtf