madhur.garghi, I am implementing Infinite scroll using MAM in Poezio
for that, I have to query messages in reverse order.
I am sending the query like this: https://xmpp.org/extensions/xep-0313.html#example-9
with the timestamp of the current 1st message in the chat as the end timestamp, I have one doubt with this: What value should I take for start timestamp here? (should it be some fixed timestamp like '2010-08-07T00:00:00Z' or taking it relative to end timestamp like 'end timestamp - 360 days')
lovetox_start is not a mandatory field, If omitted, the server SHOULD assume the value of 'start' to be equal to the date/time of the earliest message stored in the archive.
pep.The idea here is not to get the earliest message then
pep.It's to get messages before a what's already in the buffer for the room
lovetox_you do that via RSM page request
lovetox_not via mam search limit
pep.It's to get messages before what's already in the buffer for the room
lovetox_first you decide in what timespan the server should prepare messages
lovetox_these messages are not sent ! then you decide what page of that request you want
lovetox_so yes i want everything from first message in my chat to first message ever in the archive to be prepared by the server
lovetox_then i want the last page of that result
ralphmThen again syncing all of history can be pretty expensive (time, bandwidth, size). What we did at first start is get the last 7 days, and then on individual chats infinite scroll back if the user wanted to go further back.
lovetox_ralphm, this has nothing to do with syncing full history
lovetox_only because i tell the server operate within these limits, does not mean i request all the pages until i reach the limit
lovetox_but yeah one could specify something like (end - 2 weeks ) or something
ralphmOh, then I misinterpreted what you wrote.
lovetox_the problem with dates is you dont know how many messages are within a date span
lovetox_i can request backwars 1 year, and there are 0 messages
lovetox_so i request backwards until the first message in the archive, then request in 30 page junks
ralphmThe date range was for the very first sync, for all of history across contacts.
lovetox_and abort after 100 messages
ralphmWe used size limits (I think 50 or 100) for infinite scroll.
lovetox_yes ralph, for forwards sync
lovetox_date range is good
lovetox_on first start
lovetox_i do also 7 days i think
ralphmHad some with non-chat messages, which ties into handling of reactions, edits.
lovetox_everything but mam full sync from a stanza-id forward
ralphmThings like deliver/read markers, CDRs (for calls).
lovetox_thats why i still didnt implement backwards scrolling, just to many issues that can arise from it
lovetox_and im not sure full sync is so bad
lovetox_because most servers have a 3 months limit on storage anyway
pep.Tell that to my CPU :p
madhur.gargBy default, does it takes the start timestamp of the first message?
(I am not sure if that is true, but will check it again)
So, what should be good to go with nothing as a start date or with a date range of say 360 days?
lovetox_i made tests with decent servers, and 3000 messages are downloaded in a matter of seconds
pep.lovetox_, I store as much as I can on my server tbh, and I won't change that for anything
lovetox_madhur.garg, set nothing as start, and abort after receiving X pages
pep.lovetox_, we need something as start no? We don't start with an empty buffer
lovetox_pep. this is no problem means on first start you have to wait 10 seconds, instead of 5
pep.I remember dino downloading a year of private messages each time I'd reset it (testing stuff)
lovetox_yeah so? if i present you with a loading progress i dont see a problem here
lovetox_10.000 messages are probably 10 MB or something 😃
lovetox_pep. with backwards scrolling you want to define the end date, not start
lovetox_you can reverse a mam request
pep.So the end date means "From this date"?
pep.and then backwards
lovetox_see the mam requst like the DB query the server makes
lovetox_start = first message of archive, end = whatever is first in your buffer
lovetox_and start = emtpy, means first message of the archive
lovetox_at least thats what the XEP mandates
lovetox_thats only the result set then
lovetox_after that you set with RSM which page from that result you want
lovetox_obviously not the first
lovetox_you want the last page of that db query
lovetox_see 0059 for how to request the last page of a result set
ralphmI'd never want to use a server that limits how much history is kept.
pep.lovetox_, I want the last page of that query because.. it's in reverse order?
lovetox_pep. you cant reverse a query
lovetox_a query is always from start to end within all the messages of the archive
lovetox_and in the order first to last message
lovetox_if your first message in the buffer is number 10
lovetox_then you request from the server messages 1-9, then limit that result with RSM so the server gives you the last page of that result