-
madhur.garg
hi, I am implementing Infinite scroll using MAM in Poezio for that, I have to query messages in reverse order. I am sending the query like this: https://xmpp.org/extensions/xep-0313.html#example-9 with the timestamp of the current 1st message in the chat as the end timestamp, I have one doubt with this: What value should I take for start timestamp here? (should it be some fixed timestamp like '2010-08-07T00:00:00Z' or taking it relative to end timestamp like 'end timestamp - 360 days')
-
lovetox_
nothing
-
lovetox_
start is not a mandatory field, If omitted, the server SHOULD assume the value of 'start' to be equal to the date/time of the earliest message stored in the archive.
-
pep.
The idea here is not to get the earliest message then
-
pep.
It's to get messages before a what's already in the buffer for the room✎ -
lovetox_
you do that via RSM page request
-
lovetox_
not via mam search limit
-
pep.
It's to get messages before what's already in the buffer for the room ✏
-
lovetox_
first you decide in what timespan the server should prepare messages
-
lovetox_
these messages are not sent ! then you decide what page of that request you want
-
lovetox_
so yes i want everything from first message in my chat to first message ever in the archive to be prepared by the server
-
lovetox_
then i want the last page of that result
-
ralphm
Then again syncing all of history can be pretty expensive (time, bandwidth, size). What we did at first start is get the last 7 days, and then on individual chats infinite scroll back if the user wanted to go further back.
-
lovetox_
ralphm, this has nothing to do with syncing full history
-
lovetox_
only because i tell the server operate within these limits, does not mean i request all the pages until i reach the limit
-
lovetox_
but yeah one could specify something like (end - 2 weeks ) or something
-
ralphm
Oh, then I misinterpreted what you wrote.
-
lovetox_
the problem with dates is you dont know how many messages are within a date span
-
ralphm
Yes, definitely
-
lovetox_
i can request backwars 1 year, and there are 0 messages
-
lovetox_
so i request backwards until the first message in the archive, then request in 30 page junks
-
ralphm
The date range was for the very first sync, for all of history across contacts.
-
lovetox_
and abort after 100 messages
-
ralphm
We used size limits (I think 50 or 100) for infinite scroll.
-
lovetox_
yes ralph, for forwards sync
-
lovetox_
date range is good
-
lovetox_
on first start
-
lovetox_
i do also 7 days i think
-
ralphm
Had some with non-chat messages, which ties into handling of reactions, edits.
-
ralphm
Some issues
-
lovetox_
everything but mam full sync from a stanza-id forward
-
lovetox_
is messy
-
ralphm
Things like deliver/read markers, CDRs (for calls).
-
lovetox_
thats why i still didnt implement backwards scrolling, just to many issues that can arise from it
-
lovetox_
and im not sure full sync is so bad
-
lovetox_
because most servers have a 3 months limit on storage anyway
-
pep.
Tell that to my CPU :p
-
madhur.garg
By default, does it takes the start timestamp of the first message? (I am not sure if that is true, but will check it again) So, what should be good to go with nothing as a start date or with a date range of say 360 days?
-
lovetox_
i made tests with decent servers, and 3000 messages are downloaded in a matter of seconds
-
pep.
lovetox_, I store as much as I can on my server tbh, and I won't change that for anything
-
lovetox_
madhur.garg, set nothing as start, and abort after receiving X pages
-
pep.
lovetox_, we need something as start no? We don't start with an empty buffer
-
lovetox_
pep. this is no problem means on first start you have to wait 10 seconds, instead of 5
-
pep.
I remember dino downloading a year of private messages each time I'd reset it (testing stuff)
-
lovetox_
yeah so? if i present you with a loading progress i dont see a problem here
-
lovetox_
10.000 messages are probably 10 MB or something 😃
-
pep.
yeah, sure
-
lovetox_
pep. with backwards scrolling you want to define the end date, not start
-
lovetox_
you can reverse a mam request
-
pep.
So the end date means "From this date"?
-
pep.
and then backwards
-
lovetox_
see the mam requst like the DB query the server makes
-
lovetox_
start = first message of archive, end = whatever is first in your buffer
-
lovetox_
and start = emtpy, means first message of the archive
-
lovetox_
at least thats what the XEP mandates
-
lovetox_
thats only the result set then
-
lovetox_
after that you set with RSM which page from that result you want
-
lovetox_
obviously not the first
-
lovetox_
you want the last page of that db query
-
lovetox_
see 0059 for how to request the last page of a result set
-
ralphm
I'd never want to use a server that limits how much history is kept.
-
pep.
lovetox_, I want the last page of that query because.. it's in reverse order?
-
lovetox_
pep. you cant reverse a query
-
lovetox_
a query is always from start to end within all the messages of the archive
-
lovetox_
and in the order first to last message
-
lovetox_
if your first message in the buffer is number 10
-
lovetox_
then you request from the server messages 1-9, then limit that result with RSM so the server gives you the last page of that result
-
lovetox_
and if your page size=1
-
lovetox_
this means he sends you number 9
-
lovetox_
you successfully backscrolled !
-
pep.
right, ok
-
pep.
TIL https://xmpp.org/extensions/xep-0142.html
-
lovetox
wtf