XSF Discussion - 2019-07-16

hi, I am implementing Infinite scroll using MAM in Poezio for that, I have to query messages in reverse order. I am sending the query like this: https://xmpp.org/extensions/xep-0313.html#example-9 with the timestamp of the current 1st message in the chat as the end timestamp, I have one doubt with this: What value should I take for start timestamp here? (should it be some fixed timestamp like '2010-08-07T00:00:00Z' or taking it relative to end timestamp like 'end timestamp - 360 days')

nothing

start is not a mandatory field, If omitted, the server SHOULD assume the value of 'start' to be equal to the date/time of the earliest message stored in the archive.

The idea here is not to get the earliest message then

you do that via RSM page request

not via mam search limit

It's to get messages before what's already in the buffer for the room ✏

first you decide in what timespan the server should prepare messages

these messages are not sent ! then you decide what page of that request you want

so yes i want everything from first message in my chat to first message ever in the archive to be prepared by the server

then i want the last page of that result

Then again syncing all of history can be pretty expensive (time, bandwidth, size). What we did at first start is get the last 7 days, and then on individual chats infinite scroll back if the user wanted to go further back.

ralphm, this has nothing to do with syncing full history

only because i tell the server operate within these limits, does not mean i request all the pages until i reach the limit

but yeah one could specify something like (end - 2 weeks ) or something

Oh, then I misinterpreted what you wrote.

the problem with dates is you dont know how many messages are within a date span

Yes, definitely

i can request backwars 1 year, and there are 0 messages

so i request backwards until the first message in the archive, then request in 30 page junks

The date range was for the very first sync, for all of history across contacts.

and abort after 100 messages

We used size limits (I think 50 or 100) for infinite scroll.

yes ralph, for forwards sync

date range is good

on first start

i do also 7 days i think

Had some with non-chat messages, which ties into handling of reactions, edits.

Some issues

everything but mam full sync from a stanza-id forward

is messy

Things like deliver/read markers, CDRs (for calls).

thats why i still didnt implement backwards scrolling, just to many issues that can arise from it

and im not sure full sync is so bad

because most servers have a 3 months limit on storage anyway

Tell that to my CPU :p

By default, does it takes the start timestamp of the first message? (I am not sure if that is true, but will check it again) So, what should be good to go with nothing as a start date or with a date range of say 360 days?

i made tests with decent servers, and 3000 messages are downloaded in a matter of seconds

lovetox_, I store as much as I can on my server tbh, and I won't change that for anything

madhur.garg, set nothing as start, and abort after receiving X pages

lovetox_, we need something as start no? We don't start with an empty buffer

pep. this is no problem means on first start you have to wait 10 seconds, instead of 5

I remember dino downloading a year of private messages each time I'd reset it (testing stuff)

yeah so? if i present you with a loading progress i dont see a problem here

10.000 messages are probably 10 MB or something 😃

yeah, sure

pep. with backwards scrolling you want to define the end date, not start

you can reverse a mam request

So the end date means "From this date"?

and then backwards

see the mam requst like the DB query the server makes

start = first message of archive, end = whatever is first in your buffer

and start = emtpy, means first message of the archive

at least thats what the XEP mandates

thats only the result set then

after that you set with RSM which page from that result you want

obviously not the first

you want the last page of that db query

see 0059 for how to request the last page of a result set

I'd never want to use a server that limits how much history is kept.

lovetox_, I want the last page of that query because.. it's in reverse order?

pep. you cant reverse a query

a query is always from start to end within all the messages of the archive

and in the order first to last message

if your first message in the buffer is number 10

then you request from the server messages 1-9, then limit that result with RSM so the server gives you the last page of that result

and if your page size=1

this means he sends you number 9

you successfully backscrolled !

right, ok

TIL https://xmpp.org/extensions/xep-0142.html

wtf