XSF Discussion - 2024-08-31

https://draugr.de/upload/00f530bb49b4e1685114d28f8bb6fb44b2fcbc2f/HUuVyRdffeTF38pw6YCpMUYQEaUDVL8Yr8KWKVfD/Screenshot_2024-08-31-04-53-25-991_com.cheogram.android.jpg

Another day of xmpp losing messages with no indication. Not the first time I'm basically doing "manual acks". You know something is deeply screwed up when the user has to do acks...

I wonder if, given the amount of xml bloat, just screencasting would not only be more reliable but also more effecient?... Because you end up sending screenshots around to tell what was lost or not anyway... ✏

kurisu: I'm convinced you just will these issues into existence with your endless roping

kurisu: does your phone go offline for extended periods of time (hours) ?

No. If it did, it wouldn't justify this one bit. The most funny part: I did get those messages on my pc which offline at the time. I don't know if it was all of them because you can never know with xmpp.

Maybe start debugging rather than ranting. Check logs, servers, server software, clients and so on.

kurisu: no one is trying to justify. They're trying to figure out where this bug is and what is triggering it

I get a similar bug in my android emulator sometimes, but it is caused by closing the emulator for weeks and re-opening which restores the ram/app to exactly previous state and it doesn't know it was ever offline. I'm not sure why this causes such a problem but also it's not something a real phone can do so I haven't worried too much about that one

singpolyma, I wonder if there are ROMs which SIGSTOP android apps instead of killing them

to save battery

that would have a similar effect as you freezing the emulator, right?

Hmm. If it just did STOP and then CONT yes that's true it might

with RAM being cheap these days, I can see how ROMs could reach such a point

then again, I'd also expect a lot of other stuff to break if they did

It would have to do it to the foreground service as well I expect, since that's what processes messages

kurisu is on lovely MIUI so it wouldn't surprise me

those run in the same process anyway, don't they

Hmm. I don't know but since the service stays running when the app is killed I'd think not?

you might also achieve that with threads and control over the (java) runtime, but you're probably right

Given it's a recurrent issue over the years and across different clients, this seems more like some fundamental misdesign than an isolated bug.

You mean at a protocol level? There's definitely nothing at a protocol level that could cause this

I never encountered this issue anywhere else (a piece of chat history just randomly going missing) but this is by far not the first time on jabber. Funny how even some in-platform like e.g. on a social network won't have this issue, yet something specializing on instant messaging specifically does.

> You mean at a protocol level? There's definitely nothing at a protocol level that could cause this I don't think this statement is provable and practice shows otherwise. Maybe if the connection is impeccable and nothing goes down the protocol wouldn't cause this in itself... Maybe. ↺

The protocol allows you to fetch the whole history again every time you connect if you like. So if the server has the message you'll definitely see it in the protocol

… and if you're running on a server or client w/o MAM, carbons, or stream management, then yeah, that's what you get.

kurisu: what are your intentions with sharing these findings here?

You're not providing enough information for anyone to realistically be able to solve the problem that you're experiencing.

> kurisu: what are your intentions with sharing these findings here? It would seem the issue of xmpp losing messages would be relevant here. ↺

> You're not providing enough information for anyone to realistically be able to solve the problem that you're experiencing. What information should I provide? "The problem you're experiencing" like it's just me and it's not a recurrent issue. ↺

I haven't heard of anyone actually losing messages in the past couple years besides you and weird OMEMO "cannot decrypt" issues.

(well, and pidgin users. but they are in the "no MAM/Carbons/SM" category and hence don't count)

You could investigate logs of your clients and servers. Try to document a path to reliably reproduce the issue, engage with the developers of the software that you're using, etc, etc.

From what I see, you mostly join this room to complain that "it's not working again." I'm not sure how that is going to affect anything other than annoyance with others.

Is it just me or is carbons even being needed an indicator of pure insanity? Just reading the introduction for that xep is enough to get an idea of how fragile everything is. Then you kinda stop wondering why it loses messages and begin wondering why it works most of the time...

kurisu, look, if you don't like it, stop using it.

either provide actionable information or kindly refrain from bringing the negativity in here

I'd be happy for you to propose a suitable alternative.

> You could investigate logs of your clients and servers. Try to document a path to reliably reproduce the issue, engage with the developers of the software that you're using, etc, etc. This never popped up on any other platform. Maybe the approach it is using to begin with is bad? Maybe that should be changed instead of debugging every edge case ↺

> I get a similar bug in my android emulator sometimes, but it is caused by closing the emulator for weeks and re-opening which restores the ram/app to exactly previous state and it doesn't know it was ever offline. I'm not sure why this causes such a problem but also it's not something a real phone can do so I haven't worried too much about that one Conversations only requests the last 750 messages from mam, that can cause "mam holes" in busy channels, like the one in the screenshot above. That could be why it's recurrent a recurrent problem for them. Also, something I'm not sure of: fetching the 750 messages takes some time. If someone sends a message while you're still syncing history, I think that stops the syncing ↺

> This never popped up on any other platform. Maybe the approach it is using to begin with is bad? Maybe that should be changed instead of debugging every edge case Feel free to propose an alternative approach ↺

Receiving live during sync definitely should not affect anything. If it did that would be a major bug

The 750 thing is new info to me I will look into it

> Conversations only requests the last 750 messages from mam This already sounds like a problem > Holes That is most definitely a bug but probably not even considered one.

> The 750 thing is new info to me I will look into it It's hardcoded in Conversations. I think a page is 50 messages and it requests 25 pages. Ideally, the client would have setting that lets you choose how much you want to sync (like in Gajim: sync 1 day, 7 days, 1 month, all history, never) But these only concern public channels ↺

>ideally No, this isn't ideal. This is awful. Where have you seen this? An awful, awful proposition. It shouldnt be "Choose how many messages you want me to try fetching before you get holes in your message history", it should be "message history holes are not a thing". ✏

> > Conversations only requests the last 750 messages from mam > This already sounds like a problem > > Holes > That is most definitely a bug but probably not even considered one. The idea is: if your phone stays offline for a couple of weeks / months, and you login, you don't want to get thousands of messages. The problem with Conversations is that it requests the 750 messages following the last message you have on your device, instead of requesting the last 750 messages on the server ↺

That makes the "holes" more apparent

It would seem even 750 is too many to eagerly fetch. On a mobile screen you hardly ever get more than 10-20 at once I think...

Another related issue is the amount of time and bandwidth needed to regain stable connection after being offline a long time.

I am currently making a toy client and I wanted to implement endless scrolling with lazy loading. Turned out tricky and messy because of the lack of sortable mam ids. And also it turned out jabber servers configure such absurdly low speeds you actually do want to fetch a lot of messages in advance before the user scrolls up anywhere near them, because otherwise the wait is going to be annoying.

And I can't prove it but some mucs don't keep as many messages, or reboot frequently, and clients may or not reconnect automatically.

But generally what I do is: fetch next N messages whenever there are less than X pixels left in the area being scrolled above the topmost visible pixel. This seems reasonable to me.

So when scrolling up the user should never get mam holes.

Not just "fetch X last messages"

> So when scrolling up the user should never get mam holes. "mam holes" never happen when you scroll up to fetch more history.

kurisu: xmpp is not a platform. All which you experience is always either the client/server developer made an error or a decision to not do something.

Now be my guest and please name a platform that is federated has multiple server impl., has hundreds of different client impl. And you never experience any bugs on any of them.

> Is it just me or is carbons even being needed an indicator of pure insanity? > Just reading the introduction for that xep is enough to get an idea of how fragile everything is. Then you kinda stop wondering why it loses messages and begin wondering why it works most of the time... kurisu: what? Carbons is a feature not a "patch to address fragility" or whatever you just said so none of this makes sense, what are you talking about? ↺

Carbons has nothing to do with message delivery reliability in other words

It sort of can depending how you look at it. But not for MUC obviously

> So when scrolling up the user should never get mam holes. > > Not just "fetch X last messages" The hole is from what happens between _scroll up to fetch messages_ and the messages you already have

And you can obviously have multiple of those

Has nothing to do with mam lacking sortable ids

And obviously that's not a bug right

You could potentially discard old messages and not have that problem

But for a traffic constraint mobile client that's not ideal

> And obviously that's not a bug right I'm just describing the problem

>> Is it just me or is carbons even being needed an indicator of pure insanity? >> Just reading the introduction for that xep is enough to get an idea of how fragile everything is. Then you kinda stop wondering why it loses messages and begin wondering why it works most of the time... > kurisu: what? Carbons is a feature not a "patch to address fragility" or whatever you just said so none of this makes sense, what are you talking about? Just read the xep introduction ↺

Your complaint seems to be about a muc channel so carbons isn't really relevant anyway ✏

I'm not saying they are

kurisu: what's crazy about the carbons introduction exactly?

I mean since you seem to be eager to do better you can implemted im2 and provide feedback

It explains how server behaviour is basically undefined and how the xep exists to overcome that...

kurisu: no, it's defined in the RFC. My impression is it described some pre-xep hacks some servers tried

> I mean since you seem to be eager to do better you can implemted im2 and provide feedback You mean like matrix or simplex? ↺

I didn't mean that. But if you want to sure

Someone post that "xmpp sucks - I know" meme again. Not sure what point kurisu is trying to make

Turns out kurisu was just banned from that MUC during that period...

https://www.moparisthebest.com/images/xmpp-vs-matrix.jpg

> Turns out kurisu was just banned from that MUC during that period... That means nothing because the messages were retrieved on one client and not the other, which already undefined behaviour basically but that's nothing new for xmpp. And nothing ever even indicated I was banned. ↺

>> Turns out kurisu was just banned from that MUC during that period... > That means nothing because the messages were retrieved on one client and not the other, which already undefined behaviour basically but that's nothing new for xmpp. And nothing ever even indicated I was banned. kurisu: if you could clearly explain what happened and what you expect to have happened we could probably help as-is it's just an endless rant of "XMPP loses messages and everyone is fine with it and nothing else loses messages" which is just useless. To be clear there is no scenario where a modern XMPP client would lose messages, and if one was found, that'd be considered a bug to fix. ↺

I could not clearly explain that before because it was never made clear or even hinted to me that I was banned.

And as if that is even supposed to be relevant

> To be clear there is no scenario where a modern XMPP client would lose messages, and if one was found, that'd be considered a bug to fix. I have encountered those scenarios many times. Even one would be too many...

> And as if that is even supposed to be relevant Why would that not be relevant ↺

>> To be clear there is no scenario where a modern XMPP client would lose messages, and if one was found, that'd be considered a bug to fix. > I have encountered those scenarios many times. Even one would be too many... kurisu: with your custom client? Sounds like a bug ↺

Check logs and stuff. As is what are you trying to accomplish by endlessly roping here? Do you expect us to just jump ship to another protocol because you experience a bunch of issues we don't? Or what

Missing messages I can't say I've ever experienced, delayed sometimes, cannot decrypt sometimes. Bugs exist of course

If you're saying you'd like to see the UI for indicating that you were banned during some period improved, that's at least an actionable criticism

I just wonder why this is all in the xsf channel about l every two weeks and not the specific client channel.

Menel, ever wondered why client problems are only ever reported in server rooms? And server problems in client rooms? :)

Muwhaha we have one room for both our client and server

Guus, so that's where protocol issues are reported then?

he thinks xmpp is a platform, and this is the official platform channel :D ✏

> Guus, so that's where protocol issues are reported then? Those are features, not issues.

> https://www.moparisthebest.com/images/xmpp-vs-matrix.jpg I don't know about the first part, but the second part is definitely not the reaction you get lol ↺

You get the reaction you get because you are not solution oriented, you tell us things that everybody knows already, there is no need to tell us what is bad in XMPP, most people here know exactly what is bad. You can ask how to work around the bad stuff, then you get useful help, or you try to improve specs or write new one, then you also get help. You act like someone here has the responsibility to fix something. ✏

guus... like gus fring? 😮

Jesse we have to parse xml

"Say my namespace"