jdev - 2025-05-27

> In Gajim everything is in one single thread, UI has high prio, network lower prio, if there are CPU intensive ui operations, pulling from the network simply gets Stalled a bit gajim also basically locks up if any of the rooms im in get a spammer ↺

and completely shits itself if i recieve a large message

and mam sync causes the entire client to be unusable until it finishes

> You can't fire that off to independent workers, one who writes to the database and one who sends across the network, because the network might be down or the process might be killed after network send and before database write succeeds you have to handle receipt of messages too, no? ↺

goroutines are the only form of asynchronous operation that go offers, so if i want to do any two things at once i have to at least use one

like holding open the xmpp connection is a blocking operation, and so is showing the ui window, and to do both you have to move at least the connection to a separate routine

i suppose you could do db from the xmpp connection thread, but that doesnt make much sense when you dont immediately have to care if it finished writing

> and mam sync causes the entire client to be unusable until it finishes mind you this is running on a mobile ryzen 7, 64gb of ram, nvme ssd, etc. my laptop is not the issue aside from i guess not having the best single threaded performance (hence my point) ↺

Those likely aren't structure problems though, they are just python is slow as molasses problems

But also don't confuse concurrent with parallel, you don't need multiple threads to avoid blocking and do concurrent programming, brief explanation here https://doc.rust-lang.org/book/ch16-00-concurrency.html

im running late for a thing so ill have to read it later, but dont worry im not confusing them, goroutines are just genuinely the tool to acomplish both of those

youre meant to use goroutines similar to asyncio in python or async functions in js, its just that the runtime _can_ use a separate thread so you do have to use threadsafe datatypes

sounds like a poor abstraction then

i genuinely dont understand the argument at this point

What argument?

> I was about to go into the moxxy developers chatroom to ask the dev about something in the SDK (since that's the SDK I'm using), and uhh apparently nothing at moxxy.org exists anymore... muc doesn't resolve, and neither does the website Same from here, whois says it expires July 29th this year ↺

I was about to go into the moxxy developers chatroom to ask the dev about something in the SDK (since that's the SDK I'm using), and uhh apparently nothing at moxxy.org exists anymore... muc doesn't resolve, and neither does the website. Apparently something happened to the domain 2 days ago but I'm not sure what; the SDK hasn't been updated in 6 months, though the dev seems to be actively working on other things adjacent to XMPP so... very strange ✏

> Same from here, whois says it expires July 29th this year a couple whois sites say it was "updated" on May 24th of this year, but I'm not sure what the update was. In any case, DNS doesn't resolve anymore... ↺

> and mam sync causes the entire client to be unusable until it finishes Is it because of blocking though? Dino is also unusable until synchronization finishes because the screen jitters jumps around when your messages appear. I think it's just poor/unpolished design mostly. ↺

> sounds like a poor abstraction then No, the thing is, goroutines are not threads, But you can use them like threads. It's just that threads are a convenient abstraction, it's more convenient to write sequential code than to schedule callbacks every time you "context switch" ↺

>> In Gajim everything is in one single thread, UI has high prio, network lower prio, if there are CPU intensive ui operations, pulling from the network simply gets Stalled a bit > gajim also basically locks up if any of the rooms im in get a spammer Hard problem to solve. If you receive more than your GUI can handle it will lock up, I don't see how any async operation will solve that, then you spam your ui thread from your network thread

actually i mispoke earlier, goroutines are aparently _just threads_ according to the go room

> No, the thing is, goroutines are not threads, But you can use them like threads. It's just that threads are a convenient abstraction, it's more convenient to write sequential code than to schedule callbacks every time you "context switch" forgot to reply ↺

> Is it because of blocking though? Dino is also unusable until synchronization finishes because the screen jitters jumps around when your messages appear. I think it's just poor/unpolished design mostly. i dont think its because of blocking, rather python being slow and gajim needing more than one thread of my cpu can offer ↺

> >> In Gajim everything is in one single thread, UI has high prio, network lower prio, if there are CPU intensive ui operations, pulling from the network simply gets Stalled a bit > > gajim also basically locks up if any of the rooms im in get a spammer > > Hard problem to solve. If you receive more than your GUI can handle it will lock up, I don't see how any async operation will solve that, then you spam your ui thread from your network thread why is the ui rendering chats i dont have open? ↺

gtk is also slow and shitty

> actually i mispoke earlier, goroutines are aparently _just threads_ according to the go room wrong ↺

https://downloadable.pain.agency/file_share/06835569-9d5b-7d90-ba31-61ab0ff960ea/853190a7-47fe-4f7d-ba53-7a563cfaaa51.png

from someone who works at google writing go code (is my understanding, dont quote me)

> A goroutine is a lightweight thread managed by the Go runtime. - https://go.dev/tour/concurrency/1

> Is it because of blocking though? Dino is also unusable until synchronization finishes because the screen jitters jumps around when your messages appear. I think it's just poor/unpolished design mostly. also for this, i believe with the ui lib im using (fyne) i can just not update scroll and it shouldnt jump around, however im a bit ahead of myself as im still working on parsing out features from the xml ↺

It is thread in the sense that that is how it appears to the user, but it is not an OS thread. What other programming languages, e.g. python, call threads is not the same thing. Goroutine is a green thread.

i am aware of it being a "green thread" however i dont know a ton about what that means

all i know is that it is able to hit multiple cpu cores, and also has race conditions if you dont plan ahead

Python's coroutines could also be called thread, with a stretch. It's just that context switching there is explicit (await)

> Python's coroutines could also be called thread, with a stretch. It's just that context switching there is explicit (await) i mean go you have waitgroups and all mannor of different controls over blocking and context switching, and instead of awaiting async things you async execute syncronous things, no? ↺

> i am aware of it being a "green thread" however i dont know a ton about what that means It means that with regular threads, your computer will be crawling before you hit even 10,000. But with green threads, you usually can spin millions of them. Green threads are managed by the programming language runtime, while regular threads by the OS. Green threads can utilize OS threads under the hood, as go does, I believe. So for example, when a green thread is doing something CPU bound, it ends up in its own OS thread. But when it is doing something IO bound, it shouldn't run an OS thread. ↺

> It means that with regular threads, your computer will be crawling before you hit even 10,000. But with green threads, you usually can spin millions of them. > Green threads are managed by the programming language runtime, while regular threads by the OS. Green threads can utilize OS threads under the hood, as go does, I believe. So for example, when a green thread is doing something CPU bound, it ends up in its own OS thread. But when it is doing something IO bound, it shouldn't run an OS thread. ah yea okay that was my understanding of what the go runtime does, i just didnt know that was it being a "green thread" ↺

"green thread" just means the OS is not involved when doing context switches between threads. It can be faster than "native" threads (because there's no need to save and reload all cpu registers) but for compiled languages I think it restricts you to cooperative multitasking (the running thread has to do something that allows other threads to take over)

allows other threads to take over how?

io bound

select() under the hood probably

im hoping to everying i can make it not particularly cpu bound

kinda my hard limits (allowing for gui overhead) is maximum 250mb of ram and barely hitting the cpu under normal usage. this should be relatively easy using a compiled language unless i write complete spaghetti

i mostly wish to avoid io bottlenecking, i.e. being able to switch chats shouldnt get delayed by incomming messages or a db write,

(also for context as mentioned earlier i have a ryzen 7, so my barely hitting the cpu is more my metric of how okay will it do on poor devices, maybe once i get any kinda meaningful client i'll test on an old laptop i have with a shit cpu and dying slow hard drive)

Writing messages to a database really should not result in any freezes even if it is a single thread. It can only result in it if it's a poor design. For example, maybe if Gajim was smarter about fetching messages and didn't fetch a whole lot of them, but only enough to fill the screen, that would already solve the problem ( But yes, the Algorithm would be more complicated because it would then need to deal with holes in local archive). Also, if I remember correctly, SQLite has a built-in mode that grants you the same experience as if it was non blocking, you can do lots of wriyes and it will be fast, I think it's called WAL. All in all, I think that with a good design, a single OS thread would be enough for an xmpp client to work without freezes and be responsive. It's just chat. There isn't such a high throughput of messages to a single client and there isn't that much to do about them computationally. ✏

> Also, if I remember correctly, SQLite has a built-in mode that grants you the same experience as if it was non blocking, you can do lots of wriyes and it will be fast, I think it's called WAL. ah yes i was told about this, i just thought it "makes it thread-safe" though

and yeah one os thread is probably sufficient, however i know i need at least 2 routines since both the ui and connection are blocking

i didnt know that wal gave a non-blocking experience, however ill keep that in my back pocket

At most, if I wanted it to be really super smooth (and that is what I would want for me client as a user) I would spin a separate UI thread. But I would also understand that that would itself already complicate things for me as a programmer. And if I'm a one-man team and have very limited time and resources, the cost would likely outweigh the benefit. The added complexity will mean slower development and more bugs, ending up making the user experience works, not better.

i'd probably also want to do things such as file uploading/downloading in a different goroutine as that can take many minutes, however thats mostly io-bound as well

> At most, if I wanted it to be really super smooth (and that is what I would want for me client as a user) I would spin a separate UI thread. But I would also understand that that would itself already complicate things for me as a programmer. And if I'm a one-man team and have very limited time and resources, the cost would likely outweigh the benefit. The added complexity will mean slower development and more bugs, ending up making the user experience works, not better. i dont have control over ui being a separate thread, however simply having the main goroutine be for ui and doing everything else in a second goroutine is likely not going to add a ton of complexity ↺

With Go and an xmpp, I think the only metric you should consider when deciding whether to spin a new thread or not is if it makes life easier for you as the developer.

i mean some things (again upload and download of files) it is likely necessary, but i hear you and will heed the advice

okay i know stanza-id is generally for muc, but https://xmpp.org/extensions/xep-0359.html doesnt mention its specific to mucs. is it safe enough to attempt to get stanza-id first and if that is unavailable fall back to origin-id?

it would only really be used for relations and whatnot, so i figure not too terribly dangerous

These are not equivalent

they arent?

the documentation is written in a way that is very hard to parse

No, origin is attached by clients, stanza-id by servers

yes i am aware of that

And the third one is the message id attr

yeah the third one im not sure about, i can remove that fine

You need to parse all three and hold them available for other parts of your app that needs them for some logic

> You need to parse all three and hold them available for other parts of your app that needs them for some logic im not removing them, im just creating a method to get the one to use for relations ↺

the method should probably be renamed, which is fine

And you will need all three depending on what you implement

There is no single id attr for relation

Some xeps use stanza-id some others use messag-id

🥴

Message-ID and stanza-id must be written to the database

Origin-id not necessarily, at least in Gajim we don't and I didn't yet run into trouble

i plan to try and store an xml representation as equivalent as possible to what i got off the wire, just as backup

ill leave id parsing to whenever im doing operations on the message then, i suppose

xml decoding in a statically typed language is a bit annoying

also is origin-id ever not supplied?

Yes, it's added by the client if they want to

You need to expect to see it never

If they feel like it

alright ill leave it a nillable atribute

All of the mentioned IDs are nullable

at this point the only non-nullable items i have are ones that i tack on throughout processing ✏

(and the header items, i.e. from, to, etc, but thats a separate struct with how im parsing the items in) ✏

from and to are both optional not only with bad data, but also in various cases (for example no from on a c2s send) but maybe no cases that happen to matter for you

the struct for what i keep calling "header data" is already provided by mellium so im just using that. they use a custom jid object which allows you to pull out different forms of the jid so i just assume it'll handle it

jjj333_p (any pronouns): you can only use/trust stanza-id in a room that advertises support for it, otherwise client can set it also beware and handle the case where multiple stanza-id are set with the same by, treat that as no support advertised (ie ignore all together)

🫠

I noticed a dead link in https://xmpp.org/extensions/xep-0453.html section 5 "business rules". The README file is linked with a .rst extension but it was changed to a .md on github