-
jonas’
fg
-
MattJ
Hey!
-
dwd
Ge0rG, It's looking very like I'll have copious amounts of free time quite shortly, and if so I'll happily help out with performance work etc on Yax.im.
-
Ge0rG
dwd: are you really sure about that? This is a deep hole full of unshaved yaks!
-
mathieui
for the yak horde!
-
dwd
Ge0rG, In fact, I can probably help anyway, but I'd need some access to a pgsql instance if you're switching to that. I got ours to cope with 700% increase during the pandemic. It was... fun?
-
Ge0rG
dwd: TL;DR: my prosody+sqlite combo is running into a limit of ~200 messages stored into MAM per second, causing starvation of other connections
-
dwd
Ge0rG, Yeah, we have PostgreSQL handling about 1.3 million per hour at peak. I need to do that talk, don't I?
-
Ge0rG
dwd: I've done some first synthetic benchmarks of psql on the yax.im machine and it reduced per-message storage processing from 3.5ms to ~1ms
-
dwd
Ge0rG, Where are you hosted?
-
Zash
dwd, is that 360/s ?
-
Ge0rG
dwd: in Berlin, at my ISP's uplink provider, running some LXC in KVM on baremetal
-
dwd
Zash, Yeah, but it's doing realtime analytics as it goes, and encrypting the message content in the database for regulatory reasons, which is the slowdown. It does actually get faster than that, perhaps 500/s.
-
dwd
Ge0rG, So "hand installed" PostgreSQL, not something like AWS RDS.
-
Ge0rG
dwd: from my first synthetic stats, I might get from 200msg/s to something around 300-500 by switching to psql, which might or rather might not suffice to cope with the VaxBot data flood
-
Ge0rG
dwd: yeah.
-
dwd
Zash, Oh, and full-text indexing. Yeah, that costs, too.
-
Ge0rG
this is my schema: https://pastebin.ubuntu.com/p/sC5H7GRF9Q/
-
dwd
Ge0rG, Do you have pg_stat_statements available? That should tell you where the costs are. Also you don't need both the "prosody_index" and "prosodyarchive_index", the latter includes the former. Beyond that, I'd need to understand what the SQL queries are, but I can find that from the source I suppose?
-
Zash
dwd, those are different tables tho
-
dwd
Ge0rG, (Also I may have to track down and murder whoever wrote that schema using reserved words as column names)
-
Zash
Just quote your friggin column names
-
dwd
Zash, Oh, indeed. Is that a generic keystore in SQL?
-
Zash
key-value store, yes
-
dwd
Zash, Well, I'll have to thanks to the dumb naming. :-)
-
Ge0rG
dwd: the queries are done through two or so abstraction layers :(
-
Ge0rG
dwd: did I mention I'm still on sqlite?
-
dwd
Ge0rG, OK, so for postgreSQL, you'll want this: https://www.postgresql.org/docs/current/pgstatstatements.html
-
Ge0rG
dwd: noted
-
dwd
Ge0rG, And in a perfect world, we'd do internal observability with something like New Relic. Oh for a server not written in a novelty language! ;-)
-
Ge0rG
Well, I'm probably also running into a CPU bottleneck, either from creating debug logs or from memory management
-
MattJ
New Relic = $$$$
-
dwd
MattJ, Yes, but amazing, and their new pricing works out close to free for hobby projects.
-
dwd
MattJ, I believe you can get some of the benefits from Prometheus and similar, but I've not seen it done.
-
emus
Nice, F-droid is following XMPP on Fosstodon. Hopefully, they boost one or the other toot đź‘Ť
-
dwd
The guy who started F-droid, Ciaran, ran his own server for years though.
-
dwd
Lovely chap.
-
emus
ah okay nice + we have 1200 followers now. thats 400+ in the last two months I think
-
Zash
Graphs? Graphs!
-
moparisthebest
I think the one-two punch of whatsapp screwing their users and signal becoming a shitcoin wallet really helped
-
dwd
Shame so many XMPP jobs are in the funny-money business these days.
-
dwd
Though I admit I've never quite understood the connection.
-
Kev
Blockchain is trendy, XMPP is trendy, two great tastes that taste great together.✎ -
Kev
Blockchain is trendy, XML is trendy, two great tastes that taste great together. ✏
-
dwd
XMPP isn't trendy, it's just riding the XML craze?
-
Kev
Right.
-
Ge0rG
All my appointments are getting delayed, but I'd like to come to office hours
-
emus
https://fosstodon.org/@xmpp/106058677871732292
-
dwd
Well, that was fun and interesting, thanks Sam!
-
Zash
👍️
-
emus
Yes, I can agree to that. I think you hit a sweet spot, also with the XMPP Hours in general!
-
Sam
Thanks for coming!
-
Sam
I was worried about that one since we hadn't tried a roundtable, but I'm happy with it.
-
Zash
Maybe you should force us to introduce ourselves, so we get used to the concept of talking into the mic 🙂
-
Zash
Also need to figure out when to make dinner in relation to these slots.
-
Sam
Good idea.
-
Sam
Sorry about the food time overlap; this slot isn't exactly ideal but the most people marked it as green
-
Sam
Threw the notes up, sorry that the formatting didn't translate: https://wiki.xmpp.org/web/XMPP_Office_Hours/2021-04-13-Notes
-
jonas’
Kev, friendly ping https://github.com/xsf/xeps/pull/1047
-
Sam
dwd: Unsure if you were still there when I said this, but if you need to do your talk "soon", feel free to bump mine next week. But if that's too soon, there should be plenty of weeks open after the week after that
-
mathieui
Thanks for the notes Sam, I haven’t yet been able to attend office hours, but it is great!
-
Sam
I just realized I'm getting my second dose of the Pfizer vaccine right before office hours next week, so we'll see if I'm too busy creating antibodies and am hurting or falling asleep :) I have friends who said the second shot really knocked them on their ass for a day or two
-
Sam
But I don't remember what manufacturers shot that was, so maybe I'll be loopy or something during the presentation :)
-
jonas’
you should be adding a trigger warning to such statements because they can easily cause envy
-
Daniel
i was about to say the same
-
menel
Sam: that takes a few hours to sink in so you will be fine for just until the talk is over :-)
-
Sam
ooh yah, sorry
- mathieui looks at his 2024 vaccination date
-
jonas’
(I wasn’t really serious)
-
jonas’
I’m happy for all the folks who can get protection already
-
Sam
I love mastodons content warning feature, we need that for xmpp :)
-
jonas’
spoilers?
-
mathieui
Sam, we have spoiler messages already !
-
menel
Xep-spoiler warning for conversations
-
Sam
We need that to be supported in XMPP then
-
Zash
Converse.js supports it AFAIK
-
Sam
I thought it had limited usage until I joined mastodon, now I love it for anything longer than like a sentence or anything I'm just tired of seeing and assume others will be too
-
deuill
Awesome work with note-taking.
-
Sam
I have no idea who was doing that, but it wasn't me after the first few minutes; thanks to whomever silently took over while I was struggling to both call on people and pay attention and note take
-
Zash
further evidence that chair and minute taker make sense as separate roles
-
jonas’
are the notes funny? if yes, it was Tedd
-
mathieui
sorry, reading the minutes for the office hours and I don’t know if "compliance suits" is intentional or not but it is pretty funny
-
Ge0rG
mathieui: are those orange?
-
mathieui
don’t ask me, I haven’t read that XEP yet
-
Ge0rG
Probably it means the white ones, with the long arms that you bind behind the back
-
mathieui
I only have like 5 XMPP-related pieces of clothing, none of which are suits!
-
Zash
Suits instead of hoodies for the post-pandemic Summit/FOSDEM?
-
vanitasvitae
+1
-
vanitasvitae
or snobby bathrobes!
-
Zash
Doesn't the logo sorta look like an up-side-down untied tie?
-
Ge0rG
https://upload.yax.im/upload/GbsmIjpFUgjFP3q4/hmgoepprod.jpg
-
Ge0rG
Here's Bruno branded
-
dwd
XMPP dressing gowns? Or xmpp smoking jackets?
-
Ge0rG
XMPP straitjackets
-
Ge0rG
(that's at least my association with "compliance suit")
-
moparisthebest
Sam, OT but my second phizer shot gave me the same side-effects as the first, felt like I'd been punched in the arm a lot, that's all, YMMV of course :)
-
Sam
I had nothing from the first one really, just like getting a flu shot or something, tiny bit of stiffness but nothing really
-
moparisthebest
did anyone play that game in school where you and a friend would take turns punching each other's arm as hard as you could until one of you gave up? that's what it felt like lol
-
Ge0rG
Those are very strange games.
- Ge0rG is playing the game of xmpp server meltdown to help another country get vaccinated despite that country prohibiting vaccine export to where the server lives
-
moparisthebest
now *that's* a strange game
-
mathieui
which reminds me I wanted to setup a quick & dirty french vacbot
-
mathieui
happy to get jabberfr to meltdown as well for this fun game
-
mathieui
or should I recommend yax.im?
-
Ge0rG
mathieui: easy. `onMessage() { return "sometime in 2024"; }`
-
jonas’
mathieui, decentralization!
-
mathieui
Ge0rG, nah, for people above 55 it should be quite doable
-
deuill
This Office Hours session will probably be the starting point for a few more, the way discussions went. It's awesome to see people involved in the project in very different ways providing their views, and where these overlap or diverge.
-
Sam
Only if people actually think about what they want to discuss and put it on the calendar :)
-
deuill
Some threads I'm picking up: - Focus on client development and compliance. Possibly narrow scope of expected use-cases (e.g. IM vs. generic pub-sub). - C2S and S2S improvements, possibly backwards-incompatible. Looking towards better mobile compatibility.
-
theTedd
> are the notes funny? if yes, it was Tedd wasn't me; I'm still catching up
-
Ge0rG
Ketchup!
-
theTedd
Mustard!