XMPP Service Operators - 2024-03-11

Does anybody have a redundant Prosody server?

high availability?

define "high availability"

service remains online if one of the servers is taken offline

or suffers a failure

"remains online" certainly not, because prosody does not support active/active configurations ✏

hot-standby may work actually if you use a replicated storage backend, but you still need custom logic to ensure the other node is truly down.

has anyone already implemented that?

not that I know, because it is a hard problem. ✏

determining that another node is truly down is a bit tricky.

you need some kind of quorum system

with a database backend which uses that, and by configuring prosody to only use the local backend, and making prosody shut down when the backend claims that it lost quorum, it could be done.

lets say that we can determine a node health

there's a plugin for prosody's internal webserver the exposes timers, and a simple script could be made that checks that and prosody's health

possibly even latency

and if your going for a ACTIVE/BACKUP model, a circuit breaker style trip that switches to the backup if the primary server stops working or latency gets too bad.

what else needs to be done?

postgres already supports that model

what you described isn't sufficient

the latency may only look bad from your other node, e.g. because of a temporary internet routing issue

you need to ensure that before starting the backup node, the active node is killed.

the prosody daemon is killed or just not connectable?

killed.

why?

if no clients or servers are able to connect to it, what's the harm?

cronjobs inside prosody

oh

what else?

I don't understand

what do those cron jobs do? also you mentioned a replicated storage backed.

for instance expiry of uploaded files/storage, but in general arbitrary code and you cannot rely on them (not) doing a specific thing

what storage needs to be replicated? something in the filesystem or can everything be done in the database?

the only safe way to do this is to ensure prosody is *stopped* on all nodes except one.

I don't know of a way to put uploaded files into the database, so you'd likely need that (or an external service for that feature) + database for everything else.

I already have prosody's http uploads managed by an external service as part of my webserver. a crontjob on the external webserver manages file expiration based on atimes

is there anything else?

all prosody's job is for files is to generate tokens for the upload server

the safe way of doing this, as far as I know (the prosody people would know better): Prerequisites: - ensure prosody uses replicated storage for everything, e.g. a database - have reliable measurement of availibility of all nodes - have a way to "fence" (turn off) a node remotely and be sure it's actually off, even when it is already broken/unreachable Failover: 1. detect that currently active node is down 2. make sure it stays down (i.e. turn it off) 3. ensure data is replicated correctly 4. start backup node

anything other than exactly this failover procedure is gambling.

» 3. ensure data is replicated correctly is it talking about the database backend?

yes.

also something that's not covered; restoration of the primary server.

should not need extra action with proper databases, but you never know.

well restoration of the primary server is just like another failover.

doesn't seem impossible

» - have a way to "fence" (turn off) a node remotely and be sure it's actually off, even when it is already broken/unreachable would iocage stop prosody1 suffice?

I do not know what those words mean.

» 3. ensure data is replicated correctly how is this done?

» <jonas’> I do not know what those words mean. turning off the BSD jail that prosody belongs too

seems like it would be sufficient.

regarding "3. ensure data is replicated correctly" / "how is this done?" -> with a proper replicated database, that's going to be a given and doesn't need separate checking.

if you use some homebrew hot-standby filesystem sync trickery, that's a different story.

so you just mean configure a postgres cluster not run a application specific database table consistency check

yes

is there any benefit or cons to configuring prosody to use databases instead of files that it uses by default?

yes.

(but don't ask me for specifics)

> is there any benefit or cons to configuring prosody to use databases instead of files that it uses by default? Normally speed

and maintainability

and scriptability

(the discussion moved on to prosody@conference.prosody.im)

oh... ok nevermind then :)

thanks :)

Greetings to one and all

Would it be correct to assume that an agent/transport (server component) JID can not return ping?

I ask because a function that pings to own JID appears to indicate that the JID does not return ping, when JID is component.

every XMPP entity MUST respond to an IQ request - even if it doesn't understand it.

correct.

(re-asked in jdev, a more appropriate venue I think)

So, even if the component does not understand the 'ping' request, it should still return an error.

correct, MattJ.

Pardon for cross-posting

Guus, I run the component and it appears that I fail to receive ping from the component itself to itself.

Both, client and component have the same XEPs loaded.