jdev - 2025-10-29

Is there something like https://xmpp-interop-testing.github.io/ for client testing?

(Asking out of curiosity; not planning to use it at the moment but maybe in a few months)

I don't think so (unless you count installing several clients and using multiple accounts to test against them all)

how could something like that even exist generally for clients? wouldn't it need to click, type, and check for certain text and/or elements to be displayed? Sounds always specific-per-client no?

There's certainly some stuff that could be tested on the protocol level. There's plenty of interaction that, when a client receives X, it is expected to reply in a certain manner, even without user intervention. I'd expect the automation to take the form of a kind of server, that starts querying the client after some kind of setup. It's not something that the xmpp-interop test framework intends to support though.

in the client direction, it'd definitely be useful to have e.g. an ad hoc commands torture tester that attempted all valid data form elements and let you see how a client copes with them

Guus, the cases where a client has to reply to something are very limited, mostly stream init / auth stuff, and maybe jingle

disco/info, caps, ...

sure, it's not a _very_ large list - but even just checks for basic protocol conformance would be a plus.

> when a client receives X, it is expected to reply in a certain manner, even without user intervention. like what? if a remote entity can trigger a response from a client who didn't request something I'd consider that a fairly serious (security & privacy) bug, possibly even at the protocol level

And, although I'm not planning to work on this myself, I think it could be valuable. Not just because it'd give us tools to improve our clients, but also because it gives us tools that can be used to give evidence that things _are_ compliant. That can be very valuable in scenarios where the protocol or implementations are being considered as an interoperable solution - like the Dutch healthcare standard that we've been discussing this week.

moparisthebest: simple stuff like service discovery, pings - even basic checks to validate that an IQ error response includes a child element are valuable. Don't think of this as something that attempts to validate protocol conformance 100% - that's not something that you can automate - but automating the crap out of the silly, small bits (where often, bugs tend to linger) isn't the worst idea.

It would not be the first time that a client does not respond (at all - not even with an error) to an IQ stanza that it receives, if it doesn't understand the child element. Obviously a bug, quite easily tested with a high degree of certainty.

first time in 10 years im trying to do more than the standard +notfiy PEP stuff with pubsub, and just subscribing leads already to multiple questions and implementation differences

bribe Guus to add more tests for that to the interop tests

> It would not be the first time that a client does not respond (at all - not even with an error) to an IQ stanza that it receives, if it doesn't understand the child element. Obviously a bug, quite easily tested with a high degree of certainty. clients shouldn't be silently and automatically responding to attackers poking them... ↺

An IQ cannot be sent by an attacker to a client

Because you need to know full JID to send iq

Full JID knowledge is considered equivalent to knowing their presence by basically everything in the protocol

but "knows full jid" or even "has mutual presence subscription" absolutely does not imply "not attacker" though...

Well. Not the kind of attack where replying to their iqs with an error counts as an exploit

Doesn't really matter for test strategies, does it? Specifications dictate what responses, if any, are to be generated.

> Well. Not the kind of attack where replying to their iqs with an error counts as an exploit yes, exactly that kind of attack ↺

> Doesn't really matter for test strategies, does it? Specifications dictate what responses, if any, are to be generated. It does matter, specifications (especially old ones) often MUST be ignored if it's actually a terrible idea ↺

In what way is it a terrible idea though? It's a presence leak but only if you don't consider knowing the full jid to be a presence leak already, which we do

it's going to be a case by case basis, but the default should be if you don't understand something or why someone would be sending it to you, do not respond

obviously anything sending an IQ request has to be able to handle no response right? because despite RFC saying things, there are a million scenarios where you might not get one (say someone trips over a network or power cable), hopefully we can all agree on that at least?

so not responding can't actually break anything that isn't already broken