jdev - 2021-11-21

i wonder how xmpp libs do unittests

i had multiple ideas but all seem not perfect

first i could spin up a server and exchange some stanzas to test some specific cases

like publishing something to pubsub

but you cant test everything with that setup, mostly IQ stuff where the target is the server

also it gets more complex, if you need stuff setup before on the server

but in theory it would be really nice, because that really end 2 end, and you could spin up multiple servers and test

question is, is this really necessary

in theory i know wht the server must answer because of the specs

and im testing my lib here and not server implementatons

which brings me to another idea, a kind of own application, a server/client fake thingy

that does not really behave like a server or client, maybe i communicate a testcase, afterwards the app just replays some kind of setup stanza flow

such a app would be nice, because it could be independent, like i set it up, some client connects communicates a testcase according to some spec

the testcase is somewhere written down

and then the app just plays it through

for example client connects and sends: <testcase name="invalid-pubsub-event-message">

and then the server just sends you some not according to spec pubsub messages

writing such a app and a basic doc how it behaves is probably a weekend work

and people could then add new testcases with unique names, and describe them in some wiki

every client could choose which testcases it wants to execute

I do both of those things in Mellium. For unit tests on anything more complicated than "serialize this to XML and make sure the output matches" I have a dummy XMPP session thing that just records anything sent/received over it and I can have a dummy listener sending pre-canned responses, then I can check that the final exchange looks right. For integration tests I have a thing that can spin up an actual server (prosody or ejabberd right now) and do stuff against them.

The integration test can also spin up a client (mcabber or sendxmpp or itself) so that you can have a remote client to communicate with through a server

Looking for the reverse of https://matthewwild.co.uk/projects/scansion/ ?

yeah Zash

but is it really a client Zash?

what makes it to a client?

It's a client because it connects like a client, it talks like a client, it ducks like a client? :P

In its current form, it connects to a server and sends the "Someone sends" stanzas, reads something back and compares it to "Someone receives"

Not sure how much work it would be to invert it, probably not too hard actually.

Isn't it a client as long as it opens the stream with jabber:client? :)

Zash, does it need a full authenticated stream for every testcase?

So that means things that connect with websockets are not clients pep. ? :)

moparisthebest, that

lovetox, Smack does have a dummy connection and protocol verification mechanism for unit tests. And an integration test framework which runs tests against a real server. Openfire picked the integration test framework up to test the server implementation

Zash, if i understand correctly there is not much to invert, the testcase just needs to start with "receive"

and of course it needs to be added that it runs and waits for something to connect ot it

lovetox, it uses actual connections to a server. by "invert" I mean making scansion pretend to be a server and accept client connections to it, somehow. then it could probably use the same tests, just flip the meanings of "sends" and "receives" to make sense

I hoped that we will one day have someting like https://cdecker.github.io/lightning-integration/ or https://datatracker.ietf.org/doc/html/draft-dkg-openpgp-stateless-cli-01, that is, a common abstraction of an implementation that allows us to test multiple different implementations against each other

It would be pretty cool if you could use the same test scripts to test both clients and servers 😀

flow, sounds like lightyears away :D

lovetox, just needs someone with enough time and skill

so yeah, maybe lightyears, but maybe only lightseconds