lovetoxi had multiple ideas but all seem not perfect
lovetoxfirst i could spin up a server and exchange some stanzas to test some specific cases
lovetoxlike publishing something to pubsub
lovetoxbut you cant test everything with that setup, mostly IQ stuff where the target is the server
lovetoxalso it gets more complex, if you need stuff setup before on the server
lovetoxbut in theory it would be really nice, because that really end 2 end, and you could spin up multiple servers and test
lovetoxquestion is, is this really necessary
lovetoxin theory i know wht the server must answer because of the specs
lovetoxand im testing my lib here and not server implementatons
lovetoxwhich brings me to another idea, a kind of own application, a server/client fake thingy
lovetoxthat does not really behave like a server or client, maybe i communicate a testcase, afterwards the app just replays some kind of setup stanza flow
lovetoxsuch a app would be nice, because it could be independent, like i set it up, some client connects communicates a testcase according to some spec
lovetoxthe testcase is somewhere written down
lovetoxand then the app just plays it through
lovetoxfor example client connects and sends:
lovetoxand then the server just sends you some not according to spec pubsub messages
lovetoxwriting such a app and a basic doc how it behaves is probably a weekend work
lovetoxand people could then add new testcases with unique names, and describe them in some wiki
lovetoxevery client could choose which testcases it wants to execute
SamI do both of those things in Mellium. For unit tests on anything more complicated than "serialize this to XML and make sure the output matches" I have a dummy XMPP session thing that just records anything sent/received over it and I can have a dummy listener sending pre-canned responses, then I can check that the final exchange looks right. For integration tests I have a thing that can spin up an actual server (prosody or ejabberd right now) and do stuff against them.
SamThe integration test can also spin up a client (mcabber or sendxmpp or itself) so that you can have a remote client to communicate with through a server
ZashLooking for the reverse of https://matthewwild.co.uk/projects/scansion/ ?
lovetoxbut is it really a client Zash?
lovetoxwhat makes it to a client?
pep.It's a client because it connects like a client, it talks like a client, it ducks like a client? :P
ZashIn its current form, it connects to a server and sends the "Someone sends" stanzas, reads something back and compares it to "Someone receives"
ZashNot sure how much work it would be to invert it, probably not too hard actually.
pep.Isn't it a client as long as it opens the stream with jabber:client? :)
lovetoxZash, does it need a full authenticated stream for every testcase?
moparisthebestSo that means things that connect with websockets are not clients pep. ? :)
flowlovetox, Smack does have a dummy connection and protocol verification mechanism for unit tests. And an integration test framework which runs tests against a real server. Openfire picked the integration test framework up to test the server implementation
lovetoxZash, if i understand correctly there is not much to invert, the testcase just needs to start with "receive"
lovetoxand of course it needs to be added that it runs and waits for something to connect ot it
Zashlovetox, it uses actual connections to a server. by "invert" I mean making scansion pretend to be a server and accept client connections to it, somehow. then it could probably use the same tests, just flip the meanings of "sends" and "receives" to make sense
flowI hoped that we will one day have someting like https://cdecker.github.io/lightning-integration/ or https://datatracker.ietf.org/doc/html/draft-dkg-openpgp-stateless-cli-01, that is, a common abstraction of an implementation that allows us to test multiple different implementations against each other
ZashIt would be pretty cool if you could use the same test scripts to test both clients and servers 😀
lovetoxflow, sounds like lightyears away :D
flowlovetox, just needs someone with enough time and skill
flowso yeah, maybe lightyears, but maybe only lightseconds