jdev - 2024-07-04

taba: https://wiki.archlinux.org/title/Tor#Transparent_Torification

> maybe you should listen to what people suggest before just asserting that they're wrong qy: i know what he's talking about toxigoosy 🙄 ↺

> taba: https://wiki.archlinux.org/title/Tor#Transparent_Torification > In some cases it is more secure and often easier to transparently torify *an entire system* instead of configuring individual applications to use Tor's socks port, not to mention preventing DNS leaks. ✏ ↺

> not how tor works... you need clients designed with tor in mind it didnt sound like it, but sure ↺

> Transparent torification also will not protect against fingerprinting attacks on its own

>> not how tor works... you need clients designed with tor in mind > it didnt sound like it, but sure https://support.torproject.org/#faq_staying-anonymous 🙄🙄🙄🙄🙄🙄🙄🙄🙄🙄🙄🙄🙄🙄🙄🙄 ↺

"not how tor works"

> "not how tor works" 🙄 ↺

the firewall-rule thing can also be done to one user, or a network namespace (cf. orjail)

😔the module doesn't seem to work anywhere

does xml care if its tab or spaces? right now i am thinking in context of the doap but probably its applicable everywhere

the default in gnome builder seems to be spaces

MSavoritias (fae,ve), you can use either, and in most cases they are irrelevant.

actually i have a better question: is there something i am missing about defining a XML Schema Definition Language (XSD) for XMPP and then use that to restrict what we can receive? probably we are losing much of the extensibility right?

> MSavoritias (fae,ve), you can use either, and in most cases they are irrelevant. ah ok. thats good

xml:space controls some of that.

so xml is basically a lisp then :P

MSavoritias (fae,ve), in XMPP we usually don’t use the XML Schema for much, I’ve heard of some server software using them to validate client input, but I’ve never used that.

There are definitely organisations that do validation of one form or another at their boundaries as a security thing.

There is also EXI which is meant to negociate that between the client and the server.

i thought EXI was only a compression binary thing. but i guess since it does specify what exactly the things inside are it can be for the same thing

> There are definitely organisations that do validation of one form or another at their boundaries as a security thing. yeah that was my thinking. it would make parsing easier if i knew what to expect

I tend to use the schemas as a guideline about what to accept in my software, more as documentation.

MSavoritias (fae,ve), you don't want to konw what XML does to whitespace :-)

:O

in particular: within attribute values, tabs, spaces, carriage returns and newlines are folded to space. outside of attribute values, CRLF is converted to LF

and this is non-optional XML 1.0 behaviour.

ah okay. i was referring to outside of that

but noted not to touch anything inside the xml :)

Is there a server developer (specifically someone who works on an XMPP server project that has a build pipeline, for example in GitLab, GitHub, Bamboo or CircleCI) that is interested in helping Fishbowler and me with a new XMPP interop testing project that we've been working on these past few months?

We're looking for someone with fresh eyes, an alpha tester if you like. We've been so heads-down in this project, that we might miss things that we incorrectly assume are obvious. We'd like someone else to verify / comment on the usability of our project, so that we can improve things.

Depending on your pre-existing CI implementation, I expect this to take somewhere between one to four hours.

We self-host our CI in buildbot

MattJ, I think that won't be compatible at this stage. I'm trying to dive into its documentation (much of it 404's) - I think it supports a plugin architecture, which we might be able to leverage in the future to add direct support.

Why does it need special support from the CI system?

and what 404s?

The project's main purpose is to generate specific

plugins/orbs/actions that can natively be added to the CI system that's being used.

we've simply not created something for buildbot yet.

Seems more restrictive than just producing a standalone tool that can be used in any environment

many links found on https://buildbot.net/ (under 'get started' for example) yield 404 responses.

Sometimes I might want to run tests outside of CI

The Buildbot docs are at https://docs.buildbot.net/4.0.0/manual/index.html

Not sure what's going on with the site, I'll let them know

Seems they link to /current/ but readthedocs does /latest/

I believe that the cake is mostly a lie, with regards to being able to create a stand-alone tool. If anything one would exist by now. What we produce does use a common component that's wrapped in CI-specific builders, which you can run as such a stand-alone tool. I'm pretty sure that as soon as I disclose its nature, people will start having a debate about its nature / runtime requirements, etc.

I think MattJ is just saying that an interop tool does not has to be CI specific

If something not existing can be used as evidence that it's not possible, I have news... :)

That's the bit I'd like to avoid with providing CI-specific integrations that are easily hooked into existing pipelines. The drawback is that we need specific support for each CI system.

OK, let me put it this way: it _has_ existed (for at least a decade), and no-one except for Openfire was using it.

And now nobody is using it because it's not compatible with our build environment, or any of the future build environments

Sorry, that probably came out a bit harsh, I'm not intending to say you've done a bad job

It just feels like an unusual design decision, that only restricts the applicability of the project

I am thinking "simply wrap the XMPP servers into containers (or a "local" temp installation), wrap a script around it, that starts a server cointainer, points the test suite at it and repeats this for every server implementation" ✏

Making it work only on a defined list of specific provider APIs seems like it's more prone to becoming obsoleted in the future as the set of popular providers changes, or as existing providers update their APIs and interfaces

Before starting, we've tried to analyze what CI systems were in use by server implementations that we could find. I think we've added support, or are planning support for all build systems that we could identify that were in use, with the exception of Prosody's.

Whereas how to execute a standalone executable has not changed for decades

So how many server implementations do you cover?

I now start to wonder everyone is one the same page. I think you can run guus' tool standalone without a CI and simply point it to any server (implementation)

If that's true, then I take everything back :)

the result will be a list of tests that passed/failed

But it sounded to me like the whole thing only works in specific CI environments

MattJ, I expect that we're currently covering the CI systems used by 7 or 8 of the server projects

out of the 9 server projects that I tried to find the CI system in use for.

basically the tool is based on Smack's integration tests, but on steriods ✏

I'm surprised we have 9 active server projects

Astrachat, Openfire, EJabberd, Tigase, Prosody, M-Link, Jackal, Metronome, Mongose

Openfire, ejabberd, Mongoose, Prosody... do you cover closed-source ones?

various degrees of 'activeness'...

Oh, Tigase of course

And Astrachat is a from scratch implementation or a fork of?

IIRC the Jackal maintainer announced it was no longer being developed

anyway, I'm not really looking to debate the approach that we've already committed to. I definitely want to somehow be able to service Prosody too - either by a buildbot specific thing, or by something more generic like docker, or just bare-boned Smack SINT .

Guus, I am not sure if there is anything wrong with the approach, it's just that I find it strange that you talk so much about CI that it sounds like the approach requires a specific CI system

of course, I could be misunderstanding something

I just don't understand what you could possibly need to do that would require a Buildbot plugin that you can't just do directly

but basically you/we have a integration test runner and a set of integreation tests that you can point to any XMPP port

simply by "./run <ip-or-host>"

flow, that's because that I came here today looking for someone that can provide feedback of integrating our stuff in their pre-exising CI system

Right, and we have a pre-existing CI system and I have a desire to add more tests and to help you, but I feel excluded for some reason I can't fathom :)

If flow is right, and it's possible to just run the thing as a normal executable, we can of course do it

MattJ, in most CI systems, you _can_ do anything, with enough time spent. We've build something to take away much of that effort (hopefully), by providing neat integrations.

MattJ, if you are able to ramp-up a test prosody instance as part of your CI, then it should be as simple as "git clone && ./run" (of course, I simplify things here)

But if you're doing specific stuff with proprietary CI APIs then no

flow, sure, that's exactly what we do with our existing tests

ramping down is the hard part

I figured, the talk about CI was just red herring

So I was expecting this to be similar

The CI integrations are mostly wrappers for common stuff. Take input, install a runtime environment and its requirements, execute tests based on user input, parse results.

Zash, we've found that in most CI systems, the ramp down was not needed, as runners would simply be thrown away after execution anyway.

We actually ended up removing rampdown at some places, as it could only introduce failure for something that was ... basically pointless.

That may not be true for all pipelines, of course

That's not how BuildBot works

It's pretty backwards from all the modern container based CI in use today

so buildbot is similar to classic jenkins?

I do believe it's exactly Jenkins translated into Python

I am not a "container rule everything around me" person, but once we switched our CI to a container based one, the advantages became obvious

that was probably 8 years or so ago :D ✏

Next on our list is to create various container-based approaches to run these tests - we've simply not gotten there yet.

I think you only need to consider containerizing the servers, the integration test runner has little need for containerization as its java based (so java/the jvm is your container platform)

of course, one could also put it into a container and then do containers-in-containers (not kidding, that can be sensible in some cases)

we're not touching the servers.

I think what would be a huge step forward for XMPP interop (testing) is, if every server implementation also provided an official container (test) image

The only requirement that we basically have is: make sure that a server is network-reachable (and can be provisioned with users, either through inband or adhoc)

I'm not saying it is a bad idea - it is just something that we do not require.

We will provide containers with our tests, assuming that there are people out there that prefer to run a standard container format over a standard Java VM

containers that contain our test clienit/runners, I mean

(it'd also allow us to eventually build additional tests that are not written in Java, without requiring users to re-do their CI setup)

not written in Java, you must be referring to Scala 3 then :)

we have no immediate plans, but we did consider that it'd be nice to retain the same interface (from the CI's perspective) and be able to cram in whatever we need for test execution. Today, it's all Smack / SINT, but maybe we want to add something like aioxmpp, or something completely different.

Guus, MattJ: I opened https://github.com/buildbot/buildbot-website/pull/87

but, to re-iterate: we're looking for help from a server developer that (pending support for more environments in the near future) is using a CI system that we currently support, to verify that what we are producing in documentation and runtime feedback/output is ... sensible.

^this^ - feedback on the ease of integration and the quality of the feedback are core goals of the project On Guus' estimate, if it takes closer to 4 hours, rather than closer to 1 hour, we'd really want to know about that.

I am surpised that there seems to be a strong focus on a particular CI system. The one thing I've learned from Travis CI was to design CI in such a way that the CI system doesn't matter much…

There's no focus on a particular CI system at all. We've created tests and docs/tooling for those to be easily integrated in what we believe is ~80% of the existing tools used today (with all intentions to develop integrations for other scenarios in the near future), and are looking for an outsider to validate what we've built so far - again, with clear intentions to continue developing things further.

ok, the "is using a CI system that we currently support" part then confuses me

Ideally you would even want someone *not* using the CI system that you have used, to see if there are any challenges

at least, that is what \me thinks

probably I am missing something

we currently support ... every CI system used by a server project - except for maybe only Prosody at this stage.

we're also in no way finished - additional tooling / docs for other CI environments, including the one used by Prosody - will be added soon

what's the piece that is assumed missing for prosody's CI system?

but at this stage, we're just looking for some confirmation that the stuff that we _have_ build is not void of any major oversights. Fishbowler and me have been so neck-deep in this project, that we might be overlooking something obvious.

For Prosody, either a containerized version of the test runner or documentation on how to run tests on 'bare metal' will likely be beneficial.

so it's more missing documentation and not missing technology that makes prosody's CI system "unsupported"

possibly - we've simply not gotten around to that. I'm not sure if we end up documenting usage that's completely uncontainerized (as that has some drawbacks too - what is now a black box suddenly becomes part of an API of sorts) - it certainly is not out of the question.

I totally get that you want someone else to try it, what I don't get is that, even in the presence of prosody folks showing interest in the integration, you seem to have taken a strong position that it isn't ready for prosody

whereas all they seem need is a jvm to execute the integration test runner

prosody has IIRC already integration tests, so the infra to ramp up the server is already there

Exactly

what's missing is to excute the same thing with a different integration test runner ✏

and that should be trivial

I'm not saying that we're not going to build that

I'm just asking for help with stuff that we've already built.

ok I am saying you already have build that

that, sadly, does not seem to be very applicable to Prosody _het_.

yet*

that you have already build that seems to be one (major) thing where we have a different understanding

basically, replace https://hg.prosody.im/trunk/file/tip/GNUmakefile#l114 with sinttest and done

no ci, no container foo, it's like a dream :) ✏

The test implementations is not what I'm looking for today. Apart from writing test implementations, we have compiled a lot of other things (documentation, configuration, examples, stuff that generates platform-specific output) that are very specific to CI environments. Those are the things that I'm today looking for feedback on.

Don't get me wrong, I'd also love to hear if our tests run successfully on Prosody - but that will open a completely parallel line of feedback

ok, I see

specifically, one of the questions that I have is if when a test fails, if a developer can figure out why it fails. I'm looking, for example, for confirmation if the very specific way that we've currently configured sinttest to execute within our CI abstractions, combined with our documenation and configuration examples, are sufficient, or need to be improved.

> I am thinking "simply wrap the XMPP servers into containers (or a "local" temp installation), wrap a script around it, that starts a server cointainer, points the test suite at it and repeats this for every server implementation" That's exactly what I do here https://github.com/moparisthebest/xmpp-proxy/blob/master/integration/test.sh I'm with MattJ though, make it trivial to run from a terminal then a few examples of integrating into popular at this moment CIs is all that is needed ↺

What happens if a test fails?

As an example, we've been using aioxmpp for integration tests. It has flagged a couple of interesting bugs, and was very valuable.

what was an absolute pain to me is trying to figure out _why_ a test fails

first, you have to comb through the output to figure out what test fails, then you have to find the corresponding implementation, try to interpret that (which isn't easy if it's writting in a language that you're not familiar with) and then try and deduce what's going wrong.

we've tried to make that all a lot easier, by - hopefully - give you tools to not have to look at the test implementation at all. On the one hand, we've put effort in publishing per-test XMPP stanza output. We've also added references to as specific as possible the specification that each particular test is failing. Also, we've modified each test assertion in such a way that they hopefully 'make sense' when looking at them without looking at the test implementation.

that's all hooked together in our test runners - that probably _can_ be made avialable too as a stand-alone runner (but we haven't built that yet).

That's what we're asking for help with - if the tooling that we've provided is adequate to consume the output of the executed tests - which maybe is more than just execute them and be done with it.

(and, as we've bundled that up in CI-specific runners, I'm looking for someone that uses one of the CIs that we already support)