XMPP Service Operators - 2018-11-21

rate == still 0.

Maranda: you solved Jabber spam?

he disabled s2s

Ge0rG, I'm not sure how appropriate that "solved" acceptation is, but receive rate is 0, it doesn't manage to get through.

Maranda: there are two relevant metrics in a filter, false acceptance rate and false rejection rate. The former is 0, which is good, but what about the latter?

edhelas dunno about you, you may need to do that, but I'm reknowingly a tid smarter than a monkey ;)

Ge0rG, it's 0.

Maranda: you have a spam filter that doesn't block any legitimate messages, but all spam? You should be a billionnaire by now.

maybe he's the one generating all this spam 🤔

Ge0rG, I should? Good to know.

edhelas, maybe you should just focus on getting Movim to work decently on all platform instead of making stupid statements ;).

Maranda: you can rent your spam filter out, to desperate people like me, who maintain complex spam filter lists and still end up with dozens of false positives every day

Maranda: maybe edhelas is not the one making stupid statements ;)

He for sure is Ge0rG, but that's the late fashion, pick every statement Maranda makes for how truthful it may be isn't it?

Alas like what I use and its concept wasn't published and available already.

Maranda: see, I'm doing some work on blocking XMPP spam, and I have a feeling for the complexity and the trade-offs, and there is no 100% solution to it.

Ge0rG, currently there's a 100% solution for it as bots on XMPP are very dumb, the fact that you or your users may not like the trade-offs for it is *yet* another matter.

Maranda: so either your solution is not generally applicable or you must be cheating your numbers.

yes, the bots are dumb.

At least most of them.

Maranda: so where can I read up on your solution?

If I use it on lightwitch.org it's very applicable

just look at mod_spim_block Ge0rG it's no secret or rocket science, and I don't see why I should cheat on numbers if I say it reduced my spim rate (lightwitch.org's) to 0 why should I be lying? The only complaint you could make is that my numbers are a bit limited to make a valid sample, not that I'm cheating on 'em.

Maranda: is there documentation for how mod_spim_block works?

Maranda: I'm not saying you cheat on the false negative rate, I'm saying you cheat on the false positive rate.

How many people don't see the spim blacklist bounce? Or don't bother jumping through the hoops?

Maranda: if you block s2s on excessive spim, how do you know that no legitimate users on that blocked server want to talk to your users?

all of that goes into the false positive rate.

Ge0rG but that "didn't" currently happen on lightwitch.org...? ;) that's a 95% presumption but spim only source triggers the ban for 1h, I have a rather large allowance for spim hits (and you could disable that) ✏

Maranda: so you don't know for sure. ✏

100% I can't if a ban triggers Ge0rG, I can only make an assumption on what gets banned the servers involved and the number I can gather from logs (in conformance with my privacy policy), but since it's months and months of data. They're very accurate, I sort of know the rate of remote messaging traffic and to which servers they usually go and come from on my *small* server. And when I see spim originating from a remote server which usually carries legit traffic I can act accordingly. ✏

Maranda: all I wanted to say is that your solution isn't perfect either.

it's just making different trade-offs.

Ge0rG, I never said it is, I said that the rate I get is 0, if you want I can change that the false positive is *very likely* 0 but that doesn't change the fact that if you disabled the s2s ban that would really turn into 0.

at least for the moment.

and for the area of coverage of mod_spim_block.

Maranda: the false positive rate is about legitimate messages blocked as spam. So disabling s2s would turn that to 100%

erm "how disabling the s2s ban" equates to "disabling s2s" Ge0rG..?

oh, sorry. Misread you on that.

Maranda: you need to also account for all the legitimate remote users who tried to contact one of your users, got rejected and gave up at that moment.

or didn't manage to click the link / do the captcha / whatever you have there.

or didn't receive the error.

Ge0rG, the server will send a readable message with instructions to the user, if I have to take in account for "literal monkeys" or people not willing to solve the challenge then that number can never be zero, but that's stupid imho, I made it so that even as annoying as it may be humans will always be able to solve the challenge.

Maranda: I'd like to test that, what do I need to do?

Maranda: will it be triggered by a subscription request or a message? do you have a JID I can test it on?

send me a message or presence sub to maranda@lightwitch.org

> Greetings, this is the lightwitch.org server before sending a message or presence subscription to this user, please visit https://meaveen.lightwitch.org/spim/ and input the following code in the form: jB6v1S/+rm/GklakBaPHvcYX7Rg= Is that a one or an L?

Maranda: so you just lost all people who don't know how to copy&paste base64 blobs. Also people using a client without partial copy&paste

do you want me to add "copy & paste", I wondering are you just arguing for the heck of it :P?

> What's the result of the following operation, 3 multiplicated by 21: Also rather high requirements on math skills

Maranda: I'm saying that what you've made is a nerd filter

Ge0rG, high requirements :P?

Maranda: multiplication of numbers >10 is a high requirement, unfortunately

Ge0rG, tbh the only high requirement I knew of was division of any value not addition, subtraction, multiplication of upto 2 digits values ✏

Maranda: you could at least put the token into the URL parameter and let the user only fill out the math.

Maranda: you also lost everybody who doesn't speak english

Ge0rG, that removes a step, so it's not possible.

Maranda: it removes what step?

Ge0rG, challenge step, not that it's really a step but it's enough for most dumb XMPP bots, and xmpp spammers currently do not wish to add a custom parser for my challenges apparently (except some russians who did after like a year or so)

Maranda: it remains a nerd filter, not a spam filter.

Ge0rG, whatever.

Maranda: does the filter apply if you contact me first and I respond?

Ge0rG, if I send you a message you get whitelisted automatically

Maranda: also if you send a subscription request?

Hmm only a subscription no, but usually I noticed most clients sending a message together with the subscription so in that case it's covered.

Maranda: it's generally a good idea, but I think you are not hitting the right usability trade-off.

I guess that's a bug.

Maranda: if I were to implement it, I'd use a URL with a token parameter (no need to copy-paste), maybe even skip out the math filter, just "click here to unlock", and maybe do some User-Agent / IP testing

Ge0rG, my users are satisfied with the "nerd filter", I got some complaints here and there but by most they all satisfied by not getting SPIM.

and I'd only apply the filter on bad-reputation domains and messages with suspected spam content.

Maranda: because they don't know anything better :P

Ge0rG, possibly, but there're no tools, like a centralised bad reputantion domain/ip list mantained by the XSF. So I offer what I can and have the time to mantain the offer of.

I'm automatically blocking almost all incoming spam (plus some legitimate traffic) without legitimate users needing to do anything

Maranda: https://github.com/ge0rg/jabber-spam-fighting-manifesto/blob/blacklist/blacklist.md

... ;)

Maranda: ??? ;)

(empty)

:P

Maranda: the list? yeah.

Maranda: still polishing the rules.

Maranda: also reporting abuse to server admins is real work™

Maranda: I'd be glad if you would like to participate in that effort

Ge0rG, if it's not too time taking to partecipate, like sending a PR everytime (e.g. like having a proper RESTful API for submission, checking)

Maranda: we've got an internal ticket system to track progress when talking to ISPs

Ge0rG, yes but I meant the *how to submit entries* before they're evaluated, *I'd* like to do that automatically after x violations rather than having to do manual aggregation and then sending.

Maranda: how to submit entries to the blacklist? That needs to be manual.

I mean that's how it mostly (without the evaluation part) work with the e-mail blacklists.

Maranda: so you volunteer to develop the required tooling?

Ge0rG, I barely have time and will for Metronome atm, so I guess that gives you a hint :P

‎[11:27:05] ‎Ge0rG‎: Maranda: also if you send a subscription request? --> nm, also if a contact is pending that's covered.

Maranda: it would be great to have all that written in the docs for that module.

Maranda: and if you insist on copy&pasting tokens, please encode the JID or a hash of it or some other token into the URL and reduce the thing that actually needs to be typed / copy-pasted to six digits numeric

Documentation is yet time dependant and also english skills dependant. Two things I'm not very proficent on.

Ge0rG: I could though change digest method / reduce entropy fetched to make the token shorter.

generate TOTP tokens!

TOTP is a bit too much hassle to do correctly, safely and avoiding collisions for what I'm doing for now

I'm just saying.

Ge0rG, but I can reduce the token to 12 characters and make it use uppercase letters only without issues (on uniqueness and collisions also)

Maranda: you could just append it to the URL.

with only uppercase letters, you probably also get rid of the 1/l/I issue.

A new server: deshalbfrei.org

Neustradamus: reported to the owner

A good news: https://twitter.com/neustradamus/status/1065328474922061825

@uptime lightwitch.org

Maranda: lightwitch.org has been running for 13 days, 21 hours and 26 minutes

Neustradamus: you da bot :))

Neustradamus: do the PEP changes mean that Daniel's omemo_all_access is integrated?

Licaon_Kter, it was a hack, and is not needed anymore.

Link Mauve: great