XSF Discussion - 2017-12-19

jonasw, could you hit the the merge button on my protoxep when you have a minute so it makes the 24h notice window for the council agenda tomorrow?

sure!

didn’t see the mail, sorry

jonasw: thank you

daniel, mail sent :-)

What's the difference between {xep 0363} and {xep 0370}? Both can do more or less the same thing, right? 0370 doesn't require a server component but it would be feasible as well I think

pep.: Multiple matches: Stanza Headers and Internet Metadata https://xmpp.org/extensions/xep-0131.html Ad-Hoc Commands https://xmpp.org/extensions/xep-0050.html Dialback Key Generation and Validation https://xmpp.org/extensions/xep-0185.html Out of Band Data https://xmpp.org/extensions/xep-0066.html Blocking Command https://xmpp.org/extensions/xep-0191.html Atomically Compare-And-Publish PubSub Items https://xmpp.org/extensions/xep-0395.html Jingle In-Band Bytestreams Transport Method https://xmpp.org/extensions/xep-0261.html A Transport for Initiating and Negotiating Sessions (TINS) https://xmpp.org/extensions/xep-0111.html In-Band Real Time Text https://xmpp.org/extensions/xep-0301.html Spim Markers and Reports https://xmpp.org/extensions/xep-0287.html Out-of-Band Stream Data https://xmpp.org/extensions/inbox/outofband.html File Repository and Sharing https://xmpp.org/extensions/xep-0214.html Jingle In-Band Bytestreams Transport https://xmpp.org/extensions/inbox/jingle-ibb.html Stanza Interception and Filtering Technology https://xmpp.org/extensions/inbox/sift.html Mandatory-to-Implement Technologies for Jingle RTP Sessions https://xmpp.org/extensions/inbox/jingle-rtp-mti.html Spim Markers and Reports https://xmpp.org/extensions/inbox/spim.html Field Standardization for Data Forms https://xmpp.org/extensions/xep-0068.html The /me Command https://xmpp.org/extensions/xep-0245.html Stanza Interception and Filtering Technology (SIFT) https://xmpp.org/extensions/xep-0273.html In-Band Bytestreams https://xmpp.org/extensions/xep-0047.html XMPP Date and Time Profiles https://xmpp.org/extensions/xep-0082.html In-Band Registration https://xmpp.org/extensions/xep-0077.html Unique and Stable Stanza IDs https://xmpp.org/extensions/xep-0359.html Best Practices for Handling Offline Messages https://xmpp.org/extensions/xep-0160.html Extensible In-Band Registration https://xmpp.org/extensions/xep-0389.html Incident Handling https://xmpp.org/extensions/xep-0268.html Impact of TLS and DNSSEC on Dialback https://xmpp.org/extensions/xep-0344.html Form Discovery and Publishing https://xmpp.org/extensions/xep-0346.html Zero Handshake Server to Server Protocol https://xmpp.org/extensions/xep-0361.html Out-of-Band Stream Data https://xmpp.org/extensions/xep-0265.html Invisible Command https://xmpp.org/extensions/xep-0186.html Best Practices for Roster and Subscription Management https://xmpp.org/extensions/xep-0162.html Atomically Compare-And-Publish PubSub Items https://xmpp.org/extensions/inbox/cap.html

Wat

Nice

Maybe it did a search for "0363} and {xep 0370" and thought you really wanted all XEPs with the word "and" anywhere in them

Two {} commands at the same time don't work anyways

:(

-xep 0363

pep.: HTTP File Upload (Standards Track, Proposed, 2017-12-03) See: https://xmpp.org/extensions/xep-0363.html

-xep 0370

pep.: Jingle HTTP Transport Method (Standards Track, Deferred, 2017-09-11) See: https://xmpp.org/extensions/xep-0370.html

pep.: compare {xep 65} and ...

pep.: SOCKS5 Bytestreams (Standards Track, Draft, 2015-09-17) See: https://xmpp.org/extensions/xep-0065.html

-xep 260

Zash: Jingle SOCKS5 Bytestreams Transport Method (Standards Track, Draft, 2016-05-17) See: https://xmpp.org/extensions/xep-0260.html

Heh

Holy carp, do we have a bazillion XEPs mentioning "jingle"

Is there any implementation of 0370?

we need no carpy jingle

Ge0rG, what is the input field on your easy-xmpp-invitation website for?

I think that we have to start to work on spam issues seriously

we have to see how people are spamming XMPP today on a wider scale than just blocking a type of message

Oh, is it a new wave?

Link Mauve it basically never stopped

lena2521@jabber.uznam.net.pl

ya, SPAM is getting horrible, in the mood of shutting down my XMPP clients :(

Might be our anti-spam solution working great then. ^^

madaline2784@i0i0.de

for now

edhelas, it brings nothing to ban individual JIDs to the network.

it's super easy to create a new host on your XMPP serve, put a let's encrypt certificate on it and start spamming the others

yes I know

Alex, if you’re running Prosody, https://yaxim.org/blog/2017/12/12/spam-reduction-on-yax-dot-im/

what I'd like to do is have a look at all those domains, see if they are pointing to same IPs

and basically figure out from where those SPAMs are coming

but we seriously have to take the problem

edhelas, pretty sure most spammers are currently relying on insecure public infrastructure rather than using their own servers.

edhelas, just read that blog article, that’s how we’ve been handling it at JabberFR for the past year or so.

that's not exactly true

I have more spam comming from new domains than known ones

edhelas, can you share some example domains?

well I just did

There are multiple spammers, they use different techniques, and not everyone will have their JID on every spammer's list

I can give you a longer list, give me a bit of time

edhelas, blocking by domain is almost never a solution.

So while Link Mauve and I receive spam from existing domains, maybe you experience it more from a spammer who uses new domains

MattJ that's what i'm saying, we have different kind of spams, we have to differentiate them and see how we can block them

I should log all of the spams I’m blocking, to get better statistics.

well hopefully I have MAM and Movim cache for that

also, again, for now we have simple SPAM messages

Ge0rG, I’m interested in your statistics module btw.

I start to have roster subscriptions request from spammers

and the next kind will be Pubsub publications I think

so I'm already talking with ejabberd dev to put quotas and limitations on Pubsub

Wouldn’t reach nearly as many people as plain messages.

> and the next kind will be Pubsub publications I think lol that's unnecessarily complex. normal messages will reach people just fine

I'm deleting one or two accounts of users that are publishing articles with links to weird urls and warez places :)

but yes this is pretty long term

also; while i'm not denying that spam is a problem it is not really for the average user; normal users don't publish their jid everywhere. the lists the spammers are using are (in parts) very old. while *we* see a lot of spam normal users don't

yes

yes it creates load and servers and requires work from sys admins; but we shouldn't necessarily let this distract us from other problems

especially since normal users - and even we - are not the target audience of the spam

we don't speak russian and we don't usually buy stolen credit cards

ok; some of us speak russian…

ok I've compiled a list of JID that are sending messages to Movim users without been in their roster

some statistics per server :)

https://movim.eu:5280/upload/9d94237298995552fa13436420195fbca436dce7/zGvBJ61KKHv40YHUDv4obvA5SKUlfxBgfzCH3V3e/spam_servers.txt

marc: it's not an input field, it's a text selection field where you can copy the xmpp: URI from

Ge0rG, https://projects.zapb.de/tmp/easy-xmpp-invitation/

I can edit the xmpp URI

Link Mauve: thank you for sharing a private link, btw.

Oh, I saw it in public somewhere today, sorry. :x

Link Mauve: it isn't published yet, and I'm in the middle of refactoring it into a technical post on my personal blog and a high-level post on yaxim.org

which technically means I've stopped working on it for now ;)

I didn’t see any mention that it was a draft or anything either.

Link Mauve: I'm sure I wrote it's a draft where I posted it.

Damn…

Sorry. :x

You can still yank it out probably.

Link Mauve: I'm sure nobody will notice

let's hope it won't get picked up. yet.

Link Mauve: but now you made me curious, you are not a member of the places I posted it in. :P

I’m already looking for the place I got it from. ^^

Sigh. advert364@yax.im - 1600 outgoing subscription requests.

MattJ: can we have presence blocking in mod_firewall please?

"presence blocking"?

KIND: presence DROP.

Presence. Blocked.

XMP

MattJ: I'm sure users will love this.

They'll love the simplicity, which is good for UX

MattJ: I want to block/revert presence subscriptions from accounts that are spamming.

Incoming or outgoing?

MattJ: both

I've deleted some 2000 spammer accounts in the last weeks

Link Mauve: the stats for that blog non-post I gathered by grepping prosody.log for JIDs captured by mod_firewall spammer.pfw

I don't think a server should allow a new account to *have* 1600 open subscription requests

MattJ: that's a great idea.

Oh.

Link Mauve: not sure if this will help you much. `zgrep -ho 'spam:.*message.*' /var/log/prosody/prosody.log*|grep -ho "from='[a-z0-9._-]*@[^']*'" > 2017-12-13-alljids.txt`

MattJ: the other spam accounts I deleted have between 10 and 200 pending subscriptions. Which is still too much.

Ge0rG you have IBR ?

MattJ: but the worst thing is that I'm getting a dozen a subscriptions a day.

edhelas: yes

well then you know where they are coming from

Ge0rG, it’s super weird, I can’t find any mention of this blog post in my logs before I posted it. oO

on my side I'm starting to blacklist list of domains for s2s

edhelas: from Tor and open proxies

wut, is that url private? it seems awfully public to me and I've seen it around already

Ge0rG: Maybe it would be clearer to say you wanna retract presence subscription requests from mod_firewall

Zash: yes, that's exactly what I want. I'm sure I asked for that already one or two times

pep.: I can't imagine how it made the rounds, or where

the spam reduction article right?

I thought I saw you ask for "blocking"

I'm confused

Zash: to block them I need to know they are spammers before they send their spam

Maybe it would be simpler by writing another module, that would hook into the event fired by mod_firewall on someone being flagged as a spammer, and then proceed to remove its presence subscription.

pep., do you remember where you saw it?

Because grep doesn’t help. :/

Zash: ideally it should delay incoming subscriptions for a minute and just discard them if the user sends incoming spam

Ge0rG, one second would be enough currently.

So kind of like a bastard of mod_smacks and mod_csi_pump

Hm, that sounds a bit tricky for mod_firewall?

Or? What sayeth MattJ?

Some kind of tarpit has always been on my mind for mod_firewall, but it is indeed tricky

I'm okay with a separate module if it helps tame subscription spam

Ah, hmm, no I've never read it. Dec 12th 2017

Ge0rG, could you have used an URL shortener?

is it risky to publish publicly my list of blacklisted s2s servers ?

edhelas: I don't think so

I'd like to be transparent regarding my configuration

edhelas, it’s a good thing, it will allow them to start the process to get un-blacklisted.

edhelas, might also be nice to send a message to the contact address of that server when you blacklist it

Indeed.

meh

why?

Of course you can automate that

step by step :)

first publish the list

MattJ: currently I'm actively monitoring prosody.log for outgoing spam, listing all accounts registered from the same ip as the perpetrator, checking whether any of them have proper roster subscriptions and deleting all that look like spammers.

I've automated most of the steps so it boils down to copying a JID and a list of user:delete commands, but the monitoring itself is tedious and in theory easy to automate

To automate that, I'd fire an event from the firewall "reject spam" chain, and just handle the rest in a module

At least log the IP to a separate file

MattJ: yes, it would help to have a quarantine flag on accounts that could be set this way

Hmm

I've pulled a number for that one recently

MattJ: but none of this solves the incoming subscriptions problem

It merely reduces the outgoing subscriptions problem, slightly

As I said, I've deleted around 2k accounts so far.

Automatically rejecting a pending incoming subscription should be pretty straightforward to add as an action

And one real user, by accident. Which is why I want a quarantine flag that's less terminal than a deletion

Any "hold the stanza for X seconds/minutes" is full of performance and correctness problems

MattJ: yes, I know. But holding a stanza long enough to check the next stanza from the same JID might actually work without melting the server

We had a lot of discussion about this when you first had the idea

Nice idea, but the spammer only has to wait N+1 seconds

and they have plenty of time on their hands

MattJ: maybe they do, maybe not.

Whatever you choose for N, they can wait longer, and as you increase N you're going to effectively open yourself up to DoS attacks

Greylisting has turned out to work exceptionally well for email

MattJ: besides, even if they wait, they won't get past the spam filter, so they have no incentive to modify their code

What's their incentive for sending the subscription request in the first place?

btw, I don't see anything about your account quarantine flag: https://prosody.im/issues/?q=state%3Dopen+firewall

MattJ: I suppose it's too trick dumb clients / servers to accept the following message

MattJ: https://prosody.im/issues/1057

MattJ: my firewall blocks all spam messages anyway, so they won't gain anything by waiting longer

MattJ: please feel free to suggest a different method to mitigate the incoming subscriptions.

I think we ultimately ended up at UI changes on the clients when we last discussed this

That is, a subscription request should not be "noisy"

MattJ: I would accept a subscription denial from the firewall as a first step.

Yes, that can be done

MattJ: except that we haven't implemented anything after the discussion, and are repeating it now.

I'll get you your account flagging thing, which will at least help to improve your current process

and then I'll get you automated rejection/retraction of subscription requests

MattJ: thanks, that's awesome!

The tarpit thing may happen one day, or it may never happen

It's a nice idea with too many practical issues

MattJ: what about making the flagging depend on the number of roster items the user has? I.e. when pending >(to+both)

Simply because you have to queue every stanza for the same destination JID following a match, and they can send to an unbounded number of destination JIDs

MattJ: I'm sure we can also stop incoming mass subscriptions from the same JID

Just not from the same server...

Ge0rG, they'll just add bot accounts to bot accounts rosters

MattJ: maybe

MattJ: but yes, das things first please.

Good night

s/das/easy/ it's too late for auto completion