-
daniel
jonasw, could you hit the the merge button on my protoxep when you have a minute so it makes the 24h notice window for the council agenda tomorrow?
-
jonasw
sure!
-
jonasw
didn’t see the mail, sorry
-
daniel
jonasw: thank you
-
jonasw
daniel, mail sent :-)
-
pep.
What's the difference between {xep 0363} and {xep 0370}? Both can do more or less the same thing, right? 0370 doesn't require a server component but it would be feasible as well I think
-
Bunneh
pep.: Multiple matches: Stanza Headers and Internet Metadata https://xmpp.org/extensions/xep-0131.html Ad-Hoc Commands https://xmpp.org/extensions/xep-0050.html Dialback Key Generation and Validation https://xmpp.org/extensions/xep-0185.html Out of Band Data https://xmpp.org/extensions/xep-0066.html Blocking Command https://xmpp.org/extensions/xep-0191.html Atomically Compare-And-Publish PubSub Items https://xmpp.org/extensions/xep-0395.html Jingle In-Band Bytestreams Transport Method https://xmpp.org/extensions/xep-0261.html A Transport for Initiating and Negotiating Sessions (TINS) https://xmpp.org/extensions/xep-0111.html In-Band Real Time Text https://xmpp.org/extensions/xep-0301.html Spim Markers and Reports https://xmpp.org/extensions/xep-0287.html Out-of-Band Stream Data https://xmpp.org/extensions/inbox/outofband.html File Repository and Sharing https://xmpp.org/extensions/xep-0214.html Jingle In-Band Bytestreams Transport https://xmpp.org/extensions/inbox/jingle-ibb.html Stanza Interception and Filtering Technology https://xmpp.org/extensions/inbox/sift.html Mandatory-to-Implement Technologies for Jingle RTP Sessions https://xmpp.org/extensions/inbox/jingle-rtp-mti.html Spim Markers and Reports https://xmpp.org/extensions/inbox/spim.html Field Standardization for Data Forms https://xmpp.org/extensions/xep-0068.html The /me Command https://xmpp.org/extensions/xep-0245.html Stanza Interception and Filtering Technology (SIFT) https://xmpp.org/extensions/xep-0273.html In-Band Bytestreams https://xmpp.org/extensions/xep-0047.html XMPP Date and Time Profiles https://xmpp.org/extensions/xep-0082.html In-Band Registration https://xmpp.org/extensions/xep-0077.html Unique and Stable Stanza IDs https://xmpp.org/extensions/xep-0359.html Best Practices for Handling Offline Messages https://xmpp.org/extensions/xep-0160.html Extensible In-Band Registration https://xmpp.org/extensions/xep-0389.html Incident Handling https://xmpp.org/extensions/xep-0268.html Impact of TLS and DNSSEC on Dialback https://xmpp.org/extensions/xep-0344.html Form Discovery and Publishing https://xmpp.org/extensions/xep-0346.html Zero Handshake Server to Server Protocol https://xmpp.org/extensions/xep-0361.html Out-of-Band Stream Data https://xmpp.org/extensions/xep-0265.html Invisible Command https://xmpp.org/extensions/xep-0186.html Best Practices for Roster and Subscription Management https://xmpp.org/extensions/xep-0162.html Atomically Compare-And-Publish PubSub Items https://xmpp.org/extensions/inbox/cap.html
-
pep.
Wat
-
Zash
Nice
-
Zash
Maybe it did a search for "0363} and {xep 0370" and thought you really wanted all XEPs with the word "and" anywhere in them
-
Zash
Two {} commands at the same time don't work anyways
-
pep.
:(
-
pep.
-xep 0363
-
Bunneh
pep.: HTTP File Upload (Standards Track, Proposed, 2017-12-03) See: https://xmpp.org/extensions/xep-0363.html
-
pep.
-xep 0370
-
Bunneh
pep.: Jingle HTTP Transport Method (Standards Track, Deferred, 2017-09-11) See: https://xmpp.org/extensions/xep-0370.html
-
Zash
pep.: compare {xep 65} and ...
-
Bunneh
pep.: SOCKS5 Bytestreams (Standards Track, Draft, 2015-09-17) See: https://xmpp.org/extensions/xep-0065.html
-
Zash
-xep 260
-
Bunneh
Zash: Jingle SOCKS5 Bytestreams Transport Method (Standards Track, Draft, 2016-05-17) See: https://xmpp.org/extensions/xep-0260.html
-
pep.
Heh
-
Zash
Holy carp, do we have a bazillion XEPs mentioning "jingle"
-
pep.
Is there any implementation of 0370?
-
zinid
we need no carpy jingle
-
marc
Ge0rG, what is the input field on your easy-xmpp-invitation website for?
-
edhelas
I think that we have to start to work on spam issues seriously
-
edhelas
we have to see how people are spamming XMPP today on a wider scale than just blocking a type of message
-
Link Mauve
Oh, is it a new wave?
-
edhelas
Link Mauve it basically never stopped
-
edhelas
lena2521@jabber.uznam.net.pl
-
Alex
ya, SPAM is getting horrible, in the mood of shutting down my XMPP clients :(
-
Link Mauve
Might be our anti-spam solution working great then. ^^
-
edhelas
madaline2784@i0i0.de
-
edhelas
for now
-
Link Mauve
edhelas, it brings nothing to ban individual JIDs to the network.
-
edhelas
it's super easy to create a new host on your XMPP serve, put a let's encrypt certificate on it and start spamming the others
-
edhelas
yes I know
-
Link Mauve
Alex, if you’re running Prosody, https://yaxim.org/blog/2017/12/12/spam-reduction-on-yax-dot-im/
-
edhelas
what I'd like to do is have a look at all those domains, see if they are pointing to same IPs
-
edhelas
and basically figure out from where those SPAMs are coming
-
edhelas
but we seriously have to take the problem
-
Link Mauve
edhelas, pretty sure most spammers are currently relying on insecure public infrastructure rather than using their own servers.
-
Link Mauve
edhelas, just read that blog article, that’s how we’ve been handling it at JabberFR for the past year or so.
-
edhelas
that's not exactly true
-
edhelas
I have more spam comming from new domains than known ones
-
MattJ
edhelas, can you share some example domains?
-
edhelas
well I just did
-
MattJ
There are multiple spammers, they use different techniques, and not everyone will have their JID on every spammer's list
-
edhelas
I can give you a longer list, give me a bit of time
-
Link Mauve
edhelas, blocking by domain is almost never a solution.
-
MattJ
So while Link Mauve and I receive spam from existing domains, maybe you experience it more from a spammer who uses new domains
-
edhelas
MattJ that's what i'm saying, we have different kind of spams, we have to differentiate them and see how we can block them
-
Link Mauve
I should log all of the spams I’m blocking, to get better statistics.
-
edhelas
well hopefully I have MAM and Movim cache for that
-
edhelas
also, again, for now we have simple SPAM messages
-
Link Mauve
Ge0rG, I’m interested in your statistics module btw.
-
edhelas
I start to have roster subscriptions request from spammers
-
edhelas
and the next kind will be Pubsub publications I think
-
edhelas
so I'm already talking with ejabberd dev to put quotas and limitations on Pubsub
-
Link Mauve
Wouldn’t reach nearly as many people as plain messages.
-
daniel
> and the next kind will be Pubsub publications I think lol that's unnecessarily complex. normal messages will reach people just fine
-
edhelas
I'm deleting one or two accounts of users that are publishing articles with links to weird urls and warez places :)
-
edhelas
but yes this is pretty long term
-
daniel
also; while i'm not denying that spam is a problem it is not really for the average user; normal users don't publish their jid everywhere. the lists the spammers are using are (in parts) very old. while *we* see a lot of spam normal users don't
-
edhelas
yes
-
daniel
yes it creates load and servers and requires work from sys admins; but we shouldn't necessarily let this distract us from other problems
-
daniel
especially since normal users - and even we - are not the target audience of the spam
-
daniel
we don't speak russian and we don't usually buy stolen credit cards
-
daniel
ok; some of us speak russian…
-
edhelas
ok I've compiled a list of JID that are sending messages to Movim users without been in their roster
-
edhelas
some statistics per server :)
-
edhelas
https://movim.eu:5280/upload/9d94237298995552fa13436420195fbca436dce7/zGvBJ61KKHv40YHUDv4obvA5SKUlfxBgfzCH3V3e/spam_servers.txt
-
Ge0rG
marc: it's not an input field, it's a text selection field where you can copy the xmpp: URI from
-
marc
Ge0rG, https://projects.zapb.de/tmp/easy-xmpp-invitation/
-
marc
I can edit the xmpp URI
-
Ge0rG
Link Mauve: thank you for sharing a private link, btw.
-
Link Mauve
Oh, I saw it in public somewhere today, sorry. :x
-
Ge0rG
Link Mauve: it isn't published yet, and I'm in the middle of refactoring it into a technical post on my personal blog and a high-level post on yaxim.org
-
Ge0rG
which technically means I've stopped working on it for now ;)
-
Link Mauve
I didn’t see any mention that it was a draft or anything either.
-
Ge0rG
Link Mauve: I'm sure I wrote it's a draft where I posted it.
-
Link Mauve
Damn…
-
Link Mauve
Sorry. :x
-
Link Mauve
You can still yank it out probably.
-
Ge0rG
Link Mauve: I'm sure nobody will notice
-
Ge0rG
let's hope it won't get picked up. yet.
-
Ge0rG
Link Mauve: but now you made me curious, you are not a member of the places I posted it in. :P
-
Link Mauve
I’m already looking for the place I got it from. ^^
-
Ge0rG
Sigh. advert364@yax.im - 1600 outgoing subscription requests.
-
Ge0rG
MattJ: can we have presence blocking in mod_firewall please?
-
MattJ
"presence blocking"?
-
MattJ
KIND: presence DROP.
-
MattJ
Presence. Blocked.
-
MattJ
XMP
-
Ge0rG
MattJ: I'm sure users will love this.
-
MattJ
They'll love the simplicity, which is good for UX
-
Ge0rG
MattJ: I want to block/revert presence subscriptions from accounts that are spamming.
-
MattJ
Incoming or outgoing?
-
Ge0rG
MattJ: both
-
Ge0rG
I've deleted some 2000 spammer accounts in the last weeks
-
Ge0rG
Link Mauve: the stats for that blog non-post I gathered by grepping prosody.log for JIDs captured by mod_firewall spammer.pfw
-
MattJ
I don't think a server should allow a new account to *have* 1600 open subscription requests
-
Ge0rG
MattJ: that's a great idea.
-
Link Mauve
Oh.
-
Ge0rG
Link Mauve: not sure if this will help you much. `zgrep -ho 'spam:.*message.*' /var/log/prosody/prosody.log*|grep -ho "from='[a-z0-9._-]*@[^']*'" > 2017-12-13-alljids.txt`
-
Ge0rG
MattJ: the other spam accounts I deleted have between 10 and 200 pending subscriptions. Which is still too much.
-
edhelas
Ge0rG you have IBR ?
-
Ge0rG
MattJ: but the worst thing is that I'm getting a dozen a subscriptions a day.
-
Ge0rG
edhelas: yes
-
edhelas
well then you know where they are coming from
-
Link Mauve
Ge0rG, it’s super weird, I can’t find any mention of this blog post in my logs before I posted it. oO
-
edhelas
on my side I'm starting to blacklist list of domains for s2s
-
Ge0rG
edhelas: from Tor and open proxies
-
pep.
wut, is that url private? it seems awfully public to me and I've seen it around already
-
Zash
Ge0rG: Maybe it would be clearer to say you wanna retract presence subscription requests from mod_firewall
-
Ge0rG
Zash: yes, that's exactly what I want. I'm sure I asked for that already one or two times
-
Ge0rG
pep.: I can't imagine how it made the rounds, or where
-
pep.
the spam reduction article right?
-
Zash
I thought I saw you ask for "blocking"
-
pep.
I'm confused
-
Ge0rG
Zash: to block them I need to know they are spammers before they send their spam
-
Link Mauve
Maybe it would be simpler by writing another module, that would hook into the event fired by mod_firewall on someone being flagged as a spammer, and then proceed to remove its presence subscription.
-
Link Mauve
pep., do you remember where you saw it?
-
Link Mauve
Because grep doesn’t help. :/
-
Ge0rG
Zash: ideally it should delay incoming subscriptions for a minute and just discard them if the user sends incoming spam
-
Link Mauve
Ge0rG, one second would be enough currently.
-
Ge0rG
So kind of like a bastard of mod_smacks and mod_csi_pump
-
Zash
Hm, that sounds a bit tricky for mod_firewall?
-
Zash
Or? What sayeth MattJ?
-
MattJ
Some kind of tarpit has always been on my mind for mod_firewall, but it is indeed tricky
-
Ge0rG
I'm okay with a separate module if it helps tame subscription spam
-
pep.
Ah, hmm, no I've never read it. Dec 12th 2017
-
Link Mauve
Ge0rG, could you have used an URL shortener?
-
edhelas
is it risky to publish publicly my list of blacklisted s2s servers ?
-
Ge0rG
edhelas: I don't think so
-
edhelas
I'd like to be transparent regarding my configuration
-
Link Mauve
edhelas, it’s a good thing, it will allow them to start the process to get un-blacklisted.
-
pep.
edhelas, might also be nice to send a message to the contact address of that server when you blacklist it
-
Link Mauve
Indeed.
-
edhelas
meh
-
pep.
why?
-
pep.
Of course you can automate that
-
edhelas
step by step :)
-
edhelas
first publish the list
-
Ge0rG
MattJ: currently I'm actively monitoring prosody.log for outgoing spam, listing all accounts registered from the same ip as the perpetrator, checking whether any of them have proper roster subscriptions and deleting all that look like spammers.
-
Ge0rG
I've automated most of the steps so it boils down to copying a JID and a list of user:delete commands, but the monitoring itself is tedious and in theory easy to automate
-
MattJ
To automate that, I'd fire an event from the firewall "reject spam" chain, and just handle the rest in a module
-
MattJ
At least log the IP to a separate file
-
Ge0rG
MattJ: yes, it would help to have a quarantine flag on accounts that could be set this way
-
MattJ
Hmm
-
Ge0rG
I've pulled a number for that one recently
-
Ge0rG
MattJ: but none of this solves the incoming subscriptions problem
-
Ge0rG
It merely reduces the outgoing subscriptions problem, slightly
-
Ge0rG
As I said, I've deleted around 2k accounts so far.
-
MattJ
Automatically rejecting a pending incoming subscription should be pretty straightforward to add as an action
-
Ge0rG
And one real user, by accident. Which is why I want a quarantine flag that's less terminal than a deletion
-
MattJ
Any "hold the stanza for X seconds/minutes" is full of performance and correctness problems
-
Ge0rG
MattJ: yes, I know. But holding a stanza long enough to check the next stanza from the same JID might actually work without melting the server
-
MattJ
We had a lot of discussion about this when you first had the idea
-
MattJ
Nice idea, but the spammer only has to wait N+1 seconds
-
MattJ
and they have plenty of time on their hands
-
Ge0rG
MattJ: maybe they do, maybe not.
-
MattJ
Whatever you choose for N, they can wait longer, and as you increase N you're going to effectively open yourself up to DoS attacks
-
Ge0rG
Greylisting has turned out to work exceptionally well for email
-
Ge0rG
MattJ: besides, even if they wait, they won't get past the spam filter, so they have no incentive to modify their code
-
MattJ
What's their incentive for sending the subscription request in the first place?
-
MattJ
btw, I don't see anything about your account quarantine flag: https://prosody.im/issues/?q=state%3Dopen+firewall
-
Ge0rG
MattJ: I suppose it's too trick dumb clients / servers to accept the following message
-
Ge0rG
MattJ: https://prosody.im/issues/1057
-
Ge0rG
MattJ: my firewall blocks all spam messages anyway, so they won't gain anything by waiting longer
-
Ge0rG
MattJ: please feel free to suggest a different method to mitigate the incoming subscriptions.
-
MattJ
I think we ultimately ended up at UI changes on the clients when we last discussed this
-
MattJ
That is, a subscription request should not be "noisy"
-
Ge0rG
MattJ: I would accept a subscription denial from the firewall as a first step.
-
MattJ
Yes, that can be done
-
Ge0rG
MattJ: except that we haven't implemented anything after the discussion, and are repeating it now.
-
MattJ
I'll get you your account flagging thing, which will at least help to improve your current process
-
MattJ
and then I'll get you automated rejection/retraction of subscription requests
-
Ge0rG
MattJ: thanks, that's awesome!
-
MattJ
The tarpit thing may happen one day, or it may never happen
-
MattJ
It's a nice idea with too many practical issues
-
Ge0rG
MattJ: what about making the flagging depend on the number of roster items the user has? I.e. when pending >(to+both)
-
MattJ
Simply because you have to queue every stanza for the same destination JID following a match, and they can send to an unbounded number of destination JIDs
-
Ge0rG
MattJ: I'm sure we can also stop incoming mass subscriptions from the same JID
-
Ge0rG
Just not from the same server...
-
MattJ
Ge0rG, they'll just add bot accounts to bot accounts rosters
-
Ge0rG
MattJ: maybe
-
Ge0rG
MattJ: but yes, das things first please.
- Ge0rG is leaving for the night now. CU
-
MattJ
Good night
-
Ge0rG
s/das/easy/ it's too late for auto completion