jonasw, could you hit the the merge button on my protoxep when you have a minute so it makes the 24h notice window for the council agenda tomorrow?
jonasw
sure!
jonasw
didn’t see the mail, sorry
Kevhas joined
daniel
jonasw: thank you
stefandxmhas left
ralphmhas left
stefandxmhas joined
mimi89999has left
Steve Killehas left
jonasw
daniel, mail sent :-)
marchas joined
Steve Killehas left
pep.has left
ralphmhas left
Steve Killehas joined
Guushas joined
Guushas left
Guushas joined
zinidhas left
ralphmhas joined
moparisthebesthas joined
Guushas left
danielhas left
Alexhas joined
Guushas left
lskdjfhas joined
marchas left
ralphmhas joined
danielhas left
danielhas left
efrithas joined
Syndacehas left
Syndacehas joined
SouLhas left
uchas joined
ralphmhas left
matlaghas left
matlaghas left
matlaghas joined
sonnyhas left
SouLhas joined
Tobiashas joined
sonnyhas joined
ralphmhas joined
SouLhas joined
danielhas left
SouLhas joined
SouLhas joined
uchas joined
SamWhitedhas joined
SamWhitedhas joined
SouLhas joined
jcbrandhas joined
Alexhas left
SouLhas joined
danielhas left
tim@boese-ban.dehas joined
lumihas joined
SouLhas joined
Syndacehas joined
Syndacehas joined
pep.has joined
SouLhas joined
jcbrandhas left
jcbrandhas left
Ge0rGhas left
jcbrandhas left
jcbrandhas joined
vanitasvitaehas left
vanitasvitaehas joined
la|r|mahas joined
SouLhas joined
danielhas left
@Alacerhas left
@Alacerhas joined
efrithas left
@Alacerhas left
@Alacerhas joined
SouLhas left
Syndacehas left
Syndacehas joined
mrkikohas left
mrkikohas joined
Guushas left
SouLhas joined
SouLhas joined
pep.has left
Guushas left
SouLhas left
SouLhas left
tim@boese-ban.dehas joined
lumihas left
Tobiashas left
jjrhhas left
SouLhas joined
SouLhas joined
jubalhhas joined
ralphmhas joined
SouLhas joined
lovetoxhas joined
SouLhas joined
SamWhitedhas joined
SamWhitedhas joined
danielhas left
danielhas joined
jjrhhas left
jjrhhas left
SouLhas left
jjrhhas left
jjrhhas left
SouLhas left
jjrhhas left
Syndacehas left
Syndacehas joined
lskdjfhas joined
lskdjfhas joined
SamWhitedhas left
Alexhas joined
Guushas left
ralphmhas joined
SouLhas left
jcbrandhas left
Guushas left
SouLhas joined
matlaghas left
matlaghas joined
jubalhhas left
ralphmhas joined
SouLhas joined
pep.has left
SouLhas joined
SouLhas left
SouLhas left
efrithas joined
Holgerhas left
SouLhas left
efrithas left
efrithas joined
ralphmhas joined
zinidhas left
sonnyhas left
SouLhas joined
SouLhas joined
sonnyhas left
matlaghas left
SouLhas joined
SouLhas joined
lskdjfhas joined
ralphmhas joined
jubalhhas joined
jubalhhas left
ralphmhas left
SouLhas joined
Guushas left
Tobiashas joined
efrithas left
SouLhas joined
jubalhhas joined
jubalhhas left
Guushas left
SouLhas left
SouLhas left
danielhas left
SouLhas left
SouLhas joined
sonnyhas joined
danielhas left
Tobiashas joined
danielhas left
ralphmhas left
jubalhhas joined
SouLhas left
waqashas joined
SouLhas joined
SouLhas left
pep.
What's the difference between {xep 0363} and {xep 0370}? Both can do more or less the same thing, right? 0370 doesn't require a server component but it would be feasible as well I think
Bunneh
pep.: Multiple matches:
Stanza Headers and Internet Metadata https://xmpp.org/extensions/xep-0131.html
Ad-Hoc Commands https://xmpp.org/extensions/xep-0050.html
Dialback Key Generation and Validation https://xmpp.org/extensions/xep-0185.html
Out of Band Data https://xmpp.org/extensions/xep-0066.html
Blocking Command https://xmpp.org/extensions/xep-0191.html
Atomically Compare-And-Publish PubSub Items https://xmpp.org/extensions/xep-0395.html
Jingle In-Band Bytestreams Transport Method https://xmpp.org/extensions/xep-0261.html
A Transport for Initiating and Negotiating Sessions (TINS) https://xmpp.org/extensions/xep-0111.html
In-Band Real Time Text https://xmpp.org/extensions/xep-0301.html
Spim Markers and Reports https://xmpp.org/extensions/xep-0287.html
Out-of-Band Stream Data https://xmpp.org/extensions/inbox/outofband.html
File Repository and Sharing https://xmpp.org/extensions/xep-0214.html
Jingle In-Band Bytestreams Transport https://xmpp.org/extensions/inbox/jingle-ibb.html
Stanza Interception and Filtering Technology https://xmpp.org/extensions/inbox/sift.html
Mandatory-to-Implement Technologies for Jingle RTP Sessions https://xmpp.org/extensions/inbox/jingle-rtp-mti.html
Spim Markers and Reports https://xmpp.org/extensions/inbox/spim.html
Field Standardization for Data Forms https://xmpp.org/extensions/xep-0068.html
The /me Command https://xmpp.org/extensions/xep-0245.html
Stanza Interception and Filtering Technology (SIFT) https://xmpp.org/extensions/xep-0273.html
In-Band Bytestreams https://xmpp.org/extensions/xep-0047.html
XMPP Date and Time Profiles https://xmpp.org/extensions/xep-0082.html
In-Band Registration https://xmpp.org/extensions/xep-0077.html
Unique and Stable Stanza IDs https://xmpp.org/extensions/xep-0359.html
Best Practices for Handling Offline Messages https://xmpp.org/extensions/xep-0160.html
Extensible In-Band Registration https://xmpp.org/extensions/xep-0389.html
Incident Handling https://xmpp.org/extensions/xep-0268.html
Impact of TLS and DNSSEC on Dialback https://xmpp.org/extensions/xep-0344.html
Form Discovery and Publishing https://xmpp.org/extensions/xep-0346.html
Zero Handshake Server to Server Protocol https://xmpp.org/extensions/xep-0361.html
Out-of-Band Stream Data https://xmpp.org/extensions/xep-0265.html
Invisible Command https://xmpp.org/extensions/xep-0186.html
Best Practices for Roster and Subscription Management https://xmpp.org/extensions/xep-0162.html
Atomically Compare-And-Publish PubSub Items https://xmpp.org/extensions/inbox/cap.html
pep.
Wat
lskdjfhas joined
Zash
Nice
Zash
Maybe it did a search for "0363} and {xep 0370" and thought you really wanted all XEPs with the word "and" anywhere in them
jjrhhas left
Zash
Two {} commands at the same time don't work anyways
Holy carp, do we have a bazillion XEPs mentioning "jingle"
pep.
Is there any implementation of 0370?
zinid
we need no carpy jingle
lumihas joined
jjrhhas left
SouLhas joined
jjrhhas left
sonnyhas left
ralphmhas left
SouLhas joined
SouLhas left
Steve Killehas left
Steve Killehas left
jjrhhas left
waqashas left
SouLhas joined
sonnyhas joined
Steve Killehas joined
lumihas joined
marc
Ge0rG, what is the input field on your easy-xmpp-invitation website for?
jjrhhas left
sonnyhas left
sonnyhas joined
sonnyhas joined
SouLhas joined
SouLhas joined
zinidhas left
SouLhas left
ralphmhas left
la|r|mahas left
SouLhas joined
mimi89999has left
uchas left
remkohas left
mimi89999has joined
uchas joined
mimi89999has joined
Guushas left
Guushas left
ralphmhas left
Syndacehas joined
lumihas joined
SouLhas joined
vanitasvitaehas left
vanitasvitaehas joined
SouLhas left
SouLhas joined
SouLhas left
la|r|mahas joined
ralphmhas left
Syndacehas left
Syndacehas joined
McKaelhas joined
jjrhhas left
SouLhas joined
McKaelhas joined
SouLhas joined
McKaelhas joined
ralphmhas joined
Guushas left
jjrhhas left
SouLhas joined
Guushas left
jcbrandhas joined
ralphmhas left
jcbrandhas left
ralphmhas left
ralphmhas joined
jubalhhas joined
SouLhas joined
Tobiashas joined
jubalhhas left
goffihas left
jjrhhas left
danielhas left
jjrhhas left
jjrhhas left
danielhas left
jjrhhas left
SouLhas left
jjrhhas left
SouLhas left
edhelas
I think that we have to start to work on spam issues seriously
jabberatdemohas joined
edhelas
we have to see how people are spamming XMPP today on a wider scale than just blocking a type of message
Link Mauve
Oh, is it a new wave?
edhelas
Link Mauve it basically never stopped
edhelas
lena2521@jabber.uznam.net.pl
Alex
ya, SPAM is getting horrible, in the mood of shutting down my XMPP clients :(
Link Mauve
Might be our anti-spam solution working great then. ^^
edhelas
madaline2784@i0i0.de
edhelas
for now
Link Mauve
edhelas, it brings nothing to ban individual JIDs to the network.
edhelas
it's super easy to create a new host on your XMPP serve, put a let's encrypt certificate on it and start spamming the others
edhelas
yes I know
Link Mauve
Alex, if you’re running Prosody, https://yaxim.org/blog/2017/12/12/spam-reduction-on-yax-dot-im/
edhelas
what I'd like to do is have a look at all those domains, see if they are pointing to same IPs
edhelas
and basically figure out from where those SPAMs are coming
edhelas
but we seriously have to take the problem
Link Mauve
edhelas, pretty sure most spammers are currently relying on insecure public infrastructure rather than using their own servers.
Link Mauve
edhelas, just read that blog article, that’s how we’ve been handling it at JabberFR for the past year or so.
edhelas
that's not exactly true
edhelas
I have more spam comming from new domains than known ones
MattJ
edhelas, can you share some example domains?
edhelas
well I just did
MattJ
There are multiple spammers, they use different techniques, and not everyone will have their JID on every spammer's list
edhelas
I can give you a longer list, give me a bit of time
Link Mauve
edhelas, blocking by domain is almost never a solution.
MattJ
So while Link Mauve and I receive spam from existing domains, maybe you experience it more from a spammer who uses new domains
edhelas
MattJ that's what i'm saying, we have different kind of spams, we have to differentiate them and see how we can block them
Link Mauve
I should log all of the spams I’m blocking, to get better statistics.
jabberatdemohas left
edhelas
well hopefully I have MAM and Movim cache for that
edhelas
also, again, for now we have simple SPAM messages
Link Mauve
Ge0rG, I’m interested in your statistics module btw.
edhelas
I start to have roster subscriptions request from spammers
edhelas
and the next kind will be Pubsub publications I think
edhelas
so I'm already talking with ejabberd dev to put quotas and limitations on Pubsub
Link Mauve
Wouldn’t reach nearly as many people as plain messages.
uchas left
uchas joined
daniel
> and the next kind will be Pubsub publications I think
lol that's unnecessarily complex. normal messages will reach people just fine
Tobiashas left
edhelas
I'm deleting one or two accounts of users that are publishing articles with links to weird urls and warez places :)
edhelas
but yes this is pretty long term
Tobiashas joined
daniel
also; while i'm not denying that spam is a problem it is not really for the average user; normal users don't publish their jid everywhere. the lists the spammers are using are (in parts) very old. while *we* see a lot of spam normal users don't
edhelas
yes
daniel
yes it creates load and servers and requires work from sys admins; but we shouldn't necessarily let this distract us from other problems
daniel
especially since normal users - and even we - are not the target audience of the spam
daniel
we don't speak russian and we don't usually buy stolen credit cards
daniel
ok; some of us speak russian…
jjrhhas left
edhelas
ok I've compiled a list of JID that are sending messages to Movim users without been in their roster
Link Mauve: thank you for sharing a private link, btw.
ralphmhas joined
Link Mauve
Oh, I saw it in public somewhere today, sorry. :x
Ge0rG
Link Mauve: it isn't published yet, and I'm in the middle of refactoring it into a technical post on my personal blog and a high-level post on yaxim.org
jubalhhas joined
Ge0rG
which technically means I've stopped working on it for now ;)
Link Mauve
I didn’t see any mention that it was a draft or anything either.
Ge0rG
Link Mauve: I'm sure I wrote it's a draft where I posted it.
Link Mauve
Damn…
Link Mauve
Sorry. :x
Link Mauve
You can still yank it out probably.
Ge0rG
Link Mauve: I'm sure nobody will notice
Ge0rG
let's hope it won't get picked up. yet.
Ge0rG
Link Mauve: but now you made me curious, you are not a member of the places I posted it in. :P
Link Mauve
I’m already looking for the place I got it from. ^^
MattJ: can we have presence blocking in mod_firewall please?
MattJ
"presence blocking"?
MattJ
KIND: presence
DROP.
MattJ
Presence. Blocked.
MattJ
XMP
Ge0rG
MattJ: I'm sure users will love this.
MattJ
They'll love the simplicity, which is good for UX
Ge0rG
MattJ: I want to block/revert presence subscriptions from accounts that are spamming.
MattJ
Incoming or outgoing?
Ge0rG
MattJ: both
Ge0rG
I've deleted some 2000 spammer accounts in the last weeks
Ge0rG
Link Mauve: the stats for that blog non-post I gathered by grepping prosody.log for JIDs captured by mod_firewall spammer.pfw
MattJ
I don't think a server should allow a new account to *have* 1600 open subscription requests
Ge0rG
MattJ: that's a great idea.
Link Mauve
Oh.
Ge0rG
Link Mauve: not sure if this will help you much. `zgrep -ho 'spam:.*message.*' /var/log/prosody/prosody.log*|grep -ho "from='[a-z0-9._-]*@[^']*'" > 2017-12-13-alljids.txt`
Ge0rG
MattJ: the other spam accounts I deleted have between 10 and 200 pending subscriptions. Which is still too much.
edhelas
Ge0rG you have IBR ?
Ge0rG
MattJ: but the worst thing is that I'm getting a dozen a subscriptions a day.
Ge0rG
edhelas: yes
edhelas
well then you know where they are coming from
Link Mauve
Ge0rG, it’s super weird, I can’t find any mention of this blog post in my logs before I posted it. oO
jjrhhas left
edhelas
on my side I'm starting to blacklist list of domains for s2s
Ge0rG
edhelas: from Tor and open proxies
pep.
wut, is that url private? it seems awfully public to me and I've seen it around already
Zash
Ge0rG: Maybe it would be clearer to say you wanna retract presence subscription requests from mod_firewall
Ge0rG
Zash: yes, that's exactly what I want. I'm sure I asked for that already one or two times
jjrhhas left
Ge0rG
pep.: I can't imagine how it made the rounds, or where
pep.
the spam reduction article right?
Zash
I thought I saw you ask for "blocking"
pep.
I'm confused
ralphmhas joined
Ge0rG
Zash: to block them I need to know they are spammers before they send their spam
Link Mauve
Maybe it would be simpler by writing another module, that would hook into the event fired by mod_firewall on someone being flagged as a spammer, and then proceed to remove its presence subscription.
Link Mauve
pep., do you remember where you saw it?
Link Mauve
Because grep doesn’t help. :/
Ge0rG
Zash: ideally it should delay incoming subscriptions for a minute and just discard them if the user sends incoming spam
Link Mauve
Ge0rG, one second would be enough currently.
Ge0rG
So kind of like a bastard of mod_smacks and mod_csi_pump
Zash
Hm, that sounds a bit tricky for mod_firewall?
Zash
Or? What sayeth MattJ?
SamWhitedhas joined
SamWhitedhas joined
MattJ
Some kind of tarpit has always been on my mind for mod_firewall, but it is indeed tricky
Ge0rG
I'm okay with a separate module if it helps tame subscription spam
pep.
Ah, hmm, no I've never read it. Dec 12th 2017
jjrhhas left
Link Mauve
Ge0rG, could you have used an URL shortener?
edhelas
is it risky to publish publicly my list of blacklisted s2s servers ?
Ge0rG
edhelas: I don't think so
edhelas
I'd like to be transparent regarding my configuration
Link Mauve
edhelas, it’s a good thing, it will allow them to start the process to get un-blacklisted.
pep.
edhelas, might also be nice to send a message to the contact address of that server when you blacklist it
Link Mauve
Indeed.
edhelas
meh
pep.
why?
pep.
Of course you can automate that
edhelas
step by step :)
edhelas
first publish the list
jjrhhas left
jjrhhas left
Ge0rG
MattJ: currently I'm actively monitoring prosody.log for outgoing spam, listing all accounts registered from the same ip as the perpetrator, checking whether any of them have proper roster subscriptions and deleting all that look like spammers.
jjrhhas left
Ge0rG
I've automated most of the steps so it boils down to copying a JID and a list of user:delete commands, but the monitoring itself is tedious and in theory easy to automate
MattJ
To automate that, I'd fire an event from the firewall "reject spam" chain, and just handle the rest in a module
zinidhas left
MattJ
At least log the IP to a separate file
Ge0rG
MattJ: yes, it would help to have a quarantine flag on accounts that could be set this way
MattJ
Hmm
Ge0rG
I've pulled a number for that one recently
Ge0rG
MattJ: but none of this solves the incoming subscriptions problem
Ge0rG
It merely reduces the outgoing subscriptions problem, slightly
Ge0rG
As I said, I've deleted around 2k accounts so far.
MattJ
Automatically rejecting a pending incoming subscription should be pretty straightforward to add as an action
Ge0rG
And one real user, by accident. Which is why I want a quarantine flag that's less terminal than a deletion
MattJ
Any "hold the stanza for X seconds/minutes" is full of performance and correctness problems
Ge0rG
MattJ: yes, I know. But holding a stanza long enough to check the next stanza from the same JID might actually work without melting the server
MattJ
We had a lot of discussion about this when you first had the idea
MattJ
Nice idea, but the spammer only has to wait N+1 seconds
MattJ
and they have plenty of time on their hands
Ge0rG
MattJ: maybe they do, maybe not.
MattJ
Whatever you choose for N, they can wait longer, and as you increase N you're going to effectively open yourself up to DoS attacks
debaclehas left
Ge0rG
Greylisting has turned out to work exceptionally well for email
Ge0rG
MattJ: besides, even if they wait, they won't get past the spam filter, so they have no incentive to modify their code
MattJ
What's their incentive for sending the subscription request in the first place?
MattJ
btw, I don't see anything about your account quarantine flag: https://prosody.im/issues/?q=state%3Dopen+firewall
Ge0rG
MattJ: I suppose it's too trick dumb clients / servers to accept the following message
Ge0rG
MattJ: https://prosody.im/issues/1057
Ge0rG
MattJ: my firewall blocks all spam messages anyway, so they won't gain anything by waiting longer
Alexhas left
Alexhas joined
Ge0rG
MattJ: please feel free to suggest a different method to mitigate the incoming subscriptions.
MattJ
I think we ultimately ended up at UI changes on the clients when we last discussed this
MattJ
That is, a subscription request should not be "noisy"
Ge0rG
MattJ: I would accept a subscription denial from the firewall as a first step.
MattJ
Yes, that can be done
Ge0rG
MattJ: except that we haven't implemented anything after the discussion, and are repeating it now.
MattJ
I'll get you your account flagging thing, which will at least help to improve your current process
MattJ
and then I'll get you automated rejection/retraction of subscription requests
Ge0rG
MattJ: thanks, that's awesome!
MattJ
The tarpit thing may happen one day, or it may never happen
MattJ
It's a nice idea with too many practical issues
Ge0rG
MattJ: what about making the flagging depend on the number of roster items the user has? I.e. when pending >(to+both)
MattJ
Simply because you have to queue every stanza for the same destination JID following a match, and they can send to an unbounded number of destination JIDs
Ge0rG
MattJ: I'm sure we can also stop incoming mass subscriptions from the same JID
Ge0rG
Just not from the same server...
MattJ
Ge0rG, they'll just add bot accounts to bot accounts rosters