XSF Discussion - 2017-12-19

  1. daniel

    jonasw, could you hit the the merge button on my protoxep when you have a minute so it makes the 24h notice window for the council agenda tomorrow?

  2. jonasw


  3. jonasw

    didn’t see the mail, sorry

  4. daniel

    jonasw: thank you

  5. jonasw

    daniel, mail sent :-)

  6. pep.

    What's the difference between {xep 0363} and {xep 0370}? Both can do more or less the same thing, right? 0370 doesn't require a server component but it would be feasible as well I think

  7. Bunneh

    pep.: Multiple matches: Stanza Headers and Internet Metadata https://xmpp.org/extensions/xep-0131.html Ad-Hoc Commands https://xmpp.org/extensions/xep-0050.html Dialback Key Generation and Validation https://xmpp.org/extensions/xep-0185.html Out of Band Data https://xmpp.org/extensions/xep-0066.html Blocking Command https://xmpp.org/extensions/xep-0191.html Atomically Compare-And-Publish PubSub Items https://xmpp.org/extensions/xep-0395.html Jingle In-Band Bytestreams Transport Method https://xmpp.org/extensions/xep-0261.html A Transport for Initiating and Negotiating Sessions (TINS) https://xmpp.org/extensions/xep-0111.html In-Band Real Time Text https://xmpp.org/extensions/xep-0301.html Spim Markers and Reports https://xmpp.org/extensions/xep-0287.html Out-of-Band Stream Data https://xmpp.org/extensions/inbox/outofband.html File Repository and Sharing https://xmpp.org/extensions/xep-0214.html Jingle In-Band Bytestreams Transport https://xmpp.org/extensions/inbox/jingle-ibb.html Stanza Interception and Filtering Technology https://xmpp.org/extensions/inbox/sift.html Mandatory-to-Implement Technologies for Jingle RTP Sessions https://xmpp.org/extensions/inbox/jingle-rtp-mti.html Spim Markers and Reports https://xmpp.org/extensions/inbox/spim.html Field Standardization for Data Forms https://xmpp.org/extensions/xep-0068.html The /me Command https://xmpp.org/extensions/xep-0245.html Stanza Interception and Filtering Technology (SIFT) https://xmpp.org/extensions/xep-0273.html In-Band Bytestreams https://xmpp.org/extensions/xep-0047.html XMPP Date and Time Profiles https://xmpp.org/extensions/xep-0082.html In-Band Registration https://xmpp.org/extensions/xep-0077.html Unique and Stable Stanza IDs https://xmpp.org/extensions/xep-0359.html Best Practices for Handling Offline Messages https://xmpp.org/extensions/xep-0160.html Extensible In-Band Registration https://xmpp.org/extensions/xep-0389.html Incident Handling https://xmpp.org/extensions/xep-0268.html Impact of TLS and DNSSEC on Dialback https://xmpp.org/extensions/xep-0344.html Form Discovery and Publishing https://xmpp.org/extensions/xep-0346.html Zero Handshake Server to Server Protocol https://xmpp.org/extensions/xep-0361.html Out-of-Band Stream Data https://xmpp.org/extensions/xep-0265.html Invisible Command https://xmpp.org/extensions/xep-0186.html Best Practices for Roster and Subscription Management https://xmpp.org/extensions/xep-0162.html Atomically Compare-And-Publish PubSub Items https://xmpp.org/extensions/inbox/cap.html

  8. pep.


  9. Zash


  10. Zash

    Maybe it did a search for "0363} and {xep 0370" and thought you really wanted all XEPs with the word "and" anywhere in them

  11. Zash

    Two {} commands at the same time don't work anyways

  12. pep.


  13. pep.

    -xep 0363

  14. Bunneh

    pep.: HTTP File Upload (Standards Track, Proposed, 2017-12-03) See: https://xmpp.org/extensions/xep-0363.html

  15. pep.

    -xep 0370

  16. Bunneh

    pep.: Jingle HTTP Transport Method (Standards Track, Deferred, 2017-09-11) See: https://xmpp.org/extensions/xep-0370.html

  17. Zash

    pep.: compare {xep 65} and ...

  18. Bunneh

    pep.: SOCKS5 Bytestreams (Standards Track, Draft, 2015-09-17) See: https://xmpp.org/extensions/xep-0065.html

  19. Zash

    -xep 260

  20. Bunneh

    Zash: Jingle SOCKS5 Bytestreams Transport Method (Standards Track, Draft, 2016-05-17) See: https://xmpp.org/extensions/xep-0260.html

  21. pep.


  22. Zash

    Holy carp, do we have a bazillion XEPs mentioning "jingle"

  23. pep.

    Is there any implementation of 0370?

  24. zinid

    we need no carpy jingle

  25. marc

    Ge0rG, what is the input field on your easy-xmpp-invitation website for?

  26. edhelas

    I think that we have to start to work on spam issues seriously

  27. edhelas

    we have to see how people are spamming XMPP today on a wider scale than just blocking a type of message

  28. Link Mauve

    Oh, is it a new wave?

  29. edhelas

    Link Mauve it basically never stopped

  30. edhelas


  31. Alex

    ya, SPAM is getting horrible, in the mood of shutting down my XMPP clients :(

  32. Link Mauve

    Might be our anti-spam solution working great then. ^^

  33. edhelas


  34. edhelas

    for now

  35. Link Mauve

    edhelas, it brings nothing to ban individual JIDs to the network.

  36. edhelas

    it's super easy to create a new host on your XMPP serve, put a let's encrypt certificate on it and start spamming the others

  37. edhelas

    yes I know

  38. Link Mauve

    Alex, if you’re running Prosody, https://yaxim.org/blog/2017/12/12/spam-reduction-on-yax-dot-im/

  39. edhelas

    what I'd like to do is have a look at all those domains, see if they are pointing to same IPs

  40. edhelas

    and basically figure out from where those SPAMs are coming

  41. edhelas

    but we seriously have to take the problem

  42. Link Mauve

    edhelas, pretty sure most spammers are currently relying on insecure public infrastructure rather than using their own servers.

  43. Link Mauve

    edhelas, just read that blog article, that’s how we’ve been handling it at JabberFR for the past year or so.

  44. edhelas

    that's not exactly true

  45. edhelas

    I have more spam comming from new domains than known ones

  46. MattJ

    edhelas, can you share some example domains?

  47. edhelas

    well I just did

  48. MattJ

    There are multiple spammers, they use different techniques, and not everyone will have their JID on every spammer's list

  49. edhelas

    I can give you a longer list, give me a bit of time

  50. Link Mauve

    edhelas, blocking by domain is almost never a solution.

  51. MattJ

    So while Link Mauve and I receive spam from existing domains, maybe you experience it more from a spammer who uses new domains

  52. edhelas

    MattJ that's what i'm saying, we have different kind of spams, we have to differentiate them and see how we can block them

  53. Link Mauve

    I should log all of the spams I’m blocking, to get better statistics.

  54. edhelas

    well hopefully I have MAM and Movim cache for that

  55. edhelas

    also, again, for now we have simple SPAM messages

  56. Link Mauve

    Ge0rG, I’m interested in your statistics module btw.

  57. edhelas

    I start to have roster subscriptions request from spammers

  58. edhelas

    and the next kind will be Pubsub publications I think

  59. edhelas

    so I'm already talking with ejabberd dev to put quotas and limitations on Pubsub

  60. Link Mauve

    Wouldn’t reach nearly as many people as plain messages.

  61. daniel

    > and the next kind will be Pubsub publications I think lol that's unnecessarily complex. normal messages will reach people just fine

  62. edhelas

    I'm deleting one or two accounts of users that are publishing articles with links to weird urls and warez places :)

  63. edhelas

    but yes this is pretty long term

  64. daniel

    also; while i'm not denying that spam is a problem it is not really for the average user; normal users don't publish their jid everywhere. the lists the spammers are using are (in parts) very old. while *we* see a lot of spam normal users don't

  65. edhelas


  66. daniel

    yes it creates load and servers and requires work from sys admins; but we shouldn't necessarily let this distract us from other problems

  67. daniel

    especially since normal users - and even we - are not the target audience of the spam

  68. daniel

    we don't speak russian and we don't usually buy stolen credit cards

  69. daniel

    ok; some of us speak russian…

  70. edhelas

    ok I've compiled a list of JID that are sending messages to Movim users without been in their roster

  71. edhelas

    some statistics per server :)

  72. edhelas


  73. Ge0rG

    marc: it's not an input field, it's a text selection field where you can copy the xmpp: URI from

  74. marc

    Ge0rG, https://projects.zapb.de/tmp/easy-xmpp-invitation/

  75. marc

    I can edit the xmpp URI

  76. Ge0rG

    Link Mauve: thank you for sharing a private link, btw.

  77. Link Mauve

    Oh, I saw it in public somewhere today, sorry. :x

  78. Ge0rG

    Link Mauve: it isn't published yet, and I'm in the middle of refactoring it into a technical post on my personal blog and a high-level post on yaxim.org

  79. Ge0rG

    which technically means I've stopped working on it for now ;)

  80. Link Mauve

    I didn’t see any mention that it was a draft or anything either.

  81. Ge0rG

    Link Mauve: I'm sure I wrote it's a draft where I posted it.

  82. Link Mauve


  83. Link Mauve

    Sorry. :x

  84. Link Mauve

    You can still yank it out probably.

  85. Ge0rG

    Link Mauve: I'm sure nobody will notice

  86. Ge0rG

    let's hope it won't get picked up. yet.

  87. Ge0rG

    Link Mauve: but now you made me curious, you are not a member of the places I posted it in. :P

  88. Link Mauve

    I’m already looking for the place I got it from. ^^

  89. Ge0rG

    Sigh. advert364@yax.im - 1600 outgoing subscription requests.

  90. Ge0rG

    MattJ: can we have presence blocking in mod_firewall please?

  91. MattJ

    "presence blocking"?

  92. MattJ

    KIND: presence DROP.

  93. MattJ

    Presence. Blocked.

  94. MattJ


  95. Ge0rG

    MattJ: I'm sure users will love this.

  96. MattJ

    They'll love the simplicity, which is good for UX

  97. Ge0rG

    MattJ: I want to block/revert presence subscriptions from accounts that are spamming.

  98. MattJ

    Incoming or outgoing?

  99. Ge0rG

    MattJ: both

  100. Ge0rG

    I've deleted some 2000 spammer accounts in the last weeks

  101. Ge0rG

    Link Mauve: the stats for that blog non-post I gathered by grepping prosody.log for JIDs captured by mod_firewall spammer.pfw

  102. MattJ

    I don't think a server should allow a new account to *have* 1600 open subscription requests

  103. Ge0rG

    MattJ: that's a great idea.

  104. Link Mauve


  105. Ge0rG

    Link Mauve: not sure if this will help you much. `zgrep -ho 'spam:.*message.*' /var/log/prosody/prosody.log*|grep -ho "from='[a-z0-9._-]*@[^']*'" > 2017-12-13-alljids.txt`

  106. Ge0rG

    MattJ: the other spam accounts I deleted have between 10 and 200 pending subscriptions. Which is still too much.

  107. edhelas

    Ge0rG you have IBR ?

  108. Ge0rG

    MattJ: but the worst thing is that I'm getting a dozen a subscriptions a day.

  109. Ge0rG

    edhelas: yes

  110. edhelas

    well then you know where they are coming from

  111. Link Mauve

    Ge0rG, it’s super weird, I can’t find any mention of this blog post in my logs before I posted it. oO

  112. edhelas

    on my side I'm starting to blacklist list of domains for s2s

  113. Ge0rG

    edhelas: from Tor and open proxies

  114. pep.

    wut, is that url private? it seems awfully public to me and I've seen it around already

  115. Zash

    Ge0rG: Maybe it would be clearer to say you wanna retract presence subscription requests from mod_firewall

  116. Ge0rG

    Zash: yes, that's exactly what I want. I'm sure I asked for that already one or two times

  117. Ge0rG

    pep.: I can't imagine how it made the rounds, or where

  118. pep.

    the spam reduction article right?

  119. Zash

    I thought I saw you ask for "blocking"

  120. pep.

    I'm confused

  121. Ge0rG

    Zash: to block them I need to know they are spammers before they send their spam

  122. Link Mauve

    Maybe it would be simpler by writing another module, that would hook into the event fired by mod_firewall on someone being flagged as a spammer, and then proceed to remove its presence subscription.

  123. Link Mauve

    pep., do you remember where you saw it?

  124. Link Mauve

    Because grep doesn’t help. :/

  125. Ge0rG

    Zash: ideally it should delay incoming subscriptions for a minute and just discard them if the user sends incoming spam

  126. Link Mauve

    Ge0rG, one second would be enough currently.

  127. Ge0rG

    So kind of like a bastard of mod_smacks and mod_csi_pump

  128. Zash

    Hm, that sounds a bit tricky for mod_firewall?

  129. Zash

    Or? What sayeth MattJ?

  130. MattJ

    Some kind of tarpit has always been on my mind for mod_firewall, but it is indeed tricky

  131. Ge0rG

    I'm okay with a separate module if it helps tame subscription spam

  132. pep.

    Ah, hmm, no I've never read it. Dec 12th 2017

  133. Link Mauve

    Ge0rG, could you have used an URL shortener?

  134. edhelas

    is it risky to publish publicly my list of blacklisted s2s servers ?

  135. Ge0rG

    edhelas: I don't think so

  136. edhelas

    I'd like to be transparent regarding my configuration

  137. Link Mauve

    edhelas, it’s a good thing, it will allow them to start the process to get un-blacklisted.

  138. pep.

    edhelas, might also be nice to send a message to the contact address of that server when you blacklist it

  139. Link Mauve


  140. edhelas


  141. pep.


  142. pep.

    Of course you can automate that

  143. edhelas

    step by step :)

  144. edhelas

    first publish the list

  145. Ge0rG

    MattJ: currently I'm actively monitoring prosody.log for outgoing spam, listing all accounts registered from the same ip as the perpetrator, checking whether any of them have proper roster subscriptions and deleting all that look like spammers.

  146. Ge0rG

    I've automated most of the steps so it boils down to copying a JID and a list of user:delete commands, but the monitoring itself is tedious and in theory easy to automate

  147. MattJ

    To automate that, I'd fire an event from the firewall "reject spam" chain, and just handle the rest in a module

  148. MattJ

    At least log the IP to a separate file

  149. Ge0rG

    MattJ: yes, it would help to have a quarantine flag on accounts that could be set this way

  150. MattJ


  151. Ge0rG

    I've pulled a number for that one recently

  152. Ge0rG

    MattJ: but none of this solves the incoming subscriptions problem

  153. Ge0rG

    It merely reduces the outgoing subscriptions problem, slightly

  154. Ge0rG

    As I said, I've deleted around 2k accounts so far.

  155. MattJ

    Automatically rejecting a pending incoming subscription should be pretty straightforward to add as an action

  156. Ge0rG

    And one real user, by accident. Which is why I want a quarantine flag that's less terminal than a deletion

  157. MattJ

    Any "hold the stanza for X seconds/minutes" is full of performance and correctness problems

  158. Ge0rG

    MattJ: yes, I know. But holding a stanza long enough to check the next stanza from the same JID might actually work without melting the server

  159. MattJ

    We had a lot of discussion about this when you first had the idea

  160. MattJ

    Nice idea, but the spammer only has to wait N+1 seconds

  161. MattJ

    and they have plenty of time on their hands

  162. Ge0rG

    MattJ: maybe they do, maybe not.

  163. MattJ

    Whatever you choose for N, they can wait longer, and as you increase N you're going to effectively open yourself up to DoS attacks

  164. Ge0rG

    Greylisting has turned out to work exceptionally well for email

  165. Ge0rG

    MattJ: besides, even if they wait, they won't get past the spam filter, so they have no incentive to modify their code

  166. MattJ

    What's their incentive for sending the subscription request in the first place?

  167. MattJ

    btw, I don't see anything about your account quarantine flag: https://prosody.im/issues/?q=state%3Dopen+firewall

  168. Ge0rG

    MattJ: I suppose it's too trick dumb clients / servers to accept the following message

  169. Ge0rG

    MattJ: https://prosody.im/issues/1057

  170. Ge0rG

    MattJ: my firewall blocks all spam messages anyway, so they won't gain anything by waiting longer

  171. Ge0rG

    MattJ: please feel free to suggest a different method to mitigate the incoming subscriptions.

  172. MattJ

    I think we ultimately ended up at UI changes on the clients when we last discussed this

  173. MattJ

    That is, a subscription request should not be "noisy"

  174. Ge0rG

    MattJ: I would accept a subscription denial from the firewall as a first step.

  175. MattJ

    Yes, that can be done

  176. Ge0rG

    MattJ: except that we haven't implemented anything after the discussion, and are repeating it now.

  177. MattJ

    I'll get you your account flagging thing, which will at least help to improve your current process

  178. MattJ

    and then I'll get you automated rejection/retraction of subscription requests

  179. Ge0rG

    MattJ: thanks, that's awesome!

  180. MattJ

    The tarpit thing may happen one day, or it may never happen

  181. MattJ

    It's a nice idea with too many practical issues

  182. Ge0rG

    MattJ: what about making the flagging depend on the number of roster items the user has? I.e. when pending >(to+both)

  183. MattJ

    Simply because you have to queue every stanza for the same destination JID following a match, and they can send to an unbounded number of destination JIDs

  184. Ge0rG

    MattJ: I'm sure we can also stop incoming mass subscriptions from the same JID

  185. Ge0rG

    Just not from the same server...

  186. MattJ

    Ge0rG, they'll just add bot accounts to bot accounts rosters

  187. Ge0rG

    MattJ: maybe

  188. Ge0rG

    MattJ: but yes, das things first please.

  189. Ge0rG is leaving for the night now. CU

  190. MattJ

    Good night

  191. Ge0rG

    s/das/easy/ it's too late for auto completion