XSF Discussion - 2017-11-28

  1. Tobias has joined

  2. goffi has left

  3. Zash has left

  4. jjrh has left

  5. efrit has joined

  6. daniel has left

  7. daniel has joined

  8. sonny has left

  9. sonny has joined

  10. Tobias has joined

  11. lumi has joined

  12. Kev has left

  13. Tobias has joined

  14. uc has joined

  15. vanitasvitae has left

  16. efrit has left

  17. lumi has left

  18. jere has joined

  19. Tobias has joined

  20. nyco has left

  21. nyco has joined

  22. jere has joined

  23. Zash has left

  24. arc has left

  25. arc has joined

  26. la|r|ma has left

  27. arc has left

  28. arc has joined

  29. daniel has left

  30. daniel has joined

  31. la|r|ma has joined

  32. lskdjf has joined

  33. Tobias has joined

  34. Tobias has joined

  35. arc has left

  36. arc has joined

  37. arc has left

  38. arc has joined

  39. arc has left

  40. arc has joined

  41. uc has left

  42. uc has joined

  43. sonny has left

  44. sonny has joined

  45. Ge0rG has left

  46. Ge0rG has joined

  47. Ge0rG has left

  48. Ge0rG has joined

  49. arc has left

  50. arc has joined

  51. arc has left

  52. arc has joined

  53. blabla has left

  54. SamWhited has left

  55. arc has left

  56. arc has joined

  57. arc has left

  58. arc has joined

  59. arc has left

  60. arc has joined

  61. Alacer has joined

  62. Alacer has left

  63. zinid has left

  64. zinid has left

  65. Guus has left

  66. jere has joined

  67. moparisthebest has joined

  68. ralphm has left

  69. valo has joined

  70. ralphm has left

  71. zinid

    yeah, nice rules: 60% of members are in council now, so council will vote for council forever

  72. jonasw


  73. Holger has left

  74. jonasw

    I’m pretty sure that’s incorrect

  75. Ge0rG


  76. jonasw

    https://xmpp.org/about/xsf/members.html we have many more members

  77. jonasw

    even though that page needs updating

  78. jonasw

    (preparing a PR)

  79. zinid

    jonasw, I checked https://wiki.xmpp.org/web/Membership_Applications_Q4_2017

  80. Ge0rG

    those are slightly more than nine.

  81. Ge0rG

    zinid: there are four application periods per year, you need to take them all together

  82. jonasw

    zinid, reapplication is needed yearly, not quarterly

  83. zinid


  84. zinid

    so where is the full list?

  85. jonasw

    so in Q4, at least four dropped out and up to one will be added

  86. jonasw

    I linked it, zinid

  87. jonasw


  88. zinid

    jonasw, is this page up to date?

  89. jonasw

    I am updating it with the new council and board election right now

  90. jonasw

    otherwise it is

  91. zinid

    ok, I didn't know you can reapply only once a year

  92. jonasw

    if someone could double-check I didn’t mess up here, then I’ll merge it: https://github.com/xsf/xmpp.org/pull/385

  93. zinid

    whatever, I actually have a question about processing empty values in data forms

  94. zinid

    for example, the type of the <field/> is 'jid-single' and the value is <value/>

  95. zinid

    what to do in this situation?

  96. zinid

    empty string is clearly not a JID

  97. Zash

    Mmmmmmm, yeah, that's somewhat ambigous IIRC

  98. zinid

    yeah, and if we forbid empty values, then how to clear the field?

  99. Zash

    <field><value/></field> == empty string and <field/> == NULL or something like that

  100. jonasw

    zinid, that’s simply invalid if the field is <required/>

  101. moparisthebest has joined

  102. jonasw

    otherwise, I’d treat it as absent

  103. jonasw

    there is: > Note: Data provided for fields of type "jid-single" or "jid-multi" MUST contain one or more valid Jabber IDs, where validity is determined by the addressing rules defined in XMPP Core (see the Data Validation section below).

  104. zinid

    jonasw, absent meaning you should ignore it, or set it as "not set"?

  105. jonasw

    since <value/> violates that, I’d treat the field as absent or unset

  106. Zash

    There's also the case of submitting a partial form.

  107. jonasw

    which is either an error if the field was <required/> or leads to some defaulting if it wasn’t

  108. zinid

    jonasw, I'm not talking about required, it's obvious in this case

  109. jonasw

    zinid, well, if it’s not <required/> surely the business logic of the form already knows what to do if the field is missing?

  110. zinid

    jonasw, indeed, what to do? keeping the field values untouched in the database or remove it?

  111. zinid


  112. jonasw

    I’d NULL it

  113. zinid

    I actually tend to think you should not set <value/> at all if you need to erase it

  114. jonasw


  115. Zash

    <value/> == <value></value> == ""

  116. jonasw

    which isn’t a valid JID

  117. Zash


  118. Zash

    So you get an error back

  119. jonasw

    so both empty <value/> and no <value/> at all violate the note I quoted above

  120. zinid

    another issue: the type is 'jid-multi' and you got <value/><value/><value/>

  121. jonasw

    I hate XEP-0004

  122. zinid

    nah, we just should clarify it

  123. zinid

    the xep is okayish

  124. zinid


  125. jonasw

    and cut it in two pieces

  126. zinid

    psi sends <value/> inside jid-multi field, I have even work-around for this in the data form parser

  127. zinid

    so, most likely, to keep backward compatibility we need to treap empty <value/> as no <value/> at all and should set internally the field to default value (e.g. NULL)

  128. zinid

    *to treat

  129. zinid

    simply put, treat this situation as a "default value"

  130. intosi has joined

  131. moparisthebest has joined

  132. moparisthebest has joined

  133. daniel has left

  134. daniel has joined

  135. daniel has left

  136. daniel has joined

  137. daniel has left

  138. daniel has joined

  139. daniel has left

  140. daniel has joined

  141. marc has joined

  142. ralphm has joined

  143. daniel has left

  144. daniel has joined

  145. Alex has joined

  146. Guus

    Georg, Kev, could you please provide a snippet for https://xmpp.org/about/xmpp-standards-foundation.html ?

  147. goffi has joined

  148. Alex has left

  149. daniel has left

  150. daniel has joined

  151. Steve Kille has left

  152. ralphm has left

  153. Steve Kille has joined

  154. vanitasvitae has left

  155. daniel has left

  156. vanitasvitae has left

  157. moparisthebest has joined

  158. moparisthebest has joined

  159. Martin has joined

  160. daniel has left

  161. daniel has left

  162. moparisthebest has joined

  163. moparisthebest has joined

  164. jcbrand has joined

  165. jere has joined

  166. ralphm has joined

  167. efrit has joined

  168. Syndace has left

  169. Syndace has joined

  170. jubalh has joined

  171. jubalh has left

  172. Alex has joined

  173. daniel has left

  174. arc has left

  175. arc has joined

  176. intosi has left

  177. jcbrand has left

  178. jcbrand has joined

  179. arc has left

  180. arc has joined

  181. arc has left

  182. arc has joined

  183. Ge0rG has joined

  184. arc has left

  185. arc has joined

  186. la|r|ma has joined

  187. Ge0rG has left

  188. ralphm has left

  189. jubalh has joined

  190. intosi has joined

  191. lskdjf has joined

  192. Ge0rG has left

  193. lumi has joined

  194. jubalh has left

  195. moparisthebest has left

  196. moparisthebest has joined

  197. pep. has left

  198. daniel has left

  199. efrit has left

  200. blabla has joined

  201. efrit has joined

  202. blabla has left

  203. blabla has joined

  204. intosi has left

  205. blabla has left

  206. blabla has joined

  207. efrit has left

  208. jere has left

  209. jere has joined

  210. lskdjf has joined

  211. efrit has joined

  212. Alex has left

  213. intosi has joined

  214. intosi has left

  215. Zash has left

  216. Alex has joined

  217. blabla has left

  218. blabla has joined

  219. blabla has left

  220. blabla has joined

  221. zinid has left

  222. Guus


  223. daniel

    Guus: 👍

  224. Guus


  225. Guus has left

  226. Zash


  227. Valerian has joined

  228. blabla has left

  229. daniel has left

  230. intosi has joined

  231. moparisthebest has joined

  232. moparisthebest has joined

  233. intosi has left

  234. ralphm has left

  235. daniel has left

  236. daniel has joined

  237. jonasw has left

  238. Alex has left

  239. Alex has joined

  240. lskdjf has left

  241. lskdjf has joined

  242. Tobias has joined

  243. mathieui

    is there a way to know if we already voted?

  244. Kev

    Memberbot will tell you, I believe, when you say hello to it.

  245. jonasw

    yup, it will

  246. jonasw

    > (14:41:04) Memberbot: You have already participated in this election. Would you like to recast your votes? (yes / no)

  247. mathieui

    right, thanks

  248. jubalh has joined

  249. Tobias has left

  250. Holger has left

  251. Holger has left

  252. ralphm has left

  253. SouL has left

  254. daniel has left

  255. daniel has joined

  256. efrit has left

  257. Steve Kille has left

  258. efrit has joined

  259. daniel has left

  260. Valerian has left

  261. daniel has left

  262. Tobias has left

  263. Steve Kille has joined

  264. daniel has left

  265. la|r|ma has left

  266. la|r|ma has joined

  267. Valerian has joined

  268. jere has left

  269. jere has joined

  270. vanitasvitae has left

  271. ralphm has joined

  272. daniel has left

  273. daniel has left

  274. vanitasvitae has left

  275. Tobias has left

  276. jubalh has left

  277. Alex has left

  278. Alex has joined

  279. marc

    Ge0rG, jonasw, I plan to provide a field (optional or required, determined by the service) to specify the name of the inviter, what do you think?

  280. jonasw

    sorry, I lack context. where would you provide that field?

  281. marc

    jonasw, user invitation

  282. marc

    Sorry :D

  283. jonasw

    and where’d that field be?

  284. marc

    Such that we can provide something like "You were invited by X to join..." on the invitation page

  285. marc

    Ad-hoc command

  286. marc

    Filled out by the inviter

  287. jonasw


  288. jonasw

    the issue with that type of names is that they can easily be spoofed

  289. jonasw

    Ge0rG had some thoughts on this type of spoofing in the context of invitations IIRC

  290. marc

    Sure, it's not a security feature :)

  291. jonasw

    it could be misleading though

  292. marc

    Well, you have to send the URL to somebody, if the user trust this URL then there is no problem

  293. marc

    (because it trust the mail, SMS, ...)

  294. Valerian has left

  295. Valerian has joined

  296. marc

    You can think of it as security feature if somebody tries to manipulate/replace the invitation URL ;)

  297. Ge0rG

    marc: I think it depends on how you implement it.

  298. Ge0rG

    marc: if it is an additional url parameter that contains the plaintext name, it's rather Meh.

  299. marc

    Ge0rG, that's up to the service

  300. Ge0rG

    marc: if you only attach a token to the URL and the user's JID is somehow obtained from the server via the token, it's a bit better

  301. marc

    The XEP defines the field

  302. marc

    How the invitation page is implemented is out of scope of this XEP

  303. Ge0rG

    marc: I see merit in defining the OOB behavior as well

  304. marc

    In my current implementation the name is fetched from a database via the token

  305. Ge0rG

    marc: so your XMPP server must also be a web server.

  306. marc

    Ge0rG, well, providing a web page is also optional

  307. marc

    My first implementation just uses the xmpp URI

  308. marc

    and generates a QR code

  309. marc

    Works nicely without web site etc.

  310. Ge0rG

    marc: so where is the inviter's name displayed, then?

  311. daniel has left

  312. Ge0rG

    marc: does it also work nicely without an XMPP client?

  313. marc

    Ge0rG, the name is displayed on a web site, if you provide one

  314. marc

    Ge0rG, what works nicely?

  315. Ge0rG

    marc: the QR code

  316. daniel has left

  317. marc

    the QR code is display in the xmpp client

  318. marc

    but you could also just send the xmpp uri via SMS or ...

  319. Ge0rG

    marc: and then?

  320. Ge0rG

    marc: how does your friend open an xmpp: URI without an XMPP client?

  321. marc

    Ge0rG, add a link with a download URL for a xmpp client ;)

  322. Ge0rG

    marc: why not just use easy-xmpp-invitation? :P

  323. marc

    Ge0rG, what's easy-xmpp-invitation?

  324. marc

    ah I see

  325. marc

    your website template

  326. marc

    Ge0rG, that's nice but why should it be required by the XEP?

  327. marc

    I prefer a web site for invitation which displays the QR code, provides information about clients etc.

  328. marc

    But why should this be required?

  329. daniel has left

  330. jubalh has left

  331. Ge0rG

    marc: how do you generate the url for the web site where the inviter's username is displayed?

  332. marc

    Ge0rG, the name is fetched from a database not included in the URL

  333. marc

    You could include it in the URL

  334. marc

    If you like but I don't care how that's implemented

  335. Ge0rG

    marc: you send me a QR code with xmpp:.... How do I know the URL of the web page?

  336. marc

    Out of scope IMO

  337. lovetox has joined

  338. marc

    Ge0rG, either I send you a URL or a bare xmpp URI

  339. Ge0rG

    marc: where do you get the URL from?

  340. marc

    In best case you're next to me and just scan the QR code from my display of my mobile phone

  341. marc

    Ge0rG, the URL is generated by the server and sent back to the inviter

  342. Ge0rG

    marc: so the response to the ad-hoc command is an xmpp: URI and a web link?

  343. Ge0rG

    or either?

  344. marc

    Always a token

  345. marc

    And, if provided, a URL

  346. marc

    the xmpp URI can be generated by the client

  347. Ge0rG

    So the client needs to construct the xmpp: URI from the token?

  348. marc


  349. Ge0rG

    And the server could send back an easy-xmpp-invitation URL or a mod_invite URL or some other black magic?

  350. marc

    Ge0rG, it can send back a URL to some web site, yes

  351. Ge0rG

    Sorry for the many questions, I'm trying to understand the protocol.

  352. Ge0rG

    marc: will that be a generic website or a personalized one?

  353. marc

    I thought it is not that complicated and straightforward :D

  354. Ge0rG

    marc: life is full of corner cases.

  355. marc

    Ge0rG, depends on the server / service

  356. Ge0rG

    marc: so if your server sends personalized links, your client can forward the web URL and be good, but if my server returns a generic URL, and my client forwards only that, it's worthless?

  357. marc

    they could also provide a URL to some porn web site if they like ;)

  358. Ge0rG

    besides of the porn, of course.

  359. marc

    Ge0rG, sure, of course the web site should include the xmpp URI, QR code etc

  360. marc

    Otherwise this web site wouldn't make sense

  361. Ge0rG

    marc: it could be a generic registration form or the ToS

  362. marc

    No, that's not good

  363. marc

    Well, you could do this, of course

  364. marc

    But then the token is useless :D

  365. Ge0rG

    marc: that's my point.

  366. Ge0rG

    marc: so the XEP needs to specify that the returned URL is a personalized one that also contains the token (or a different token with the same functionality)

  367. daniel has left

  368. marc

    Ge0rG, we could also make the token optional and the server can send a URL of some generic invitation web site

  369. Ge0rG

    marc: you don't need a XEP for that, do you?

  370. marc

    Ge0rG, if you would like to have a standard for clients to request an inviation URL you do

  371. marc

    Otherwise, where do you get this URL from? Search on the web?

  372. marc


  373. marc

    I'm not saying that this is the best "response" from a service but it is better than nothing :)

  374. Ge0rG

    marc: I'm saying that it's worse than nothing, if it is expected to be a specific URL

  375. Ge0rG

    the client needs to decide based on what it receives.

  376. Ge0rG

    if it receives [URL=https://yax.im/register, token=deadbeaf], it doesn't know whether it must send on both or only the URL

  377. Ge0rG

    so the URL needs to provide the same functionality as the token

  378. Ge0rG

    besides, it might be useful to return not just a token but an xmpp: URI, because the server might be better suited to construct it than the client

  379. Ge0rG

    then the server could return xmpp:free-hosting.com?token=deadbeef to a client logged in as user@legacy.free-hosting.com

  380. marc

    Sure, the server can also return the xmpp URI

  381. lskdjf has left

  382. marc

    Ge0rG, but the URL can still be optional, right?

  383. daniel has left

  384. Tobias has joined

  385. Ge0rG

    marc: yes. The URL should be optional, I just would like to prevent "smart" server operators entering a generic URL there

  386. marc

    Ge0rG, yes, but we can not avoid it anyway ;)

  387. Ge0rG

    marc: but we can make it illegal via the XEP

  388. marc


  389. marc

    Ge0rG, btw, I think your easy-xmpp web site should auto-detect the operating system / browser and suggest a client :)

  390. Ge0rG

    marc: I think you are right, which is why I wrote that into the TODO

  391. marc

    ah nice ;)

  392. marc

    didn't read it

  393. Ge0rG

    too bad :P

  394. sonny has joined

  395. Kev

    I've just updated the Board mailing list. It's now current members + Council Chair + ED + Secretary.

  396. Guus

    ... ED ...

  397. Guus

    Ah, Exec. Director?

  398. Guus

    Thanks Kev

  399. sonny has joined

  400. jonasw

    Guus, so I’m not the only one having to re-think each time they see the abbreviation "ED"

  401. Guus

    "each time" <-- you've seen it before then? :)

  402. Valerian has left

  403. efrit has left

  404. moparisthebest

    ha well I found a fool proof spam prevention system that would work for xmpp too, it's terrible, but it'd work

  405. moparisthebest

    our support got a ticket at work from a user who wasn't getting our emails because they have a spam prevention system that replies to the email with a link the sender has to click to allow the email through, or whitelist the address, or something

  406. efrit has joined

  407. jonasw

    Guus, members list

  408. moparisthebest

    so for unsolicited chat or subscription requests, server could message the user an http link that needs clicked before allowing it through... :/

  409. SamWhited

    moparisthebest: that's basically the same as the proof-of-work model we talked about, except less automated

  410. moparisthebest

    where is this discussion? must have missed it

  411. SamWhited

    that could be the fallback if a PoW message gets sent but the potential spammers client doesn't support it

  412. SamWhited

    moparisthebest: I'm not sure where or when; it was a while ago. TL;DR if you get a message from someone you don't know, send their client a relatively expensive problem they have to compute and respond to before you'll show the message to the user

  413. moparisthebest

    I actually think this would work better in xmpp vs email, this user wanted the person at our company monitoring the system.noreply@ourdomain.com email to click the link...

  414. moparisthebest

    yea the combination might be good

  415. SamWhited

    I've got a TODO to write up a spec for that actually; I should do that this weekend.

  416. jubalh has joined

  417. moparisthebest

    you should :)

  418. jonasw

    *SHOULD :-)

  419. SamWhited moves it up to the top of his list from the bottom that's off the screen and therefore was forgotten about

  420. moparisthebest

    with a message a link fallback, servers could sanely turn it on before client support was widespread

  421. SamWhited

    actually, my TODO was for using it with IBR2, but the same challenge could be reused for both probably

  422. moparisthebest

    and you don't need to specify what happens with the link, if that happens you expect human intervention, servers might whitelist automatically, have a captcha, or whatever

  423. Holger

    What part of this needs a spec?

  424. moparisthebest

    the automated proof of work part

  425. moparisthebest

    the sending a link would not

  426. SamWhited

    Holger: the link part doesn't, but if we want clients to automatically respond to PoW challenges from the server that part needs a spec

  427. vanitasvitae has left

  428. Holger

    Ah the client responds without automatically? What happens to people with old clients?

  429. Holger


  430. SamWhited

    Holger: that's why you include a link to a captcha or something in the body. Old clients show that, you can click it and complete a human challenge instead

  431. MattJ

    They click the link

  432. Holger


  433. SamWhited

    Just like joining MUCs on Jabber.ru, which I think does this with clients that don't support their captcha forms (you get a link to the same form on the web)

  434. MattJ

    and here you go: https://xmpp.org/extensions/xep-0158.html#challenge-hashcash

  435. MattJ

    Later in the document: "A challenger MAY provide a text question in the <body/> element of a challenge stanza for clients that do not support CAPTCHA forms."

  436. SamWhited

    oh hey, would you look at that, been done

  437. MattJ

    The problem I see with proof-of-work is that spammers have access to lots of CPU cycles (that typically aren't really theirs), and real users don't

  438. SamWhited

    It's true, it doesn't stop botnets, it just stops every person with a laptop from being able to register hundreds of accounts and then send spam with them all day

  439. Ge0rG

    Especially mobile users don't.

  440. MattJ

    So you'll flatten a user's battery a bit, and you'll cost a spammer some miniscule amount of money on a botnet or captcha-solving platform

  441. SamWhited

    *slows down (not stops)

  442. SamWhited

    It still forces them to use a botnet or captcha-solving platform, which is more work. And most users will rarely have to do this, only spammers would need to do it regularly

  443. MattJ

    Pretty sure the user's battery will drain before the spammers deem it not cost-effective

  444. SamWhited

    I suspect anyways; I bet most "normal" people only communicate with people on their roster, and if we drain a users battery because their phone is in a botnet I'm pretty okay with that

  445. MattJ

    SamWhited, we had CAPTCHA on register.jabber.org, it didn't stop them, just slowed them down - doesn't solve much at the end of the day

  446. SamWhited

    Slowing them down is really the only point; it's the best we can do

  447. Ge0rG

    Let's slow them down by retracting subscriptions from known spammers.

  448. uc has joined

  449. moparisthebest

    it wouldn't drain a mobile users battery at all, except a tiny amount once when they send a message to a new user they haven't sent one to before, right?

  450. Ge0rG

    moparisthebest: there is a little problem there: PoW is much more energy-expensive on a mobile CPU than on a GPU cluster.

  451. moparisthebest

    and if botnets do indeed bother and implement this, you can just fall back to manual human-intervention-required link challenge

  452. moparisthebest

    Ge0rG, but it happens once per new message to new user max?

  453. moparisthebest

    so like, maybe 30 times over the course of an entire xmpp account's life?

  454. moparisthebest

    maybe other people are far more popular than I am idk

  455. jonasw

    Ge0rG, depends on the PoW

  456. jonasw

    memory-hard proofs come into mind

  457. ralphm has left

  458. jonasw

    scrypt or something

  459. moparisthebest

    and yea you could do something that is harder on gpus, yep scrypt

  460. Ge0rG

    jonasw: right, because OOM isn't a thing on mobile :P

  461. daniel has left

  462. jonasw

    Ge0rG, it is, but using lareg amount of memory is expensive on GPUs

  463. jonasw

    (and on FPGAs)

  464. Ge0rG

    jonasw: we aren't talking about bitcoin mining parallelization here

  465. Ge0rG

    or whatever-scrypt-coin

  466. jonasw

    where large is something like ~100 MiB already, depending on the type of operations

  467. Ge0rG

    You can't expect an old Android phone to provide 100MB (plus JVM overhead) to calculate a PoW

  468. jonasw

    I would expect that to work actually

  469. jonasw

    I bet firefox needs more

  470. jonasw

    firefox gets swapped out then, I’m fine with that

  471. Ge0rG

    jonasw: "Bug report: my browser gets killed when I add a friend"

  472. daniel has left

  473. moparisthebest

    an old android phone can't run conversations either so who cares

  474. Ge0rG

    I think this is insane.

  475. jonasw

    your browser gets killed always anyways on those machines.

  476. mathieui

    this is crazy, yes

  477. SamWhited

    We'd have to think about that, but the point is that we can think of something that's not too bad for mobile users but still does the job

  478. moparisthebest

    then you disable it on your client and solve http links by hand Ge0rG ?

  479. jonasw

    I can’t really use my broswer + any other app on my Galaxy S3.

  480. Ge0rG

    moparisthebest: no, I just switch to WhatsBook.

  481. jonasw

    with WhatsBook, you need mutual subscription first anyways?

  482. moparisthebest

    are you thinking this is something that would happen often?

  483. jonasw

    I thought we had that as a possible sensible solution, too

  484. Holger

    jonasw: You don't.

  485. efrit has left

  486. jonasw

    Holger, hm, when we discussed this a few weeks ago, I think the consensus was that you need it, but w/e

  487. daniel has left

  488. Ge0rG

    moparisthebest: unless we make PoW prohibitively expensive for normal users, spammers won't be slowed / stopped by it.

  489. Holger

    jonasw: I remember someone claiming that and me not finding a single commercial messenger that actually does that.

  490. Holger

    jonasw: And I think it's unusable as a general solution.

  491. Holger

    Though admittedly this PoW idea sounds even worse.

  492. SamWhited

    If memory consumption and GPUs turn out to really be an issue it could also use an algorithm that relies on cache locality

  493. vanitasvitae has left

  494. vanitasvitae has joined

  495. moparisthebest

    Ge0rG, bcrypt and scrypt would like to disagree with you

  496. moparisthebest

    they were explicitly designed to be not so bad for normal users, and prohibitive for bad guys

  497. jonasw

    funny question, couldn’t the PoW be solved by a users server?

  498. MattJ

    Yes, this would be interesting :)

  499. mathieui

    new DoS way

  500. jonasw

    you’d quota that of course etc., but for the occasional adding of a contact…?

  501. MattJ

    mathieui, it would encourage server admins to lock down their servers better :)

  502. jonasw

    like, 3 PoW / day / user, 60 PoW / hour in total or so

  503. SamWhited

    jonasw: that's an interesting idea; sounds like a possible DoS, but servers could be smart about it and say "this user is generating too many PoWs, what are they doing?"

  504. jonasw

    SamWhited, exactly

  505. jonasw

    that’d solve the mobile issue

  506. moparisthebest

    that only helps good public servers being abused to send spam

  507. Ge0rG

    moparisthebest: https://bitcoinware.net/collections/gridseed-dual-ltc-and-btc-miner

  508. moparisthebest

    but, good thing to help I guess

  509. mathieui

    moparisthebest, and it will slow down bad servers just as well

  510. jonasw

    it also gives servers an interesting metric on users sending a lot of subscriptions/new messages

  511. SamWhited

    That only works if the server supports the PoW thing though; otherwise you still have to issue it to the client (or issue a captcha)

  512. jonasw

    and if the server is overloaded with PoW, it could simply forward it to the client.

  513. Ge0rG

    it would be better to have users do PoW to pay for their server.

  514. jonasw

    SamWhited, ofc.

  515. SamWhited

    So if a spammer has spun up their own server, you still have to support the original model

  516. moparisthebest

    Ge0rG, so xmpp spammers are going to buy super expensive miners, and try to use them for xmpp PoW ? seems unlikely

  517. jonasw

    "the original model", SamWhited?

  518. SamWhited

    jonasw: sending a captcha/PoW to the client, I mean

  519. jonasw

    you’d send it to the user, the server can decide to intercept the PoW request and handle it by itself or forward it to the client

  520. Ge0rG

    moparisthebest: spammers will either distribute PoW to their botnet, where you will be paying for the PoW, or rent some gigahashes.

  521. jonasw

    I’m not sure what you mean

  522. moparisthebest

    I don't think they really have botnets now, just register on open servers and spam until banned

  523. Ge0rG

    moparisthebest: please stop arguing. The PoW battle has been lost.

  524. jonasw

    Ge0rG, not sure

  525. moparisthebest

    there is no golden solves everything problem, you can only annoy them a bit, slow them down for a bit

  526. daniel has left

  527. jonasw

    if you have a botnet, you’re probably better off putting that computing power into $cryptocoin

  528. jonasw

    instead of trying to spam

  529. moparisthebest

    it's an arms race :P

  530. Ge0rG

    moparisthebest: we can make life really miserable for mobile users and slightly annoy spammers.

  531. moparisthebest

    you still haven't said how it would hurt mobile users at all?

  532. Ge0rG

    jonasw: you need orders of magnitude more power for viable commercial mining

  533. SamWhited

    Right, if you have a botnet to spam with we can't stop you right now, but we can slow down your botnet so that you send less spam and possibly make it prohibitively expensive for the people who spin up a single server in their basemenet to send out spam

  534. la|r|ma has left

  535. moparisthebest

    are you mistakenly thinking this would happen on every message or something?

  536. SamWhited

    And we can do it without affecting users at all, I suspect

  537. SamWhited

    *well behaved users

  538. moparisthebest

    how often do you add or message new users? how often does a normal user do that? I'm thinking very rarely

  539. Ge0rG

    moparisthebest: just to repeat my argument: if you make it sufficiently hard for spammers, it will be prohibitive for normal users.

  540. moparisthebest

    and again that's just wrong Ge0rG

  541. SamWhited

    Ge0rG: we're disagreeing with that argument. Why do you think it would be prohibitvely expensive for normal users?

  542. jonasw

    I doubt we’ll reach an argument here

  543. jonasw

    we need data

  544. Ge0rG

    moparisthebest: okay, suggest a number of scrypt operations a user needs to perform for an initial contact.

  545. jonasw

    maybe some google scholar-ing on how botnets operate nowadays?

  546. SamWhited

    Spammers send lots of messages to people that aren't on their contact lists, normal users don't. That asymetry is what we're targeting.

  547. Ge0rG

    SamWhited: we can target that with simple statistics, without annoying anyone.

  548. moparisthebest

    Ge0rG, whatever takes about 4ish seconds on say a samsung galaxy s4

  549. SamWhited

    Ge0rG: where and how do you get those statistics?

  550. MattJ

    SamWhited, I agree that I think this is the best place to solve this problem

  551. Holger

    "Q: My app crashes (or crashes other apps) when adding a contact. A: Don't worry, I know that you don't often add contacts."

  552. moparisthebest

    yea you wouldn't make it crash

  553. jonasw

    also, shouldn’t this discussion move to spam@?

  554. SamWhited

    Holger: that's why we have to do research and find something that won't OOM everyone but is still relatively tricky

  555. SamWhited

    Too many rooms.

  556. jonasw

    Holger, I still claim that users on devices with this low amount of memory are used to that

  557. Holger

    We can add that to our response then.

  558. Ge0rG

    Okay, just to get some numbers. "I managed to pull around 5.6KHash/sec on my Nexus 7 with all four threads." from https://rumorscity.com/2014/01/07/how-to-mine-litecoin-with-android/

  559. MattJ

    Proof of work aside, it's very easy to identify accounts sending to a lot of non-contacts. Spammers will switch to subscription requests, but it's equally easy to spot (new?) accounts sending lots of subscription requests, and this is uncommon (not impossible for a legitimate user, just uncommon)

  560. Holger


  561. Ge0rG

    So we are at ~20 KHashes for a first-contact

  562. moparisthebest

    MattJ, you mean easy if you run a server with lots of users I guess?

  563. MattJ

    Easy on any server

  564. jonasw

    MattJ, you assume that the server isn’t malicious

  565. moparisthebest

    or, easy for a public server to spot new users of it's server

  566. jonasw

    I find that assumption incorrect

  567. SamWhited

    MattJ: I agree, server operators should be doing that

  568. MattJ

    jonasw, that is an assumption in this case, malicious servers are a different thing

  569. waqas has joined

  570. moparisthebest

    yea explain how my private server with 4 xmpp accounts can see any of this data, easily or not

  571. MattJ

    But that's not a problem we have today

  572. stefandxm has left

  573. jonasw

    MattJ, it is

  574. jonasw

    I think

  575. Holger

    And malicious servers won't respond to PoW requests?

  576. moparisthebest

    Holger, if they don't do the pow or click the link and do whatever is there, no requests get through to the client?

  577. jonasw

    fighting spam at the source is of course most sensible, but hard in a distributed system

  578. moparisthebest

    so, it's not a problem

  579. SamWhited

    Holger: they will respond to PoW requests, and that's the point. It will slow them down because they'll be responding to *lots* of them.

  580. moparisthebest

    and if they do pow and too much spam still gets through, turn off pow and go back to manual links

  581. mathieui

    also: if a server implements the PoW thing, it could "wall" the subscriptions unless it’s mutual

  582. moparisthebest

    make them play an html5 punch the monkey game and beat a high score to whitelist their jid :P

  583. mathieui

    (e.g. "user A and user B want to communicate, B’s server does not implement PoW, B adds A, A sees nothing; A adds B because they know about it: the subscription is established")

  584. moparisthebest

    or A's server sends an https link to user B and when clicked lets it go through

  585. Ge0rG

    you can rent 500MH/s for three hours for ~3USD. That accounts for 100 Millions spam messages.

  586. Ge0rG

    if you price a single spam message at 20KH scrypt.

  587. MattJ

    Case closed :)

  588. Holger

    SamWhited: I got the idea, I just don't see the gain in throttling them to, dunno, some hundreds of thousands of messages per day and CPU core or whatever. Do they really send millions per day right now?

  589. Holger

    What Ge0rG said.

  590. jonasw

    Ge0rG, "ouch"

  591. Ge0rG

    moparisthebest: so will you stop now?

  592. zinid

    right, what will happen is that spam will become a bit more expensive for the customers 🙂

  593. Ge0rG

    source for MH price: https://www.miningrigrentals.com/rigs/scrypt

  594. SamWhited

    Holger: I don't know. More research would definitely be required, I just suspect we could slow them down a bit

  595. moparisthebest

    and are you sure you can use that for arbitrary scrypt challenges Ge0rG ?

  596. Ge0rG

    moparisthebest: again, spammers will just use whatever botnet they can get away with.

  597. jonasw

    moparisthebest, if you can’t *now*, as soon as you can use this to make some financial gain (e.g. spam), they will adapt so that it can be used for that

  598. daniel has left

  599. Ge0rG

    moparisthebest: even when running on desktop Windows malware, you are several orders of magnitude faster than on a mobile device.

  600. moparisthebest

    possibly, but then you are excluding spammers who don't have botnets

  601. jonasw

    moparisthebest, even a spammer with their own server can easily outperform that challenge

  602. moparisthebest

    are there challenges that are faster on ARM than amd64 ?

  603. Ge0rG

    moparisthebest: let's just exclude all spammers by definition. Congratulations, we have won the spam war.

  604. jonasw

    20 kH is not much

  605. Ge0rG

    I can go back to work now.

  606. SamWhited

    Botnets would (hopefully) still be slowed down, non-botnets it might be too expensive. It's not a panacea, but it still seems like it could be a benefit if the idea works.

  607. moparisthebest

    right there is no panacea

  608. Ge0rG

    Right. So please pretty please let's focus on the ideas that make it actually worse for spammers than for users.

  609. moparisthebest

    like this one

  610. moparisthebest

    or, what is your other idea?

  611. Ge0rG

    moparisthebest: read up my posts on the operators@ and standards@ MLs.

  612. zinid

    digitalocean started to provide high performance servers for computational tasks, for the record, I think 500MH/s would be nothing for those

  613. jjrh has left

  614. Ge0rG

    moparisthebest: I've exceeded my time budget for convincing you on pointless scrypt performance calculations, sorry.

  615. Holger

    moparisthebest: Mine is server-side filtering based on both message contents and meta data. Works relatively well for email.

  616. moparisthebest

    Ge0rG, I've been following a bit, but iirc most are geared towards running large-ish public servers with lots of data to analyze

  617. moparisthebest

    and nothing for small private servers

  618. Holger

    SpamAssassin works just fine for small private email servers.

  619. moparisthebest

    it does for email, I was under the impression most xmpp spam was too small for it to be effective

  620. Ge0rG

    there are many theoretical solutions that just won't work out. Like a distributed p2p reputation system.

  621. SamWhited

    Holger: I've been wondering about that too; do you have that tied to an XMPP server? How well does it work for the shorter XMPP messages?

  622. Ge0rG

    Some months ago, 99% of spam was multi-line Russian with multiple links.

  623. Ge0rG

    Easy to kill, just filter multi-line messages from strangers.

  624. Ge0rG

    Then, they switched to single-line messages with two pastebin links. Still pretty easy.

  625. moparisthebest

    sure and it's easy to do stuff like that, doesn't solve everything though, just like PoW doesn't

  626. SamWhited

    Those are good things to do as well

  627. Ge0rG

    moparisthebest: yes, but PoW WILL ANNOY USERS

  628. Holger

    SamWhited: I think it'll work quite well. Problem is it requires work.

  629. zinid

    Holger, except that Bayes sucks for short messages

  630. moparisthebest

    besides I look forward to a future when all messages are encrypted base64 blobs :P

  631. Holger

    moparisthebest: So all solutions are equally adequate because none are perfect?

  632. Ge0rG

    Now they send a subscription request + multiline / pastebin

  633. SamWhited

    Yah, I suspect zinid is right and it won't work so well for XMPP messages, but I'd love to be proven wrong

  634. Holger

    zinid: Yes I would definitely not rely just on Bayes.

  635. moparisthebest

    no, I'm saying you should implement non-perfect solutions because those are the only solutions we have Holger

  636. waqas has left

  637. Ge0rG

    moparisthebest: you are saying we should implement bad solutions because there are no perfect ones.

  638. lumi has left

  639. Holger

    moparisthebest: And I'm saying some non-perfect solutions are way better than others. So "nothing is perfect" is not a very convincing reasoning.

  640. Ge0rG

    I haven't seen a spam message in weeks. Just the subscription requests

  641. zinid

    Holger, there is very little research for short messages, I recall a couple of papers only, you can search google scholar with "sms spam" query

  642. Ge0rG

    BTW, my next proposals would be: - revert a pending subscription if a sender is classified as spam - implement a cache of URLs sent by strangers, with counters, and block messages with a count > 3

  643. Ge0rG

    MattJ: ^

  644. moparisthebest

    and, again, both of those solutions only work for large public servers Ge0rG

  645. moparisthebest

    I'm literally the only user of my xmpp server to get any spam

  646. Ge0rG

    moparisthebest: you know what? spammers are sending messages to non-existent accounts. You could easily set up a honeypot

  647. SamWhited

    Ge0rG: I agree, both of those things are a good start and should probably be done (although they also probably need a way of dealing with false positives, but that's a different problem)

  648. Ge0rG

    moparisthebest: please log on your server messages sent to non-existent accounts

  649. moparisthebest

    I'm not saying they aren't good solutions for large public servers, or that you shouldn't implement them, I just also want something that makes running your own server practical

  650. MattJ

    Ge0rG, #2 won't work, the URLs can vary too easily

  651. zinid

    moparisthebest, set spam traps 🙂

  652. Ge0rG

    MattJ: it's expensive to create a large number of shortened links

  653. MattJ

    Trivial to append #uuid

  654. Syndace has left

  655. Ge0rG

    MattJ: besides, we could HEAD them :P

  656. MattJ

    Trivial to append ?uuid

  657. zinid

    moparisthebest, i.e. non-unsed accounts, but you publish them everywhere 😉

  658. moparisthebest

    yea but is that what we suggest to people wanting to run their own servers

  659. moparisthebest

    start your server, setup SRV records, post non-existant accounts at random places

  660. Holger

    zinid: Yes, I don't have papers to prove my point. But many of the criteria used for email classification are unrelated to the length of the message body.

  661. SamWhited

    Holger, zinid: seems worth trying either way

  662. Ge0rG

    moparisthebest: you could just block messages from strangers with an URL.

  663. zinid

    Holger, yes, maybe you can do that, no doubt, although the result is unpredictable

  664. Ge0rG

    yax.im will send an error response "Blocked due to abuse", so a real sender actually will see a message about being blocked

  665. uc has joined

  666. moparisthebest

    Ge0rG, what about subscription spam

  667. zinid

    what about captcha btw?

  668. Ge0rG

    moparisthebest: see above

  669. moparisthebest

    also none of this works for encryption

  670. moparisthebest

    spammers will soon start sending omemo messages :P

  671. Holger

    Yes that's one of my complaints with E2EE.

  672. Ge0rG

    moparisthebest: I'm sure they will

  673. Ge0rG

    I'm eagerly awaiting the day when I can block E2EE on my server :P

  674. zinid

    Holger, ah, yes, E2EE, hehe

  675. moparisthebest

    so back around to the original plan of replying with a https link with arbitrary things to solve on it

  676. Ge0rG

    moparisthebest: yes, let your friends play xbill or whack-a-mole.

  677. Ge0rG

    or solve a reCAPTCHA.

  678. moparisthebest

    yes, once, doesn't seem like too much to ask?

  679. mathieui

    maybe we could embed flash applications as base64 inside subscription replies

  680. Ge0rG

    moparisthebest: you can do all that on your private server

  681. Ge0rG

    moparisthebest: but it doesn't work well for public services

  682. moparisthebest

    obviously doesn't happen if you send them an easy xmpp invite url or whatever

  683. moparisthebest

    mathieui, it's 2017 you mean html5 applications

  684. moparisthebest

    seems like it'd work equally well for public servers?

  685. moparisthebest

    public servers just additionally have much more to go on, so maybe it wouldn't always be necessary

  686. Ge0rG

    Holger: please tell MattJ about the normal-message-to-full-JID thing.

  687. Holger

    The "some clients do that so servers can't throw the message away" thing?

  688. jjrh has left

  689. Holger

    Some clients do that so servers can't throw the message away.

  690. Holger

    Well they can but some users will be yelling at you.

  691. jjrh has left

  692. Ge0rG

    Holger: yeah, that thing. Was it just a misuse of Gajim, or was that a really f***ing big problem?

  693. jcbrand has left

  694. Holger

    The main problem was some mobile and/or JavaScript libraries doing that by default or something.

  695. Steve Kille has left

  696. Holger

    Not sure about public clients. Maybe it was just Gajim.

  697. Ge0rG

    Holger: I'd like to know which clients exactly... :>

  698. Holger

    I don't remember more than that sorry.

  699. MattJ

    Meh, I'm still going to go for it

  700. jjrh has left

  701. Holger

    I think the real issue is that type=normal is overloaded to do unrelated things.

  702. Ge0rG


  703. Ge0rG

    Like MAM responses from a remote MUC

  704. Martin has left

  705. Ge0rG

    And Carbons and ACKs and many other XEPs

  706. Ge0rG

    And type influences both the meaning and the routing.

  707. zinid has left

  708. Holger

    Yes on the one hand there's these type=normal messages addressed to individual *devices* and on the other there's type=normal messages addressed to humans, i.e. to accounts.

  709. Steve Kille has joined

  710. daniel has left

  711. Holger

    If now everyone agrees that the latter should be addressed to the bare JID, the problem is partly solved.

  712. Holger

    But the RFC disagrees, and so does some of the existing client code.

  713. Ge0rG

    Holger: RFC6121 says you are not required to persist normal full-JID messages.

  714. Ge0rG

    I'm not sure resource locking is part of RFC

  715. Holger

    It's part of 6121.

  716. Holger


  717. daniel has left

  718. Ge0rG


  719. goffi has left

  720. peter has joined

  721. daniel has left

  722. SamWhited has left

  723. waqas has joined

  724. lskdjf has joined

  725. lskdjf has left

  726. ralphm has left

  727. lskdjf has left

  728. lskdjf has left

  729. lskdjf has left

  730. lskdjf has left

  731. lskdjf has joined

  732. lskdjf has left

  733. lskdjf has left

  734. lskdjf has left

  735. lskdjf has joined

  736. lskdjf has left

  737. lskdjf has joined

  738. jonasw has left

  739. daniel has left

  740. daniel has joined

  741. lskdjf has left

  742. lskdjf has joined

  743. jonasw has left

  744. lskdjf has left

  745. lskdjf has joined

  746. goffi has joined

  747. lskdjf has joined

  748. ralphm has left

  749. Valerian has joined

  750. lskdjf has left

  751. blabla has joined

  752. lskdjf has joined

  753. lskdjf has joined

  754. ralphm has left

  755. lskdjf has left

  756. lskdjf has left

  757. zinid

    Holger: I see only SHOULDs there, am I missing something?

  758. zinid

    but of course that doesn't mean existing code won't break ;)

  759. Kev has left

  760. lskdjf has left

  761. arc has left

  762. arc has joined

  763. stefandxm has joined

  764. lskdjf has left

  765. lskdjf has left

  766. Valerian has left

  767. Valerian has joined

  768. lskdjf has left

  769. lskdjf has left

  770. lskdjf has left

  771. lskdjf has left

  772. lskdjf has left

  773. lskdjf has left

  774. blabla has joined

  775. blabla has joined

  776. stefandxm has left

  777. Holger

    Yes only SHOULDs.

  778. sonny has left

  779. sonny has joined

  780. sonny has joined

  781. sonny has joined

  782. Ge0rG

    can an XML attribute value contain newlines?

  783. zinid

    Ge0rG: seems like expat eats it

  784. zinid

    but probably better to escape it

  785. blabla has joined

  786. sonny has left

  787. sonny has joined

  788. lskdjf has joined

  789. blabla has left

  790. blabla has joined

  791. blabla has joined

  792. blabla has joined

  793. stefandxm has joined

  794. ralphm has left

  795. Tobias has joined

  796. sonny has left

  797. daniel has left

  798. Valerian has left

  799. Valerian has joined

  800. Valerian has left

  801. daniel has joined

  802. Valerian has joined

  803. blabla has joined

  804. jjrh has left

  805. jjrh has left

  806. jjrh has left

  807. jjrh has left

  808. jjrh has left

  809. jjrh has left

  810. ralphm has joined

  811. jubalh has joined

  812. jjrh has left

  813. mimi89999 has joined

  814. jubalh has left

  815. lovetox has left

  816. lumi has joined

  817. valo has left

  818. valo has joined

  819. jubalh has joined

  820. jjrh has left

  821. zinid has left

  822. daniel has left

  823. Valerian has left

  824. Valerian has joined

  825. Valerian has left

  826. daniel has joined

  827. pep. has left

  828. Tobias has joined

  829. Syndace has joined

  830. jjrh has left

  831. jjrh has left

  832. pep. has left

  833. daniel has left

  834. jubalh has joined

  835. daniel has joined

  836. sonny has joined

  837. Neustradamus has left

  838. Neustradamus has joined

  839. Valerian has joined

  840. sonny has joined

  841. daniel has left

  842. daniel has joined

  843. ralphm has left

  844. Steve Kille has left

  845. jubalh has joined

  846. jjrh has left

  847. jubalh has left

  848. daniel has left

  849. daniel has joined

  850. SamWhited has left

  851. daniel has left

  852. daniel has joined

  853. ralphm has joined

  854. jere has joined

  855. Guus has left

  856. uc has joined

  857. uc has joined

  858. Alex has left

  859. jjrh has left

  860. Steve Kille has joined

  861. Alex has joined

  862. jere has left

  863. jere has joined

  864. uc has joined

  865. daniel has left

  866. ralphm has left

  867. moparisthebest has joined

  868. pep. has left

  869. jjrh has left

  870. ralphm has left

  871. jjrh has left

  872. jubalh has joined

  873. Zash has left

  874. peter has left

  875. Valerian has left

  876. Valerian has joined

  877. la|r|ma has joined

  878. Valerian has left

  879. Valerian has joined

  880. efrit has joined

  881. Kev has joined

  882. peter has joined

  883. marc has left

  884. jubalh has left

  885. daniel has left

  886. pep. has left

  887. jjrh has left

  888. vanitasvitae has left

  889. nyco has left

  890. goffi has left

  891. jjrh has left

  892. daniel has left

  893. goffi has joined

  894. uc has joined

  895. nyco has joined

  896. vanitasvitae has left

  897. marc has left

  898. daniel has left

  899. daniel has left

  900. peter has left

  901. Valerian has left

  902. Valerian has joined

  903. Valerian has left

  904. Valerian has joined

  905. uc has joined

  906. Valerian has left

  907. peter has joined

  908. vanitasvitae has left

  909. arc has left

  910. arc has joined

  911. arc has left

  912. arc has joined

  913. daniel has left

  914. daniel has left

  915. efrit has left

  916. daniel has left

  917. daniel has left

  918. Holger has left

  919. Alex has left