XSF logo XSF Discussion - 2017-11-28


  1. Tobias has joined
  2. goffi has left
  3. Zash has left
  4. jjrh has left
  5. efrit has joined
  6. daniel has left
  7. daniel has joined
  8. sonny has left
  9. sonny has joined
  10. Tobias has joined
  11. lumi has joined
  12. Kev has left
  13. Tobias has joined
  14. uc has joined
  15. vanitasvitae has left
  16. efrit has left
  17. lumi has left
  18. jere has joined
  19. Tobias has joined
  20. nyco has left
  21. nyco has joined
  22. jere has joined
  23. Zash has left
  24. arc has left
  25. arc has joined
  26. la|r|ma has left
  27. arc has left
  28. arc has joined
  29. daniel has left
  30. daniel has joined
  31. la|r|ma has joined
  32. lskdjf has joined
  33. Tobias has joined
  34. Tobias has joined
  35. arc has left
  36. arc has joined
  37. arc has left
  38. arc has joined
  39. arc has left
  40. arc has joined
  41. uc has left
  42. uc has joined
  43. sonny has left
  44. sonny has joined
  45. Ge0rG has left
  46. Ge0rG has joined
  47. Ge0rG has left
  48. Ge0rG has joined
  49. arc has left
  50. arc has joined
  51. arc has left
  52. arc has joined
  53. blabla has left
  54. SamWhited has left
  55. arc has left
  56. arc has joined
  57. arc has left
  58. arc has joined
  59. arc has left
  60. arc has joined
  61. Alacer has joined
  62. Alacer has left
  63. zinid has left
  64. zinid has left
  65. Guus has left
  66. jere has joined
  67. moparisthebest has joined
  68. ralphm has left
  69. valo has joined
  70. ralphm has left
  71. zinid yeah, nice rules: 60% of members are in council now, so council will vote for council forever
  72. jonasw what?
  73. Holger has left
  74. jonasw I’m pretty sure that’s incorrect
  75. Ge0rG indeed.
  76. jonasw https://xmpp.org/about/xsf/members.html we have many more members
  77. jonasw even though that page needs updating
  78. jonasw (preparing a PR)
  79. zinid jonasw, I checked https://wiki.xmpp.org/web/Membership_Applications_Q4_2017
  80. Ge0rG those are slightly more than nine.
  81. Ge0rG zinid: there are four application periods per year, you need to take them all together
  82. jonasw zinid, reapplication is needed yearly, not quarterly
  83. zinid ah
  84. zinid so where is the full list?
  85. jonasw so in Q4, at least four dropped out and up to one will be added
  86. jonasw I linked it, zinid
  87. jonasw https://xmpp.org/about/xsf/members.html
  88. zinid jonasw, is this page up to date?
  89. jonasw I am updating it with the new council and board election right now
  90. jonasw otherwise it is
  91. zinid ok, I didn't know you can reapply only once a year
  92. jonasw if someone could double-check I didn’t mess up here, then I’ll merge it: https://github.com/xsf/xmpp.org/pull/385
  93. zinid whatever, I actually have a question about processing empty values in data forms
  94. zinid for example, the type of the <field/> is 'jid-single' and the value is <value/>
  95. zinid what to do in this situation?
  96. zinid empty string is clearly not a JID
  97. Zash Mmmmmmm, yeah, that's somewhat ambigous IIRC
  98. zinid yeah, and if we forbid empty values, then how to clear the field?
  99. Zash <field><value/></field> == empty string and <field/> == NULL or something like that
  100. jonasw zinid, that’s simply invalid if the field is <required/>
  101. moparisthebest has joined
  102. jonasw otherwise, I’d treat it as absent
  103. jonasw there is: > Note: Data provided for fields of type "jid-single" or "jid-multi" MUST contain one or more valid Jabber IDs, where validity is determined by the addressing rules defined in XMPP Core (see the Data Validation section below).
  104. zinid jonasw, absent meaning you should ignore it, or set it as "not set"?
  105. jonasw since <value/> violates that, I’d treat the field as absent or unset
  106. Zash There's also the case of submitting a partial form.
  107. jonasw which is either an error if the field was <required/> or leads to some defaulting if it wasn’t
  108. zinid jonasw, I'm not talking about required, it's obvious in this case
  109. jonasw zinid, well, if it’s not <required/> surely the business logic of the form already knows what to do if the field is missing?
  110. zinid jonasw, indeed, what to do? keeping the field values untouched in the database or remove it?
  111. zinid *value
  112. jonasw I’d NULL it
  113. zinid I actually tend to think you should not set <value/> at all if you need to erase it
  114. jonasw possibly
  115. Zash <value/> == <value></value> == ""
  116. jonasw which isn’t a valid JID
  117. Zash Nope
  118. Zash So you get an error back
  119. jonasw so both empty <value/> and no <value/> at all violate the note I quoted above
  120. zinid another issue: the type is 'jid-multi' and you got <value/><value/><value/>
  121. jonasw I hate XEP-0004
  122. zinid nah, we just should clarify it
  123. zinid the xep is okayish
  124. zinid IMHO
  125. jonasw and cut it in two pieces
  126. zinid psi sends <value/> inside jid-multi field, I have even work-around for this in the data form parser
  127. zinid so, most likely, to keep backward compatibility we need to treap empty <value/> as no <value/> at all and should set internally the field to default value (e.g. NULL)
  128. zinid *to treat
  129. zinid simply put, treat this situation as a "default value"
  130. intosi has joined
  131. moparisthebest has joined
  132. moparisthebest has joined
  133. daniel has left
  134. daniel has joined
  135. daniel has left
  136. daniel has joined
  137. daniel has left
  138. daniel has joined
  139. daniel has left
  140. daniel has joined
  141. marc has joined
  142. ralphm has joined
  143. daniel has left
  144. daniel has joined
  145. Alex has joined
  146. Guus Georg, Kev, could you please provide a snippet for https://xmpp.org/about/xmpp-standards-foundation.html ?
  147. goffi has joined
  148. Alex has left
  149. daniel has left
  150. daniel has joined
  151. Steve Kille has left
  152. ralphm has left
  153. Steve Kille has joined
  154. vanitasvitae has left
  155. daniel has left
  156. vanitasvitae has left
  157. moparisthebest has joined
  158. moparisthebest has joined
  159. Martin has joined
  160. daniel has left
  161. daniel has left
  162. moparisthebest has joined
  163. moparisthebest has joined
  164. jcbrand has joined
  165. jere has joined
  166. ralphm has joined
  167. efrit has joined
  168. Syndace has left
  169. Syndace has joined
  170. jubalh has joined
  171. jubalh has left
  172. Alex has joined
  173. daniel has left
  174. arc has left
  175. arc has joined
  176. intosi has left
  177. jcbrand has left
  178. jcbrand has joined
  179. arc has left
  180. arc has joined
  181. arc has left
  182. arc has joined
  183. Ge0rG has joined
  184. arc has left
  185. arc has joined
  186. la|r|ma has joined
  187. Ge0rG has left
  188. ralphm has left
  189. jubalh has joined
  190. intosi has joined
  191. lskdjf has joined
  192. Ge0rG has left
  193. lumi has joined
  194. jubalh has left
  195. moparisthebest has left
  196. moparisthebest has joined
  197. pep. has left
  198. daniel has left
  199. efrit has left
  200. blabla has joined
  201. efrit has joined
  202. blabla has left
  203. blabla has joined
  204. intosi has left
  205. blabla has left
  206. blabla has joined
  207. efrit has left
  208. jere has left
  209. jere has joined
  210. lskdjf has joined
  211. efrit has joined
  212. Alex has left
  213. intosi has joined
  214. intosi has left
  215. Zash has left
  216. Alex has joined
  217. blabla has left
  218. blabla has joined
  219. blabla has left
  220. blabla has joined
  221. zinid has left
  222. Guus test
  223. daniel Guus: 👍
  224. Guus tx
  225. Guus has left
  226. Zash rx
  227. Valerian has joined
  228. blabla has left
  229. daniel has left
  230. intosi has joined
  231. moparisthebest has joined
  232. moparisthebest has joined
  233. intosi has left
  234. ralphm has left
  235. daniel has left
  236. daniel has joined
  237. jonasw has left
  238. Alex has left
  239. Alex has joined
  240. lskdjf has left
  241. lskdjf has joined
  242. Tobias has joined
  243. mathieui is there a way to know if we already voted?
  244. Kev Memberbot will tell you, I believe, when you say hello to it.
  245. jonasw yup, it will
  246. jonasw > (14:41:04) Memberbot: You have already participated in this election. Would you like to recast your votes? (yes / no)
  247. mathieui right, thanks
  248. jubalh has joined
  249. Tobias has left
  250. Holger has left
  251. Holger has left
  252. ralphm has left
  253. SouL has left
  254. daniel has left
  255. daniel has joined
  256. efrit has left
  257. Steve Kille has left
  258. efrit has joined
  259. daniel has left
  260. Valerian has left
  261. daniel has left
  262. Tobias has left
  263. Steve Kille has joined
  264. daniel has left
  265. la|r|ma has left
  266. la|r|ma has joined
  267. Valerian has joined
  268. jere has left
  269. jere has joined
  270. vanitasvitae has left
  271. ralphm has joined
  272. daniel has left
  273. daniel has left
  274. vanitasvitae has left
  275. Tobias has left
  276. jubalh has left
  277. Alex has left
  278. Alex has joined
  279. marc Ge0rG, jonasw, I plan to provide a field (optional or required, determined by the service) to specify the name of the inviter, what do you think?
  280. jonasw sorry, I lack context. where would you provide that field?
  281. marc jonasw, user invitation
  282. marc Sorry :D
  283. jonasw and where’d that field be?
  284. marc Such that we can provide something like "You were invited by X to join..." on the invitation page
  285. marc Ad-hoc command
  286. marc Filled out by the inviter
  287. jonasw hmm
  288. jonasw the issue with that type of names is that they can easily be spoofed
  289. jonasw Ge0rG had some thoughts on this type of spoofing in the context of invitations IIRC
  290. marc Sure, it's not a security feature :)
  291. jonasw it could be misleading though
  292. marc Well, you have to send the URL to somebody, if the user trust this URL then there is no problem
  293. marc (because it trust the mail, SMS, ...)
  294. Valerian has left
  295. Valerian has joined
  296. marc You can think of it as security feature if somebody tries to manipulate/replace the invitation URL ;)
  297. Ge0rG marc: I think it depends on how you implement it.
  298. Ge0rG marc: if it is an additional url parameter that contains the plaintext name, it's rather Meh.
  299. marc Ge0rG, that's up to the service
  300. Ge0rG marc: if you only attach a token to the URL and the user's JID is somehow obtained from the server via the token, it's a bit better
  301. marc The XEP defines the field
  302. marc How the invitation page is implemented is out of scope of this XEP
  303. Ge0rG marc: I see merit in defining the OOB behavior as well
  304. marc In my current implementation the name is fetched from a database via the token
  305. Ge0rG marc: so your XMPP server must also be a web server.
  306. marc Ge0rG, well, providing a web page is also optional
  307. marc My first implementation just uses the xmpp URI
  308. marc and generates a QR code
  309. marc Works nicely without web site etc.
  310. Ge0rG marc: so where is the inviter's name displayed, then?
  311. daniel has left
  312. Ge0rG marc: does it also work nicely without an XMPP client?
  313. marc Ge0rG, the name is displayed on a web site, if you provide one
  314. marc Ge0rG, what works nicely?
  315. Ge0rG marc: the QR code
  316. daniel has left
  317. marc the QR code is display in the xmpp client
  318. marc but you could also just send the xmpp uri via SMS or ...
  319. Ge0rG marc: and then?
  320. Ge0rG marc: how does your friend open an xmpp: URI without an XMPP client?
  321. marc Ge0rG, add a link with a download URL for a xmpp client ;)
  322. Ge0rG marc: why not just use easy-xmpp-invitation? :P
  323. marc Ge0rG, what's easy-xmpp-invitation?
  324. marc ah I see
  325. marc your website template
  326. marc Ge0rG, that's nice but why should it be required by the XEP?
  327. marc I prefer a web site for invitation which displays the QR code, provides information about clients etc.
  328. marc But why should this be required?
  329. daniel has left
  330. jubalh has left
  331. Ge0rG marc: how do you generate the url for the web site where the inviter's username is displayed?
  332. marc Ge0rG, the name is fetched from a database not included in the URL
  333. marc You could include it in the URL
  334. marc If you like but I don't care how that's implemented
  335. Ge0rG marc: you send me a QR code with xmpp:.... How do I know the URL of the web page?
  336. marc Out of scope IMO
  337. lovetox has joined
  338. marc Ge0rG, either I send you a URL or a bare xmpp URI
  339. Ge0rG marc: where do you get the URL from?
  340. marc In best case you're next to me and just scan the QR code from my display of my mobile phone
  341. marc Ge0rG, the URL is generated by the server and sent back to the inviter
  342. Ge0rG marc: so the response to the ad-hoc command is an xmpp: URI and a web link?
  343. Ge0rG or either?
  344. marc Always a token
  345. marc And, if provided, a URL
  346. marc the xmpp URI can be generated by the client
  347. Ge0rG So the client needs to construct the xmpp: URI from the token?
  348. marc Yes
  349. Ge0rG And the server could send back an easy-xmpp-invitation URL or a mod_invite URL or some other black magic?
  350. marc Ge0rG, it can send back a URL to some web site, yes
  351. Ge0rG Sorry for the many questions, I'm trying to understand the protocol.
  352. Ge0rG marc: will that be a generic website or a personalized one?
  353. marc I thought it is not that complicated and straightforward :D
  354. Ge0rG marc: life is full of corner cases.
  355. marc Ge0rG, depends on the server / service
  356. Ge0rG marc: so if your server sends personalized links, your client can forward the web URL and be good, but if my server returns a generic URL, and my client forwards only that, it's worthless?
  357. marc they could also provide a URL to some porn web site if they like ;)
  358. Ge0rG besides of the porn, of course.
  359. marc Ge0rG, sure, of course the web site should include the xmpp URI, QR code etc
  360. marc Otherwise this web site wouldn't make sense
  361. Ge0rG marc: it could be a generic registration form or the ToS
  362. marc No, that's not good
  363. marc Well, you could do this, of course
  364. marc But then the token is useless :D
  365. Ge0rG marc: that's my point.
  366. Ge0rG marc: so the XEP needs to specify that the returned URL is a personalized one that also contains the token (or a different token with the same functionality)
  367. daniel has left
  368. marc Ge0rG, we could also make the token optional and the server can send a URL of some generic invitation web site
  369. Ge0rG marc: you don't need a XEP for that, do you?
  370. marc Ge0rG, if you would like to have a standard for clients to request an inviation URL you do
  371. marc Otherwise, where do you get this URL from? Search on the web?
  372. marc Guessing?
  373. marc I'm not saying that this is the best "response" from a service but it is better than nothing :)
  374. Ge0rG marc: I'm saying that it's worse than nothing, if it is expected to be a specific URL
  375. Ge0rG the client needs to decide based on what it receives.
  376. Ge0rG if it receives [URL=https://yax.im/register, token=deadbeaf], it doesn't know whether it must send on both or only the URL
  377. Ge0rG so the URL needs to provide the same functionality as the token
  378. Ge0rG besides, it might be useful to return not just a token but an xmpp: URI, because the server might be better suited to construct it than the client
  379. Ge0rG then the server could return xmpp:free-hosting.com?token=deadbeef to a client logged in as user@legacy.free-hosting.com
  380. marc Sure, the server can also return the xmpp URI
  381. lskdjf has left
  382. marc Ge0rG, but the URL can still be optional, right?
  383. daniel has left
  384. Tobias has joined
  385. Ge0rG marc: yes. The URL should be optional, I just would like to prevent "smart" server operators entering a generic URL there
  386. marc Ge0rG, yes, but we can not avoid it anyway ;)
  387. Ge0rG marc: but we can make it illegal via the XEP
  388. marc :D
  389. marc Ge0rG, btw, I think your easy-xmpp web site should auto-detect the operating system / browser and suggest a client :)
  390. Ge0rG marc: I think you are right, which is why I wrote that into the TODO
  391. marc ah nice ;)
  392. marc didn't read it
  393. Ge0rG too bad :P
  394. sonny has joined
  395. Kev I've just updated the Board mailing list. It's now current members + Council Chair + ED + Secretary.
  396. Guus ... ED ...
  397. Guus Ah, Exec. Director?
  398. Guus Thanks Kev
  399. sonny has joined
  400. jonasw Guus, so I’m not the only one having to re-think each time they see the abbreviation "ED"
  401. Guus "each time" <-- you've seen it before then? :)
  402. Valerian has left
  403. efrit has left
  404. moparisthebest ha well I found a fool proof spam prevention system that would work for xmpp too, it's terrible, but it'd work
  405. moparisthebest our support got a ticket at work from a user who wasn't getting our emails because they have a spam prevention system that replies to the email with a link the sender has to click to allow the email through, or whitelist the address, or something
  406. efrit has joined
  407. jonasw Guus, members list
  408. moparisthebest so for unsolicited chat or subscription requests, server could message the user an http link that needs clicked before allowing it through... :/
  409. SamWhited moparisthebest: that's basically the same as the proof-of-work model we talked about, except less automated
  410. moparisthebest where is this discussion? must have missed it
  411. SamWhited that could be the fallback if a PoW message gets sent but the potential spammers client doesn't support it
  412. SamWhited moparisthebest: I'm not sure where or when; it was a while ago. TL;DR if you get a message from someone you don't know, send their client a relatively expensive problem they have to compute and respond to before you'll show the message to the user
  413. moparisthebest I actually think this would work better in xmpp vs email, this user wanted the person at our company monitoring the system.noreply@ourdomain.com email to click the link...
  414. moparisthebest yea the combination might be good
  415. SamWhited I've got a TODO to write up a spec for that actually; I should do that this weekend.
  416. jubalh has joined
  417. moparisthebest you should :)
  418. jonasw *SHOULD :-)
  419. SamWhited moves it up to the top of his list from the bottom that's off the screen and therefore was forgotten about
  420. moparisthebest with a message a link fallback, servers could sanely turn it on before client support was widespread
  421. SamWhited actually, my TODO was for using it with IBR2, but the same challenge could be reused for both probably
  422. moparisthebest and you don't need to specify what happens with the link, if that happens you expect human intervention, servers might whitelist automatically, have a captcha, or whatever
  423. Holger What part of this needs a spec?
  424. moparisthebest the automated proof of work part
  425. moparisthebest the sending a link would not
  426. SamWhited Holger: the link part doesn't, but if we want clients to automatically respond to PoW challenges from the server that part needs a spec
  427. vanitasvitae has left
  428. Holger Ah the client responds without automatically? What happens to people with old clients?
  429. Holger s/without//
  430. SamWhited Holger: that's why you include a link to a captcha or something in the body. Old clients show that, you can click it and complete a human challenge instead
  431. MattJ They click the link
  432. Holger Ah.
  433. SamWhited Just like joining MUCs on Jabber.ru, which I think does this with clients that don't support their captcha forms (you get a link to the same form on the web)
  434. MattJ and here you go: https://xmpp.org/extensions/xep-0158.html#challenge-hashcash
  435. MattJ Later in the document: "A challenger MAY provide a text question in the <body/> element of a challenge stanza for clients that do not support CAPTCHA forms."
  436. SamWhited oh hey, would you look at that, been done
  437. MattJ The problem I see with proof-of-work is that spammers have access to lots of CPU cycles (that typically aren't really theirs), and real users don't
  438. SamWhited It's true, it doesn't stop botnets, it just stops every person with a laptop from being able to register hundreds of accounts and then send spam with them all day
  439. Ge0rG Especially mobile users don't.
  440. MattJ So you'll flatten a user's battery a bit, and you'll cost a spammer some miniscule amount of money on a botnet or captcha-solving platform
  441. SamWhited *slows down (not stops)
  442. SamWhited It still forces them to use a botnet or captcha-solving platform, which is more work. And most users will rarely have to do this, only spammers would need to do it regularly
  443. MattJ Pretty sure the user's battery will drain before the spammers deem it not cost-effective
  444. SamWhited I suspect anyways; I bet most "normal" people only communicate with people on their roster, and if we drain a users battery because their phone is in a botnet I'm pretty okay with that
  445. MattJ SamWhited, we had CAPTCHA on register.jabber.org, it didn't stop them, just slowed them down - doesn't solve much at the end of the day
  446. SamWhited Slowing them down is really the only point; it's the best we can do
  447. Ge0rG Let's slow them down by retracting subscriptions from known spammers.
  448. uc has joined
  449. moparisthebest it wouldn't drain a mobile users battery at all, except a tiny amount once when they send a message to a new user they haven't sent one to before, right?
  450. Ge0rG moparisthebest: there is a little problem there: PoW is much more energy-expensive on a mobile CPU than on a GPU cluster.
  451. moparisthebest and if botnets do indeed bother and implement this, you can just fall back to manual human-intervention-required link challenge
  452. moparisthebest Ge0rG, but it happens once per new message to new user max?
  453. moparisthebest so like, maybe 30 times over the course of an entire xmpp account's life?
  454. moparisthebest maybe other people are far more popular than I am idk
  455. jonasw Ge0rG, depends on the PoW
  456. jonasw memory-hard proofs come into mind
  457. ralphm has left
  458. jonasw scrypt or something
  459. moparisthebest and yea you could do something that is harder on gpus, yep scrypt
  460. Ge0rG jonasw: right, because OOM isn't a thing on mobile :P
  461. daniel has left
  462. jonasw Ge0rG, it is, but using lareg amount of memory is expensive on GPUs
  463. jonasw (and on FPGAs)
  464. Ge0rG jonasw: we aren't talking about bitcoin mining parallelization here
  465. Ge0rG or whatever-scrypt-coin
  466. jonasw where large is something like ~100 MiB already, depending on the type of operations
  467. Ge0rG You can't expect an old Android phone to provide 100MB (plus JVM overhead) to calculate a PoW
  468. jonasw I would expect that to work actually
  469. jonasw I bet firefox needs more
  470. jonasw firefox gets swapped out then, I’m fine with that
  471. Ge0rG jonasw: "Bug report: my browser gets killed when I add a friend"
  472. daniel has left
  473. moparisthebest an old android phone can't run conversations either so who cares
  474. Ge0rG I think this is insane.
  475. jonasw your browser gets killed always anyways on those machines.
  476. mathieui this is crazy, yes
  477. SamWhited We'd have to think about that, but the point is that we can think of something that's not too bad for mobile users but still does the job
  478. moparisthebest then you disable it on your client and solve http links by hand Ge0rG ?
  479. jonasw I can’t really use my broswer + any other app on my Galaxy S3.
  480. Ge0rG moparisthebest: no, I just switch to WhatsBook.
  481. jonasw with WhatsBook, you need mutual subscription first anyways?
  482. moparisthebest are you thinking this is something that would happen often?
  483. jonasw I thought we had that as a possible sensible solution, too
  484. Holger jonasw: You don't.
  485. efrit has left
  486. jonasw Holger, hm, when we discussed this a few weeks ago, I think the consensus was that you need it, but w/e
  487. daniel has left
  488. Ge0rG moparisthebest: unless we make PoW prohibitively expensive for normal users, spammers won't be slowed / stopped by it.
  489. Holger jonasw: I remember someone claiming that and me not finding a single commercial messenger that actually does that.
  490. Holger jonasw: And I think it's unusable as a general solution.
  491. Holger Though admittedly this PoW idea sounds even worse.
  492. SamWhited If memory consumption and GPUs turn out to really be an issue it could also use an algorithm that relies on cache locality
  493. vanitasvitae has left
  494. vanitasvitae has joined
  495. moparisthebest Ge0rG, bcrypt and scrypt would like to disagree with you
  496. moparisthebest they were explicitly designed to be not so bad for normal users, and prohibitive for bad guys
  497. jonasw funny question, couldn’t the PoW be solved by a users server?
  498. MattJ Yes, this would be interesting :)
  499. mathieui new DoS way
  500. jonasw you’d quota that of course etc., but for the occasional adding of a contact…?
  501. MattJ mathieui, it would encourage server admins to lock down their servers better :)
  502. jonasw like, 3 PoW / day / user, 60 PoW / hour in total or so
  503. SamWhited jonasw: that's an interesting idea; sounds like a possible DoS, but servers could be smart about it and say "this user is generating too many PoWs, what are they doing?"
  504. jonasw SamWhited, exactly
  505. jonasw that’d solve the mobile issue
  506. moparisthebest that only helps good public servers being abused to send spam
  507. Ge0rG moparisthebest: https://bitcoinware.net/collections/gridseed-dual-ltc-and-btc-miner
  508. moparisthebest but, good thing to help I guess
  509. mathieui moparisthebest, and it will slow down bad servers just as well
  510. jonasw it also gives servers an interesting metric on users sending a lot of subscriptions/new messages
  511. SamWhited That only works if the server supports the PoW thing though; otherwise you still have to issue it to the client (or issue a captcha)
  512. jonasw and if the server is overloaded with PoW, it could simply forward it to the client.
  513. Ge0rG it would be better to have users do PoW to pay for their server.
  514. jonasw SamWhited, ofc.
  515. SamWhited So if a spammer has spun up their own server, you still have to support the original model
  516. moparisthebest Ge0rG, so xmpp spammers are going to buy super expensive miners, and try to use them for xmpp PoW ? seems unlikely
  517. jonasw "the original model", SamWhited?
  518. SamWhited jonasw: sending a captcha/PoW to the client, I mean
  519. jonasw you’d send it to the user, the server can decide to intercept the PoW request and handle it by itself or forward it to the client
  520. Ge0rG moparisthebest: spammers will either distribute PoW to their botnet, where you will be paying for the PoW, or rent some gigahashes.
  521. jonasw I’m not sure what you mean
  522. moparisthebest I don't think they really have botnets now, just register on open servers and spam until banned
  523. Ge0rG moparisthebest: please stop arguing. The PoW battle has been lost.
  524. jonasw Ge0rG, not sure
  525. moparisthebest there is no golden solves everything problem, you can only annoy them a bit, slow them down for a bit
  526. daniel has left
  527. jonasw if you have a botnet, you’re probably better off putting that computing power into $cryptocoin
  528. jonasw instead of trying to spam
  529. moparisthebest it's an arms race :P
  530. Ge0rG moparisthebest: we can make life really miserable for mobile users and slightly annoy spammers.
  531. moparisthebest you still haven't said how it would hurt mobile users at all?
  532. Ge0rG jonasw: you need orders of magnitude more power for viable commercial mining
  533. SamWhited Right, if you have a botnet to spam with we can't stop you right now, but we can slow down your botnet so that you send less spam and possibly make it prohibitively expensive for the people who spin up a single server in their basemenet to send out spam
  534. la|r|ma has left
  535. moparisthebest are you mistakenly thinking this would happen on every message or something?
  536. SamWhited And we can do it without affecting users at all, I suspect
  537. SamWhited *well behaved users
  538. moparisthebest how often do you add or message new users? how often does a normal user do that? I'm thinking very rarely
  539. Ge0rG moparisthebest: just to repeat my argument: if you make it sufficiently hard for spammers, it will be prohibitive for normal users.
  540. moparisthebest and again that's just wrong Ge0rG
  541. SamWhited Ge0rG: we're disagreeing with that argument. Why do you think it would be prohibitvely expensive for normal users?
  542. jonasw I doubt we’ll reach an argument here
  543. jonasw we need data
  544. Ge0rG moparisthebest: okay, suggest a number of scrypt operations a user needs to perform for an initial contact.
  545. jonasw maybe some google scholar-ing on how botnets operate nowadays?
  546. SamWhited Spammers send lots of messages to people that aren't on their contact lists, normal users don't. That asymetry is what we're targeting.
  547. Ge0rG SamWhited: we can target that with simple statistics, without annoying anyone.
  548. moparisthebest Ge0rG, whatever takes about 4ish seconds on say a samsung galaxy s4
  549. SamWhited Ge0rG: where and how do you get those statistics?
  550. MattJ SamWhited, I agree that I think this is the best place to solve this problem
  551. Holger "Q: My app crashes (or crashes other apps) when adding a contact. A: Don't worry, I know that you don't often add contacts."
  552. moparisthebest yea you wouldn't make it crash
  553. jonasw also, shouldn’t this discussion move to spam@?
  554. SamWhited Holger: that's why we have to do research and find something that won't OOM everyone but is still relatively tricky
  555. SamWhited Too many rooms.
  556. jonasw Holger, I still claim that users on devices with this low amount of memory are used to that
  557. Holger We can add that to our response then.
  558. Ge0rG Okay, just to get some numbers. "I managed to pull around 5.6KHash/sec on my Nexus 7 with all four threads." from https://rumorscity.com/2014/01/07/how-to-mine-litecoin-with-android/
  559. MattJ Proof of work aside, it's very easy to identify accounts sending to a lot of non-contacts. Spammers will switch to subscription requests, but it's equally easy to spot (new?) accounts sending lots of subscription requests, and this is uncommon (not impossible for a legitimate user, just uncommon)
  560. Holger Indeed.
  561. Ge0rG So we are at ~20 KHashes for a first-contact
  562. moparisthebest MattJ, you mean easy if you run a server with lots of users I guess?
  563. MattJ Easy on any server
  564. jonasw MattJ, you assume that the server isn’t malicious
  565. moparisthebest or, easy for a public server to spot new users of it's server
  566. jonasw I find that assumption incorrect
  567. SamWhited MattJ: I agree, server operators should be doing that
  568. MattJ jonasw, that is an assumption in this case, malicious servers are a different thing
  569. waqas has joined
  570. moparisthebest yea explain how my private server with 4 xmpp accounts can see any of this data, easily or not
  571. MattJ But that's not a problem we have today
  572. stefandxm has left
  573. jonasw MattJ, it is
  574. jonasw I think
  575. Holger And malicious servers won't respond to PoW requests?
  576. moparisthebest Holger, if they don't do the pow or click the link and do whatever is there, no requests get through to the client?
  577. jonasw fighting spam at the source is of course most sensible, but hard in a distributed system
  578. moparisthebest so, it's not a problem
  579. SamWhited Holger: they will respond to PoW requests, and that's the point. It will slow them down because they'll be responding to *lots* of them.
  580. moparisthebest and if they do pow and too much spam still gets through, turn off pow and go back to manual links
  581. mathieui also: if a server implements the PoW thing, it could "wall" the subscriptions unless it’s mutual
  582. moparisthebest make them play an html5 punch the monkey game and beat a high score to whitelist their jid :P
  583. mathieui (e.g. "user A and user B want to communicate, B’s server does not implement PoW, B adds A, A sees nothing; A adds B because they know about it: the subscription is established")
  584. moparisthebest or A's server sends an https link to user B and when clicked lets it go through
  585. Ge0rG you can rent 500MH/s for three hours for ~3USD. That accounts for 100 Millions spam messages.
  586. Ge0rG if you price a single spam message at 20KH scrypt.
  587. MattJ Case closed :)
  588. Holger SamWhited: I got the idea, I just don't see the gain in throttling them to, dunno, some hundreds of thousands of messages per day and CPU core or whatever. Do they really send millions per day right now?
  589. Holger What Ge0rG said.
  590. jonasw Ge0rG, "ouch"
  591. Ge0rG moparisthebest: so will you stop now?
  592. zinid right, what will happen is that spam will become a bit more expensive for the customers 🙂
  593. Ge0rG source for MH price: https://www.miningrigrentals.com/rigs/scrypt
  594. SamWhited Holger: I don't know. More research would definitely be required, I just suspect we could slow them down a bit
  595. moparisthebest and are you sure you can use that for arbitrary scrypt challenges Ge0rG ?
  596. Ge0rG moparisthebest: again, spammers will just use whatever botnet they can get away with.
  597. jonasw moparisthebest, if you can’t *now*, as soon as you can use this to make some financial gain (e.g. spam), they will adapt so that it can be used for that
  598. daniel has left
  599. Ge0rG moparisthebest: even when running on desktop Windows malware, you are several orders of magnitude faster than on a mobile device.
  600. moparisthebest possibly, but then you are excluding spammers who don't have botnets
  601. jonasw moparisthebest, even a spammer with their own server can easily outperform that challenge
  602. moparisthebest are there challenges that are faster on ARM than amd64 ?
  603. Ge0rG moparisthebest: let's just exclude all spammers by definition. Congratulations, we have won the spam war.
  604. jonasw 20 kH is not much
  605. Ge0rG I can go back to work now.
  606. SamWhited Botnets would (hopefully) still be slowed down, non-botnets it might be too expensive. It's not a panacea, but it still seems like it could be a benefit if the idea works.
  607. moparisthebest right there is no panacea
  608. Ge0rG Right. So please pretty please let's focus on the ideas that make it actually worse for spammers than for users.
  609. moparisthebest like this one
  610. moparisthebest or, what is your other idea?
  611. Ge0rG moparisthebest: read up my posts on the operators@ and standards@ MLs.
  612. zinid digitalocean started to provide high performance servers for computational tasks, for the record, I think 500MH/s would be nothing for those
  613. jjrh has left
  614. Ge0rG moparisthebest: I've exceeded my time budget for convincing you on pointless scrypt performance calculations, sorry.
  615. Holger moparisthebest: Mine is server-side filtering based on both message contents and meta data. Works relatively well for email.
  616. moparisthebest Ge0rG, I've been following a bit, but iirc most are geared towards running large-ish public servers with lots of data to analyze
  617. moparisthebest and nothing for small private servers
  618. Holger SpamAssassin works just fine for small private email servers.
  619. moparisthebest it does for email, I was under the impression most xmpp spam was too small for it to be effective
  620. Ge0rG there are many theoretical solutions that just won't work out. Like a distributed p2p reputation system.
  621. SamWhited Holger: I've been wondering about that too; do you have that tied to an XMPP server? How well does it work for the shorter XMPP messages?
  622. Ge0rG Some months ago, 99% of spam was multi-line Russian with multiple links.
  623. Ge0rG Easy to kill, just filter multi-line messages from strangers.
  624. Ge0rG Then, they switched to single-line messages with two pastebin links. Still pretty easy.
  625. moparisthebest sure and it's easy to do stuff like that, doesn't solve everything though, just like PoW doesn't
  626. SamWhited Those are good things to do as well
  627. Ge0rG moparisthebest: yes, but PoW WILL ANNOY USERS
  628. Holger SamWhited: I think it'll work quite well. Problem is it requires work.
  629. zinid Holger, except that Bayes sucks for short messages
  630. moparisthebest besides I look forward to a future when all messages are encrypted base64 blobs :P
  631. Holger moparisthebest: So all solutions are equally adequate because none are perfect?
  632. Ge0rG Now they send a subscription request + multiline / pastebin
  633. SamWhited Yah, I suspect zinid is right and it won't work so well for XMPP messages, but I'd love to be proven wrong
  634. Holger zinid: Yes I would definitely not rely just on Bayes.
  635. moparisthebest no, I'm saying you should implement non-perfect solutions because those are the only solutions we have Holger
  636. waqas has left
  637. Ge0rG moparisthebest: you are saying we should implement bad solutions because there are no perfect ones.
  638. lumi has left
  639. Holger moparisthebest: And I'm saying some non-perfect solutions are way better than others. So "nothing is perfect" is not a very convincing reasoning.
  640. Ge0rG I haven't seen a spam message in weeks. Just the subscription requests
  641. zinid Holger, there is very little research for short messages, I recall a couple of papers only, you can search google scholar with "sms spam" query
  642. Ge0rG BTW, my next proposals would be: - revert a pending subscription if a sender is classified as spam - implement a cache of URLs sent by strangers, with counters, and block messages with a count > 3
  643. Ge0rG MattJ: ^
  644. moparisthebest and, again, both of those solutions only work for large public servers Ge0rG
  645. moparisthebest I'm literally the only user of my xmpp server to get any spam
  646. Ge0rG moparisthebest: you know what? spammers are sending messages to non-existent accounts. You could easily set up a honeypot
  647. SamWhited Ge0rG: I agree, both of those things are a good start and should probably be done (although they also probably need a way of dealing with false positives, but that's a different problem)
  648. Ge0rG moparisthebest: please log on your server messages sent to non-existent accounts
  649. moparisthebest I'm not saying they aren't good solutions for large public servers, or that you shouldn't implement them, I just also want something that makes running your own server practical
  650. MattJ Ge0rG, #2 won't work, the URLs can vary too easily
  651. zinid moparisthebest, set spam traps 🙂
  652. Ge0rG MattJ: it's expensive to create a large number of shortened links
  653. MattJ Trivial to append #uuid
  654. Syndace has left
  655. Ge0rG MattJ: besides, we could HEAD them :P
  656. MattJ Trivial to append ?uuid
  657. zinid moparisthebest, i.e. non-unsed accounts, but you publish them everywhere 😉
  658. moparisthebest yea but is that what we suggest to people wanting to run their own servers
  659. moparisthebest start your server, setup SRV records, post non-existant accounts at random places
  660. Holger zinid: Yes, I don't have papers to prove my point. But many of the criteria used for email classification are unrelated to the length of the message body.
  661. SamWhited Holger, zinid: seems worth trying either way
  662. Ge0rG moparisthebest: you could just block messages from strangers with an URL.
  663. zinid Holger, yes, maybe you can do that, no doubt, although the result is unpredictable
  664. Ge0rG yax.im will send an error response "Blocked due to abuse", so a real sender actually will see a message about being blocked
  665. uc has joined
  666. moparisthebest Ge0rG, what about subscription spam
  667. zinid what about captcha btw?
  668. Ge0rG moparisthebest: see above
  669. moparisthebest also none of this works for encryption
  670. moparisthebest spammers will soon start sending omemo messages :P
  671. Holger Yes that's one of my complaints with E2EE.
  672. Ge0rG moparisthebest: I'm sure they will
  673. Ge0rG I'm eagerly awaiting the day when I can block E2EE on my server :P
  674. zinid Holger, ah, yes, E2EE, hehe
  675. moparisthebest so back around to the original plan of replying with a https link with arbitrary things to solve on it
  676. Ge0rG moparisthebest: yes, let your friends play xbill or whack-a-mole.
  677. Ge0rG or solve a reCAPTCHA.
  678. moparisthebest yes, once, doesn't seem like too much to ask?
  679. mathieui maybe we could embed flash applications as base64 inside subscription replies
  680. Ge0rG moparisthebest: you can do all that on your private server
  681. Ge0rG moparisthebest: but it doesn't work well for public services
  682. moparisthebest obviously doesn't happen if you send them an easy xmpp invite url or whatever
  683. moparisthebest mathieui, it's 2017 you mean html5 applications
  684. moparisthebest seems like it'd work equally well for public servers?
  685. moparisthebest public servers just additionally have much more to go on, so maybe it wouldn't always be necessary
  686. Ge0rG Holger: please tell MattJ about the normal-message-to-full-JID thing.
  687. Holger The "some clients do that so servers can't throw the message away" thing?
  688. jjrh has left
  689. Holger Some clients do that so servers can't throw the message away.
  690. Holger Well they can but some users will be yelling at you.
  691. jjrh has left
  692. Ge0rG Holger: yeah, that thing. Was it just a misuse of Gajim, or was that a really f***ing big problem?
  693. jcbrand has left
  694. Holger The main problem was some mobile and/or JavaScript libraries doing that by default or something.
  695. Steve Kille has left
  696. Holger Not sure about public clients. Maybe it was just Gajim.
  697. Ge0rG Holger: I'd like to know which clients exactly... :>
  698. Holger I don't remember more than that sorry.
  699. MattJ Meh, I'm still going to go for it
  700. jjrh has left
  701. Holger I think the real issue is that type=normal is overloaded to do unrelated things.
  702. Ge0rG Right.
  703. Ge0rG Like MAM responses from a remote MUC
  704. Martin has left
  705. Ge0rG And Carbons and ACKs and many other XEPs
  706. Ge0rG And type influences both the meaning and the routing.
  707. zinid has left
  708. Holger Yes on the one hand there's these type=normal messages addressed to individual *devices* and on the other there's type=normal messages addressed to humans, i.e. to accounts.
  709. Steve Kille has joined
  710. daniel has left
  711. Holger If now everyone agrees that the latter should be addressed to the bare JID, the problem is partly solved.
  712. Holger But the RFC disagrees, and so does some of the existing client code.
  713. Ge0rG Holger: RFC6121 says you are not required to persist normal full-JID messages.
  714. Ge0rG I'm not sure resource locking is part of RFC
  715. Holger It's part of 6121.
  716. Holger https://tools.ietf.org/html/rfc6121#section-5.1
  717. daniel has left
  718. Ge0rG Bummer.
  719. goffi has left
  720. peter has joined
  721. daniel has left
  722. SamWhited has left
  723. waqas has joined
  724. lskdjf has joined
  725. lskdjf has left
  726. ralphm has left
  727. lskdjf has left
  728. lskdjf has left
  729. lskdjf has left
  730. lskdjf has left
  731. lskdjf has joined
  732. lskdjf has left
  733. lskdjf has left
  734. lskdjf has left
  735. lskdjf has joined
  736. lskdjf has left
  737. lskdjf has joined
  738. jonasw has left
  739. daniel has left
  740. daniel has joined
  741. lskdjf has left
  742. lskdjf has joined
  743. jonasw has left
  744. lskdjf has left
  745. lskdjf has joined
  746. goffi has joined
  747. lskdjf has joined
  748. ralphm has left
  749. Valerian has joined
  750. lskdjf has left
  751. blabla has joined
  752. lskdjf has joined
  753. lskdjf has joined
  754. ralphm has left
  755. lskdjf has left
  756. lskdjf has left
  757. zinid Holger: I see only SHOULDs there, am I missing something?
  758. zinid but of course that doesn't mean existing code won't break ;)
  759. Kev has left
  760. lskdjf has left
  761. arc has left
  762. arc has joined
  763. stefandxm has joined
  764. lskdjf has left
  765. lskdjf has left
  766. Valerian has left
  767. Valerian has joined
  768. lskdjf has left
  769. lskdjf has left
  770. lskdjf has left
  771. lskdjf has left
  772. lskdjf has left
  773. lskdjf has left
  774. blabla has joined
  775. blabla has joined
  776. stefandxm has left
  777. Holger Yes only SHOULDs.
  778. sonny has left
  779. sonny has joined
  780. sonny has joined
  781. sonny has joined
  782. Ge0rG can an XML attribute value contain newlines?
  783. zinid Ge0rG: seems like expat eats it
  784. zinid but probably better to escape it
  785. blabla has joined
  786. sonny has left
  787. sonny has joined
  788. lskdjf has joined
  789. blabla has left
  790. blabla has joined
  791. blabla has joined
  792. blabla has joined
  793. stefandxm has joined
  794. ralphm has left
  795. Tobias has joined
  796. sonny has left
  797. daniel has left
  798. Valerian has left
  799. Valerian has joined
  800. Valerian has left
  801. daniel has joined
  802. Valerian has joined
  803. blabla has joined
  804. jjrh has left
  805. jjrh has left
  806. jjrh has left
  807. jjrh has left
  808. jjrh has left
  809. jjrh has left
  810. ralphm has joined
  811. jubalh has joined
  812. jjrh has left
  813. mimi89999 has joined
  814. jubalh has left
  815. lovetox has left
  816. lumi has joined
  817. valo has left
  818. valo has joined
  819. jubalh has joined
  820. jjrh has left
  821. zinid has left
  822. daniel has left
  823. Valerian has left
  824. Valerian has joined
  825. Valerian has left
  826. daniel has joined
  827. pep. has left
  828. Tobias has joined
  829. Syndace has joined
  830. jjrh has left
  831. jjrh has left
  832. pep. has left
  833. daniel has left
  834. jubalh has joined
  835. daniel has joined
  836. sonny has joined
  837. Neustradamus has left
  838. Neustradamus has joined
  839. Valerian has joined
  840. sonny has joined
  841. daniel has left
  842. daniel has joined
  843. ralphm has left
  844. Steve Kille has left
  845. jubalh has joined
  846. jjrh has left
  847. jubalh has left
  848. daniel has left
  849. daniel has joined
  850. SamWhited has left
  851. daniel has left
  852. daniel has joined
  853. ralphm has joined
  854. jere has joined
  855. Guus has left
  856. uc has joined
  857. uc has joined
  858. Alex has left
  859. jjrh has left
  860. Steve Kille has joined
  861. Alex has joined
  862. jere has left
  863. jere has joined
  864. uc has joined
  865. daniel has left
  866. ralphm has left
  867. moparisthebest has joined
  868. pep. has left
  869. jjrh has left
  870. ralphm has left
  871. jjrh has left
  872. jubalh has joined
  873. Zash has left
  874. peter has left
  875. Valerian has left
  876. Valerian has joined
  877. la|r|ma has joined
  878. Valerian has left
  879. Valerian has joined
  880. efrit has joined
  881. Kev has joined
  882. peter has joined
  883. marc has left
  884. jubalh has left
  885. daniel has left
  886. pep. has left
  887. jjrh has left
  888. vanitasvitae has left
  889. nyco has left
  890. goffi has left
  891. jjrh has left
  892. daniel has left
  893. goffi has joined
  894. uc has joined
  895. nyco has joined
  896. vanitasvitae has left
  897. marc has left
  898. daniel has left
  899. daniel has left
  900. peter has left
  901. Valerian has left
  902. Valerian has joined
  903. Valerian has left
  904. Valerian has joined
  905. uc has joined
  906. Valerian has left
  907. peter has joined
  908. vanitasvitae has left
  909. arc has left
  910. arc has joined
  911. arc has left
  912. arc has joined
  913. daniel has left
  914. daniel has left
  915. efrit has left
  916. daniel has left
  917. daniel has left
  918. Holger has left
  919. Alex has left