yeah, nice rules: 60% of members are in council now, so council will vote for council forever
jonasw
what?
Holgerhas left
jonasw
I’m pretty sure that’s incorrect
Ge0rG
indeed.
jonasw
https://xmpp.org/about/xsf/members.html we have many more members
jonasw
even though that page needs updating
jonasw
(preparing a PR)
zinid
jonasw, I checked https://wiki.xmpp.org/web/Membership_Applications_Q4_2017
Ge0rG
those are slightly more than nine.
Ge0rG
zinid: there are four application periods per year, you need to take them all together
jonasw
zinid, reapplication is needed yearly, not quarterly
zinid
ah
zinid
so where is the full list?
jonasw
so in Q4, at least four dropped out and up to one will be added
jonasw
I linked it, zinid
jonasw
https://xmpp.org/about/xsf/members.html
zinid
jonasw, is this page up to date?
jonasw
I am updating it with the new council and board election right now
jonasw
otherwise it is
zinid
ok, I didn't know you can reapply only once a year
jonasw
if someone could double-check I didn’t mess up here, then I’ll merge it: https://github.com/xsf/xmpp.org/pull/385
zinid
whatever, I actually have a question about processing empty values in data forms
zinid
for example, the type of the <field/> is 'jid-single' and the value is <value/>
zinid
what to do in this situation?
zinid
empty string is clearly not a JID
Zash
Mmmmmmm, yeah, that's somewhat ambigous IIRC
zinid
yeah, and if we forbid empty values, then how to clear the field?
Zash
<field><value/></field> == empty string and <field/> == NULL or something like that
jonasw
zinid, that’s simply invalid if the field is <required/>
moparisthebesthas joined
jonasw
otherwise, I’d treat it as absent
jonasw
there is:
> Note: Data provided for fields of type "jid-single" or "jid-multi" MUST contain one or more valid Jabber IDs, where validity is determined by the addressing rules defined in XMPP Core (see the Data Validation section below).
zinid
jonasw, absent meaning you should ignore it, or set it as "not set"?
jonasw
since <value/> violates that, I’d treat the field as absent or unset
Zash
There's also the case of submitting a partial form.
jonasw
which is either an error if the field was <required/> or leads to some defaulting if it wasn’t
zinid
jonasw, I'm not talking about required, it's obvious in this case
jonasw
zinid, well, if it’s not <required/> surely the business logic of the form already knows what to do if the field is missing?
zinid
jonasw, indeed, what to do? keeping the field values untouched in the database or remove it?
zinid
*value
jonasw
I’d NULL it
zinid
I actually tend to think you should not set <value/> at all if you need to erase it
jonasw
possibly
Zash
<value/> == <value></value> == ""
jonasw
which isn’t a valid JID
Zash
Nope
Zash
So you get an error back
jonasw
so both empty <value/> and no <value/> at all violate the note I quoted above
zinid
another issue: the type is 'jid-multi' and you got <value/><value/><value/>
jonasw
I hate XEP-0004
zinid
nah, we just should clarify it
zinid
the xep is okayish
zinid
IMHO
jonasw
and cut it in two pieces
zinid
psi sends <value/> inside jid-multi field, I have even work-around for this in the data form parser
zinid
so, most likely, to keep backward compatibility we need to treap empty <value/> as no <value/> at all and should set internally the field to default value (e.g. NULL)
zinid
*to treat
zinid
simply put, treat this situation as a "default value"
intosihas joined
moparisthebesthas joined
moparisthebesthas joined
danielhas left
danielhas joined
danielhas left
danielhas joined
danielhas left
danielhas joined
danielhas left
danielhas joined
marchas joined
ralphmhas joined
danielhas left
danielhas joined
Alexhas joined
Guus
Georg, Kev, could you please provide a snippet for https://xmpp.org/about/xmpp-standards-foundation.html ?
goffihas joined
Alexhas left
danielhas left
danielhas joined
Steve Killehas left
ralphmhas left
Steve Killehas joined
vanitasvitaehas left
danielhas left
vanitasvitaehas left
moparisthebesthas joined
moparisthebesthas joined
Martinhas joined
danielhas left
danielhas left
moparisthebesthas joined
moparisthebesthas joined
jcbrandhas joined
jerehas joined
ralphmhas joined
efrithas joined
Syndacehas left
Syndacehas joined
jubalhhas joined
jubalhhas left
Alexhas joined
danielhas left
archas left
archas joined
intosihas left
jcbrandhas left
jcbrandhas joined
archas left
archas joined
archas left
archas joined
Ge0rGhas joined
archas left
archas joined
la|r|mahas joined
Ge0rGhas left
ralphmhas left
jubalhhas joined
intosihas joined
lskdjfhas joined
Ge0rGhas left
lumihas joined
jubalhhas left
moparisthebesthas left
moparisthebesthas joined
pep.has left
danielhas left
efrithas left
blablahas joined
efrithas joined
blablahas left
blablahas joined
intosihas left
blablahas left
blablahas joined
efrithas left
jerehas left
jerehas joined
lskdjfhas joined
efrithas joined
Alexhas left
intosihas joined
intosihas left
Zashhas left
Alexhas joined
blablahas left
blablahas joined
blablahas left
blablahas joined
zinidhas left
Guus
test
daniel
Guus: 👍
Guus
tx
Guushas left
Zash
rx
Valerianhas joined
blablahas left
danielhas left
intosihas joined
moparisthebesthas joined
moparisthebesthas joined
intosihas left
ralphmhas left
danielhas left
danielhas joined
jonaswhas left
Alexhas left
Alexhas joined
lskdjfhas left
lskdjfhas joined
Tobiashas joined
mathieui
is there a way to know if we already voted?
Kev
Memberbot will tell you, I believe, when you say hello to it.
jonasw
yup, it will
jonasw
> (14:41:04) Memberbot: You have already participated in this election. Would you like to recast your votes? (yes / no)
mathieui
right, thanks
jubalhhas joined
Tobiashas left
Holgerhas left
Holgerhas left
ralphmhas left
SouLhas left
danielhas left
danielhas joined
efrithas left
Steve Killehas left
efrithas joined
danielhas left
Valerianhas left
danielhas left
Tobiashas left
Steve Killehas joined
danielhas left
la|r|mahas left
la|r|mahas joined
Valerianhas joined
jerehas left
jerehas joined
vanitasvitaehas left
ralphmhas joined
danielhas left
danielhas left
vanitasvitaehas left
Tobiashas left
jubalhhas left
Alexhas left
Alexhas joined
marc
Ge0rG, jonasw, I plan to provide a field (optional or required, determined by the service) to specify the name of the inviter, what do you think?
jonasw
sorry, I lack context. where would you provide that field?
marc
jonasw, user invitation
marc
Sorry :D
jonasw
and where’d that field be?
marc
Such that we can provide something like "You were invited by X to join..." on the invitation page
marc
Ad-hoc command
marc
Filled out by the inviter
jonasw
hmm
jonasw
the issue with that type of names is that they can easily be spoofed
jonasw
Ge0rG had some thoughts on this type of spoofing in the context of invitations IIRC
marc
Sure, it's not a security feature :)
jonasw
it could be misleading though
marc
Well, you have to send the URL to somebody, if the user trust this URL then there is no problem
marc
(because it trust the mail, SMS, ...)
Valerianhas left
Valerianhas joined
marc
You can think of it as security feature if somebody tries to manipulate/replace the invitation URL ;)
Ge0rG
marc: I think it depends on how you implement it.
Ge0rG
marc: if it is an additional url parameter that contains the plaintext name, it's rather Meh.
marc
Ge0rG, that's up to the service
Ge0rG
marc: if you only attach a token to the URL and the user's JID is somehow obtained from the server via the token, it's a bit better
marc
The XEP defines the field
marc
How the invitation page is implemented is out of scope of this XEP
Ge0rG
marc: I see merit in defining the OOB behavior as well
marc
In my current implementation the name is fetched from a database via the token
Ge0rG
marc: so your XMPP server must also be a web server.
marc
Ge0rG, well, providing a web page is also optional
marc
My first implementation just uses the xmpp URI
marc
and generates a QR code
marc
Works nicely without web site etc.
Ge0rG
marc: so where is the inviter's name displayed, then?
danielhas left
Ge0rG
marc: does it also work nicely without an XMPP client?
marc
Ge0rG, the name is displayed on a web site, if you provide one
marc
Ge0rG, what works nicely?
Ge0rG
marc: the QR code
danielhas left
marc
the QR code is display in the xmpp client
marc
but you could also just send the xmpp uri via SMS or ...
Ge0rG
marc: and then?
Ge0rG
marc: how does your friend open an xmpp: URI without an XMPP client?
marc
Ge0rG, add a link with a download URL for a xmpp client ;)
Ge0rG
marc: why not just use easy-xmpp-invitation? :P
marc
Ge0rG, what's easy-xmpp-invitation?
marc
ah I see
marc
your website template
marc
Ge0rG, that's nice but why should it be required by the XEP?
marc
I prefer a web site for invitation which displays the QR code, provides information about clients etc.
marc
But why should this be required?
danielhas left
jubalhhas left
Ge0rG
marc: how do you generate the url for the web site where the inviter's username is displayed?
marc
Ge0rG, the name is fetched from a database not included in the URL
marc
You could include it in the URL
marc
If you like but I don't care how that's implemented
Ge0rG
marc: you send me a QR code with xmpp:.... How do I know the URL of the web page?
marc
Out of scope IMO
lovetoxhas joined
marc
Ge0rG, either I send you a URL or a bare xmpp URI
Ge0rG
marc: where do you get the URL from?
marc
In best case you're next to me and just scan the QR code from my display of my mobile phone
marc
Ge0rG, the URL is generated by the server and sent back to the inviter
Ge0rG
marc: so the response to the ad-hoc command is an xmpp: URI and a web link?
Ge0rG
or either?
marc
Always a token
marc
And, if provided, a URL
marc
the xmpp URI can be generated by the client
Ge0rG
So the client needs to construct the xmpp: URI from the token?
marc
Yes
Ge0rG
And the server could send back an easy-xmpp-invitation URL or a mod_invite URL or some other black magic?
marc
Ge0rG, it can send back a URL to some web site, yes
Ge0rG
Sorry for the many questions, I'm trying to understand the protocol.
Ge0rG
marc: will that be a generic website or a personalized one?
marc
I thought it is not that complicated and straightforward :D
Ge0rG
marc: life is full of corner cases.
marc
Ge0rG, depends on the server / service
Ge0rG
marc: so if your server sends personalized links, your client can forward the web URL and be good, but if my server returns a generic URL, and my client forwards only that, it's worthless?
marc
they could also provide a URL to some porn web site if they like ;)
Ge0rG
besides of the porn, of course.
marc
Ge0rG, sure, of course the web site should include the xmpp URI, QR code etc
marc
Otherwise this web site wouldn't make sense
Ge0rG
marc: it could be a generic registration form or the ToS
marc
No, that's not good
marc
Well, you could do this, of course
marc
But then the token is useless :D
Ge0rG
marc: that's my point.
Ge0rG
marc: so the XEP needs to specify that the returned URL is a personalized one that also contains the token (or a different token with the same functionality)
danielhas left
marc
Ge0rG, we could also make the token optional and the server can send a URL of some generic invitation web site
Ge0rG
marc: you don't need a XEP for that, do you?
marc
Ge0rG, if you would like to have a standard for clients to request an inviation URL you do
marc
Otherwise, where do you get this URL from? Search on the web?
marc
Guessing?
marc
I'm not saying that this is the best "response" from a service but it is better than nothing :)
Ge0rG
marc: I'm saying that it's worse than nothing, if it is expected to be a specific URL
Ge0rG
the client needs to decide based on what it receives.
Ge0rG
if it receives [URL=https://yax.im/register, token=deadbeaf], it doesn't know whether it must send on both or only the URL
Ge0rG
so the URL needs to provide the same functionality as the token
Ge0rG
besides, it might be useful to return not just a token but an xmpp: URI, because the server might be better suited to construct it than the client
Ge0rG
then the server could return xmpp:free-hosting.com?token=deadbeef to a client logged in as user@legacy.free-hosting.com
marc
Sure, the server can also return the xmpp URI
lskdjfhas left
marc
Ge0rG, but the URL can still be optional, right?
danielhas left
Tobiashas joined
Ge0rG
marc: yes. The URL should be optional, I just would like to prevent "smart" server operators entering a generic URL there
marc
Ge0rG, yes, but we can not avoid it anyway ;)
Ge0rG
marc: but we can make it illegal via the XEP
marc
:D
marc
Ge0rG, btw, I think your easy-xmpp web site should auto-detect the operating system / browser and suggest a client :)
Ge0rG
marc: I think you are right, which is why I wrote that into the TODO
marc
ah nice ;)
marc
didn't read it
Ge0rG
too bad :P
sonnyhas joined
Kev
I've just updated the Board mailing list. It's now current members + Council Chair + ED + Secretary.
Guus
... ED ...
Guus
Ah, Exec. Director?
Guus
Thanks Kev
sonnyhas joined
jonasw
Guus, so I’m not the only one having to re-think each time they see the abbreviation "ED"
Guus
"each time" <-- you've seen it before then? :)
Valerianhas left
efrithas left
moparisthebest
ha well I found a fool proof spam prevention system that would work for xmpp too, it's terrible, but it'd work
moparisthebest
our support got a ticket at work from a user who wasn't getting our emails because they have a spam prevention system that replies to the email with a link the sender has to click to allow the email through, or whitelist the address, or something
efrithas joined
jonasw
Guus, members list
moparisthebest
so for unsolicited chat or subscription requests, server could message the user an http link that needs clicked before allowing it through... :/
SamWhited
moparisthebest: that's basically the same as the proof-of-work model we talked about, except less automated
moparisthebest
where is this discussion? must have missed it
SamWhited
that could be the fallback if a PoW message gets sent but the potential spammers client doesn't support it
SamWhited
moparisthebest: I'm not sure where or when; it was a while ago. TL;DR if you get a message from someone you don't know, send their client a relatively expensive problem they have to compute and respond to before you'll show the message to the user
moparisthebest
I actually think this would work better in xmpp vs email, this user wanted the person at our company monitoring the system.noreply@ourdomain.com email to click the link...
moparisthebest
yea the combination might be good
SamWhited
I've got a TODO to write up a spec for that actually; I should do that this weekend.
jubalhhas joined
moparisthebest
you should :)
jonasw
*SHOULD :-)
SamWhitedmoves it up to the top of his list from the bottom that's off the screen and therefore was forgotten about
moparisthebest
with a message a link fallback, servers could sanely turn it on before client support was widespread
SamWhited
actually, my TODO was for using it with IBR2, but the same challenge could be reused for both probably
moparisthebest
and you don't need to specify what happens with the link, if that happens you expect human intervention, servers might whitelist automatically, have a captcha, or whatever
Holger
What part of this needs a spec?
moparisthebest
the automated proof of work part
moparisthebest
the sending a link would not
SamWhited
Holger: the link part doesn't, but if we want clients to automatically respond to PoW challenges from the server that part needs a spec
vanitasvitaehas left
Holger
Ah the client responds without automatically? What happens to people with old clients?
Holger
s/without//
SamWhited
Holger: that's why you include a link to a captcha or something in the body. Old clients show that, you can click it and complete a human challenge instead
MattJ
They click the link
Holger
Ah.
SamWhited
Just like joining MUCs on Jabber.ru, which I think does this with clients that don't support their captcha forms (you get a link to the same form on the web)
MattJ
and here you go: https://xmpp.org/extensions/xep-0158.html#challenge-hashcash
MattJ
Later in the document: "A challenger MAY provide a text question in the <body/> element of a challenge stanza for clients that do not support CAPTCHA forms."
SamWhited
oh hey, would you look at that, been done
MattJ
The problem I see with proof-of-work is that spammers have access to lots of CPU cycles (that typically aren't really theirs), and real users don't
SamWhited
It's true, it doesn't stop botnets, it just stops every person with a laptop from being able to register hundreds of accounts and then send spam with them all day
Ge0rG
Especially mobile users don't.
MattJ
So you'll flatten a user's battery a bit, and you'll cost a spammer some miniscule amount of money on a botnet or captcha-solving platform
SamWhited
*slows down (not stops)
SamWhited
It still forces them to use a botnet or captcha-solving platform, which is more work. And most users will rarely have to do this, only spammers would need to do it regularly
MattJ
Pretty sure the user's battery will drain before the spammers deem it not cost-effective
SamWhited
I suspect anyways; I bet most "normal" people only communicate with people on their roster, and if we drain a users battery because their phone is in a botnet I'm pretty okay with that
MattJ
SamWhited, we had CAPTCHA on register.jabber.org, it didn't stop them, just slowed them down - doesn't solve much at the end of the day
SamWhited
Slowing them down is really the only point; it's the best we can do
Ge0rG
Let's slow them down by retracting subscriptions from known spammers.
uchas joined
moparisthebest
it wouldn't drain a mobile users battery at all, except a tiny amount once when they send a message to a new user they haven't sent one to before, right?
Ge0rG
moparisthebest: there is a little problem there: PoW is much more energy-expensive on a mobile CPU than on a GPU cluster.
moparisthebest
and if botnets do indeed bother and implement this, you can just fall back to manual human-intervention-required link challenge
moparisthebest
Ge0rG, but it happens once per new message to new user max?
moparisthebest
so like, maybe 30 times over the course of an entire xmpp account's life?
moparisthebest
maybe other people are far more popular than I am idk
jonasw
Ge0rG, depends on the PoW
jonasw
memory-hard proofs come into mind
ralphmhas left
jonasw
scrypt or something
moparisthebest
and yea you could do something that is harder on gpus, yep scrypt
Ge0rG
jonasw: right, because OOM isn't a thing on mobile :P
danielhas left
jonasw
Ge0rG, it is, but using lareg amount of memory is expensive on GPUs
jonasw
(and on FPGAs)
Ge0rG
jonasw: we aren't talking about bitcoin mining parallelization here
Ge0rG
or whatever-scrypt-coin
jonasw
where large is something like ~100 MiB already, depending on the type of operations
Ge0rG
You can't expect an old Android phone to provide 100MB (plus JVM overhead) to calculate a PoW
jonasw
I would expect that to work actually
jonasw
I bet firefox needs more
jonasw
firefox gets swapped out then, I’m fine with that
Ge0rG
jonasw: "Bug report: my browser gets killed when I add a friend"
danielhas left
moparisthebest
an old android phone can't run conversations either so who cares
Ge0rG
I think this is insane.
jonasw
your browser gets killed always anyways on those machines.
mathieui
this is crazy, yes
SamWhited
We'd have to think about that, but the point is that we can think of something that's not too bad for mobile users but still does the job
moparisthebest
then you disable it on your client and solve http links by hand Ge0rG ?
jonasw
I can’t really use my broswer + any other app on my Galaxy S3.
Ge0rG
moparisthebest: no, I just switch to WhatsBook.
jonasw
with WhatsBook, you need mutual subscription first anyways?
moparisthebest
are you thinking this is something that would happen often?
jonasw
I thought we had that as a possible sensible solution, too
Holger
jonasw: You don't.
efrithas left
jonasw
Holger, hm, when we discussed this a few weeks ago, I think the consensus was that you need it, but w/e
danielhas left
Ge0rG
moparisthebest: unless we make PoW prohibitively expensive for normal users, spammers won't be slowed / stopped by it.
Holger
jonasw: I remember someone claiming that and me not finding a single commercial messenger that actually does that.
Holger
jonasw: And I think it's unusable as a general solution.
Holger
Though admittedly this PoW idea sounds even worse.
SamWhited
If memory consumption and GPUs turn out to really be an issue it could also use an algorithm that relies on cache locality
vanitasvitaehas left
vanitasvitaehas joined
moparisthebest
Ge0rG, bcrypt and scrypt would like to disagree with you
moparisthebest
they were explicitly designed to be not so bad for normal users, and prohibitive for bad guys
jonasw
funny question, couldn’t the PoW be solved by a users server?
MattJ
Yes, this would be interesting :)
mathieui
new DoS way
jonasw
you’d quota that of course etc., but for the occasional adding of a contact…?
MattJ
mathieui, it would encourage server admins to lock down their servers better :)
jonasw
like, 3 PoW / day / user, 60 PoW / hour in total or so
SamWhited
jonasw: that's an interesting idea; sounds like a possible DoS, but servers could be smart about it and say "this user is generating too many PoWs, what are they doing?"
jonasw
SamWhited, exactly
jonasw
that’d solve the mobile issue
moparisthebest
that only helps good public servers being abused to send spam
moparisthebest, and it will slow down bad servers just as well
jonasw
it also gives servers an interesting metric on users sending a lot of subscriptions/new messages
SamWhited
That only works if the server supports the PoW thing though; otherwise you still have to issue it to the client (or issue a captcha)
jonasw
and if the server is overloaded with PoW, it could simply forward it to the client.
Ge0rG
it would be better to have users do PoW to pay for their server.
jonasw
SamWhited, ofc.
SamWhited
So if a spammer has spun up their own server, you still have to support the original model
moparisthebest
Ge0rG, so xmpp spammers are going to buy super expensive miners, and try to use them for xmpp PoW ? seems unlikely
jonasw
"the original model", SamWhited?
SamWhited
jonasw: sending a captcha/PoW to the client, I mean
jonasw
you’d send it to the user, the server can decide to intercept the PoW request and handle it by itself or forward it to the client
Ge0rG
moparisthebest: spammers will either distribute PoW to their botnet, where you will be paying for the PoW, or rent some gigahashes.
jonasw
I’m not sure what you mean
moparisthebest
I don't think they really have botnets now, just register on open servers and spam until banned
Ge0rG
moparisthebest: please stop arguing. The PoW battle has been lost.
jonasw
Ge0rG, not sure
moparisthebest
there is no golden solves everything problem, you can only annoy them a bit, slow them down for a bit
danielhas left
jonasw
if you have a botnet, you’re probably better off putting that computing power into $cryptocoin
jonasw
instead of trying to spam
moparisthebest
it's an arms race :P
Ge0rG
moparisthebest: we can make life really miserable for mobile users and slightly annoy spammers.
moparisthebest
you still haven't said how it would hurt mobile users at all?
Ge0rG
jonasw: you need orders of magnitude more power for viable commercial mining
SamWhited
Right, if you have a botnet to spam with we can't stop you right now, but we can slow down your botnet so that you send less spam and possibly make it prohibitively expensive for the people who spin up a single server in their basemenet to send out spam
la|r|mahas left
moparisthebest
are you mistakenly thinking this would happen on every message or something?
SamWhited
And we can do it without affecting users at all, I suspect
SamWhited
*well behaved users
moparisthebest
how often do you add or message new users? how often does a normal user do that? I'm thinking very rarely
Ge0rG
moparisthebest: just to repeat my argument: if you make it sufficiently hard for spammers, it will be prohibitive for normal users.
moparisthebest
and again that's just wrong Ge0rG
SamWhited
Ge0rG: we're disagreeing with that argument. Why do you think it would be prohibitvely expensive for normal users?
jonasw
I doubt we’ll reach an argument here
jonasw
we need data
Ge0rG
moparisthebest: okay, suggest a number of scrypt operations a user needs to perform for an initial contact.
jonasw
maybe some google scholar-ing on how botnets operate nowadays?
SamWhited
Spammers send lots of messages to people that aren't on their contact lists, normal users don't. That asymetry is what we're targeting.
Ge0rG
SamWhited: we can target that with simple statistics, without annoying anyone.
moparisthebest
Ge0rG, whatever takes about 4ish seconds on say a samsung galaxy s4
SamWhited
Ge0rG: where and how do you get those statistics?
MattJ
SamWhited, I agree that I think this is the best place to solve this problem
Holger
"Q: My app crashes (or crashes other apps) when adding a contact. A: Don't worry, I know that you don't often add contacts."
moparisthebest
yea you wouldn't make it crash
jonasw
also, shouldn’t this discussion move to spam@?
SamWhited
Holger: that's why we have to do research and find something that won't OOM everyone but is still relatively tricky
SamWhited
Too many rooms.
jonasw
Holger, I still claim that users on devices with this low amount of memory are used to that
Holger
We can add that to our response then.
Ge0rG
Okay, just to get some numbers. "I managed to pull around 5.6KHash/sec on my Nexus 7 with all four threads." from https://rumorscity.com/2014/01/07/how-to-mine-litecoin-with-android/
MattJ
Proof of work aside, it's very easy to identify accounts sending to a lot of non-contacts. Spammers will switch to subscription requests, but it's equally easy to spot (new?) accounts sending lots of subscription requests, and this is uncommon (not impossible for a legitimate user, just uncommon)
Holger
Indeed.
Ge0rG
So we are at ~20 KHashes for a first-contact
moparisthebest
MattJ, you mean easy if you run a server with lots of users I guess?
MattJ
Easy on any server
jonasw
MattJ, you assume that the server isn’t malicious
moparisthebest
or, easy for a public server to spot new users of it's server
jonasw
I find that assumption incorrect
SamWhited
MattJ: I agree, server operators should be doing that
MattJ
jonasw, that is an assumption in this case, malicious servers are a different thing
waqashas joined
moparisthebest
yea explain how my private server with 4 xmpp accounts can see any of this data, easily or not
MattJ
But that's not a problem we have today
stefandxmhas left
jonasw
MattJ, it is
jonasw
I think
Holger
And malicious servers won't respond to PoW requests?
moparisthebest
Holger, if they don't do the pow or click the link and do whatever is there, no requests get through to the client?
jonasw
fighting spam at the source is of course most sensible, but hard in a distributed system
moparisthebest
so, it's not a problem
SamWhited
Holger: they will respond to PoW requests, and that's the point. It will slow them down because they'll be responding to *lots* of them.
moparisthebest
and if they do pow and too much spam still gets through, turn off pow and go back to manual links
mathieui
also: if a server implements the PoW thing, it could "wall" the subscriptions unless it’s mutual
moparisthebest
make them play an html5 punch the monkey game and beat a high score to whitelist their jid :P
mathieui
(e.g. "user A and user B want to communicate, B’s server does not implement PoW, B adds A, A sees nothing; A adds B because they know about it: the subscription is established")
moparisthebest
or A's server sends an https link to user B and when clicked lets it go through
Ge0rG
you can rent 500MH/s for three hours for ~3USD. That accounts for 100 Millions spam messages.
Ge0rG
if you price a single spam message at 20KH scrypt.
MattJ
Case closed :)
Holger
SamWhited: I got the idea, I just don't see the gain in throttling them to, dunno, some hundreds of thousands of messages per day and CPU core or whatever. Do they really send millions per day right now?
Holger
What Ge0rG said.
jonasw
Ge0rG, "ouch"
Ge0rG
moparisthebest: so will you stop now?
zinid
right, what will happen is that spam will become a bit more expensive for the customers 🙂
Ge0rG
source for MH price: https://www.miningrigrentals.com/rigs/scrypt
SamWhited
Holger: I don't know. More research would definitely be required, I just suspect we could slow them down a bit
moparisthebest
and are you sure you can use that for arbitrary scrypt challenges Ge0rG ?
Ge0rG
moparisthebest: again, spammers will just use whatever botnet they can get away with.
jonasw
moparisthebest, if you can’t *now*, as soon as you can use this to make some financial gain (e.g. spam), they will adapt so that it can be used for that
danielhas left
Ge0rG
moparisthebest: even when running on desktop Windows malware, you are several orders of magnitude faster than on a mobile device.
moparisthebest
possibly, but then you are excluding spammers who don't have botnets
jonasw
moparisthebest, even a spammer with their own server can easily outperform that challenge
moparisthebest
are there challenges that are faster on ARM than amd64 ?
Ge0rG
moparisthebest: let's just exclude all spammers by definition. Congratulations, we have won the spam war.
jonasw
20 kH is not much
Ge0rG
I can go back to work now.
SamWhited
Botnets would (hopefully) still be slowed down, non-botnets it might be too expensive. It's not a panacea, but it still seems like it could be a benefit if the idea works.
moparisthebest
right there is no panacea
Ge0rG
Right. So please pretty please let's focus on the ideas that make it actually worse for spammers than for users.
moparisthebest
like this one
moparisthebest
or, what is your other idea?
Ge0rG
moparisthebest: read up my posts on the operators@ and standards@ MLs.
zinid
digitalocean started to provide high performance servers for computational tasks, for the record, I think 500MH/s would be nothing for those
jjrhhas left
Ge0rG
moparisthebest: I've exceeded my time budget for convincing you on pointless scrypt performance calculations, sorry.
Holger
moparisthebest: Mine is server-side filtering based on both message contents and meta data. Works relatively well for email.
moparisthebest
Ge0rG, I've been following a bit, but iirc most are geared towards running large-ish public servers with lots of data to analyze
moparisthebest
and nothing for small private servers
Holger
SpamAssassin works just fine for small private email servers.
moparisthebest
it does for email, I was under the impression most xmpp spam was too small for it to be effective
Ge0rG
there are many theoretical solutions that just won't work out. Like a distributed p2p reputation system.
SamWhited
Holger: I've been wondering about that too; do you have that tied to an XMPP server? How well does it work for the shorter XMPP messages?
Ge0rG
Some months ago, 99% of spam was multi-line Russian with multiple links.
Ge0rG
Easy to kill, just filter multi-line messages from strangers.
Ge0rG
Then, they switched to single-line messages with two pastebin links. Still pretty easy.
moparisthebest
sure and it's easy to do stuff like that, doesn't solve everything though, just like PoW doesn't
SamWhited
Those are good things to do as well
Ge0rG
moparisthebest: yes, but PoW WILL ANNOY USERS
Holger
SamWhited: I think it'll work quite well. Problem is it requires work.
zinid
Holger, except that Bayes sucks for short messages
moparisthebest
besides I look forward to a future when all messages are encrypted base64 blobs :P
Holger
moparisthebest: So all solutions are equally adequate because none are perfect?
Ge0rG
Now they send a subscription request + multiline / pastebin
SamWhited
Yah, I suspect zinid is right and it won't work so well for XMPP messages, but I'd love to be proven wrong
Holger
zinid: Yes I would definitely not rely just on Bayes.
moparisthebest
no, I'm saying you should implement non-perfect solutions because those are the only solutions we have Holger
waqashas left
Ge0rG
moparisthebest: you are saying we should implement bad solutions because there are no perfect ones.
lumihas left
Holger
moparisthebest: And I'm saying some non-perfect solutions are way better than others. So "nothing is perfect" is not a very convincing reasoning.
Ge0rG
I haven't seen a spam message in weeks. Just the subscription requests
zinid
Holger, there is very little research for short messages, I recall a couple of papers only, you can search google scholar with "sms spam" query
Ge0rG
BTW, my next proposals would be:
- revert a pending subscription if a sender is classified as spam
- implement a cache of URLs sent by strangers, with counters, and block messages with a count > 3
Ge0rG
MattJ: ^
moparisthebest
and, again, both of those solutions only work for large public servers Ge0rG
moparisthebest
I'm literally the only user of my xmpp server to get any spam
Ge0rG
moparisthebest: you know what? spammers are sending messages to non-existent accounts. You could easily set up a honeypot
SamWhited
Ge0rG: I agree, both of those things are a good start and should probably be done (although they also probably need a way of dealing with false positives, but that's a different problem)
Ge0rG
moparisthebest: please log on your server messages sent to non-existent accounts
moparisthebest
I'm not saying they aren't good solutions for large public servers, or that you shouldn't implement them, I just also want something that makes running your own server practical
MattJ
Ge0rG, #2 won't work, the URLs can vary too easily
zinid
moparisthebest, set spam traps 🙂
Ge0rG
MattJ: it's expensive to create a large number of shortened links
MattJ
Trivial to append #uuid
Syndacehas left
Ge0rG
MattJ: besides, we could HEAD them :P
MattJ
Trivial to append ?uuid
zinid
moparisthebest, i.e. non-unsed accounts, but you publish them everywhere 😉
moparisthebest
yea but is that what we suggest to people wanting to run their own servers
moparisthebest
start your server, setup SRV records, post non-existant accounts at random places
Holger
zinid: Yes, I don't have papers to prove my point. But many of the criteria used for email classification are unrelated to the length of the message body.
SamWhited
Holger, zinid: seems worth trying either way
Ge0rG
moparisthebest: you could just block messages from strangers with an URL.
zinid
Holger, yes, maybe you can do that, no doubt, although the result is unpredictable
Ge0rG
yax.im will send an error response "Blocked due to abuse", so a real sender actually will see a message about being blocked
uchas joined
moparisthebest
Ge0rG, what about subscription spam
zinid
what about captcha btw?
Ge0rG
moparisthebest: see above
moparisthebest
also none of this works for encryption
moparisthebest
spammers will soon start sending omemo messages :P
Holger
Yes that's one of my complaints with E2EE.
Ge0rG
moparisthebest: I'm sure they will
Ge0rG
I'm eagerly awaiting the day when I can block E2EE on my server :P
zinid
Holger, ah, yes, E2EE, hehe
moparisthebest
so back around to the original plan of replying with a https link with arbitrary things to solve on it
Ge0rG
moparisthebest: yes, let your friends play xbill or whack-a-mole.
Ge0rG
or solve a reCAPTCHA.
moparisthebest
yes, once, doesn't seem like too much to ask?
mathieui
maybe we could embed flash applications as base64 inside subscription replies
Ge0rG
moparisthebest: you can do all that on your private server
Ge0rG
moparisthebest: but it doesn't work well for public services
moparisthebest
obviously doesn't happen if you send them an easy xmpp invite url or whatever
moparisthebest
mathieui, it's 2017 you mean html5 applications
moparisthebest
seems like it'd work equally well for public servers?
moparisthebest
public servers just additionally have much more to go on, so maybe it wouldn't always be necessary
Ge0rG
Holger: please tell MattJ about the normal-message-to-full-JID thing.
Holger
The "some clients do that so servers can't throw the message away" thing?
jjrhhas left
Holger
Some clients do that so servers can't throw the message away.
Holger
Well they can but some users will be yelling at you.
jjrhhas left
Ge0rG
Holger: yeah, that thing. Was it just a misuse of Gajim, or was that a really f***ing big problem?
jcbrandhas left
Holger
The main problem was some mobile and/or JavaScript libraries doing that by default or something.
Steve Killehas left
Holger
Not sure about public clients. Maybe it was just Gajim.
Ge0rG
Holger: I'd like to know which clients exactly... :>
Holger
I don't remember more than that sorry.
MattJ
Meh, I'm still going to go for it
jjrhhas left
Holger
I think the real issue is that type=normal is overloaded to do unrelated things.
Ge0rG
Right.
Ge0rG
Like MAM responses from a remote MUC
Martinhas left
Ge0rG
And Carbons and ACKs and many other XEPs
Ge0rG
And type influences both the meaning and the routing.
zinidhas left
Holger
Yes on the one hand there's these type=normal messages addressed to individual *devices* and on the other there's type=normal messages addressed to humans, i.e. to accounts.
Steve Killehas joined
danielhas left
Holger
If now everyone agrees that the latter should be addressed to the bare JID, the problem is partly solved.
Holger
But the RFC disagrees, and so does some of the existing client code.
Ge0rG
Holger: RFC6121 says you are not required to persist normal full-JID messages.
Ge0rG
I'm not sure resource locking is part of RFC
Holger
It's part of 6121.
Holger
https://tools.ietf.org/html/rfc6121#section-5.1
danielhas left
Ge0rG
Bummer.
goffihas left
peterhas joined
danielhas left
SamWhitedhas left
waqashas joined
lskdjfhas joined
lskdjfhas left
ralphmhas left
lskdjfhas left
lskdjfhas left
lskdjfhas left
lskdjfhas left
lskdjfhas joined
lskdjfhas left
lskdjfhas left
lskdjfhas left
lskdjfhas joined
lskdjfhas left
lskdjfhas joined
jonaswhas left
danielhas left
danielhas joined
lskdjfhas left
lskdjfhas joined
jonaswhas left
lskdjfhas left
lskdjfhas joined
goffihas joined
lskdjfhas joined
ralphmhas left
Valerianhas joined
lskdjfhas left
blablahas joined
lskdjfhas joined
lskdjfhas joined
ralphmhas left
lskdjfhas left
lskdjfhas left
zinid
Holger: I see only SHOULDs there, am I missing something?
zinid
but of course that doesn't mean existing code won't break ;)