XSF Discussion - 2017-12-05

Ge0rG, Did you take a look at the XEP?

marc: you mean at the examples? 😜 yes, I did

Ge0rG, yes the examples ;) Any objections so far? :P

marc: I still think that the ad-hoc command has two parameters too many. Also not sure about just adding another element to the IBR request

Ge0rG, these two elements are still optional ;)

Ge0rG, do you have an other idea for IBR?

marc: I'm not sure means I don't know if this can be made legal by the XEP. The alternative would be to use full fledged Data Forms

marc: and it needs some way to integrate with PARS, so that users who already have an account will be accounted for as well

Ge0rG, what do you mean? can you give me an exmaple?

Yay. Spim from @yax.im.

Hi, https://news.ycombinator.com/item?id=15850597

some votes may help ;)

intosi, also got it and put the domain on my blocklist. but maybe Ge0rG can get his act together and prevent these things

Of course! yes goffi :D

upvoted :p

Ge0rG: How's your acting?

jonasw, no ProtoXEP annoucement for ISR 0.0.5? I also don't see a council trello card for it

goffi, upvoted, also read the blog post, sounds great!

goffi, why does the vidoe have white bars top and bottom?

Tobias: because it's old blog renderer and the CSS is bad

Flow, no

I don’t do that normally, I thought your mail was announcement enough

maybe CC council@ next time

but can't use the new one yet (which is responsive), there is not yet atom feed or pagination.

you shouldn't enforce width and height for the video size

(ProtoXEP updates are out-of-process, so there’s no tooling for this)

jonasw, ok, i've a feeling that people will refuse to deal with the submission if it's not officially announced. Would you be so kind and add a card to council's trello?

I doubt that people will refuse that

I am in a meeting right now

hope so

remind me in a few hours, then I’ll do that

Tobias: seriously? You've blocked yax.im?

Ge0rG, if domains send spam my way, i block them :)

Tobias, maybe first contact the admin?

that’s kinda extreme

it's even the less harsh approach, compared to others who whitelist domains they do s2s with :)

especially if the admin is part of the XMPP core-ish community

jonasw, well...i can unblock them

even did so after a week

Tobias: that makes me very sad, still

it also makes me very sad that i received spam from your domain

Tobias: so what would be your proposed solution? Shut down all public servers?

no...apply some limitations to new accounts on public servers

Oh, I'm with Tobias: I've blocked various domains that delivered nothing to mine, except spam.

and monitor behaviour of new accounts on public servers

i have more than 50 domains on my block list

oh, i ~10 :)

Tobias: I'm doing the following: - limit IBR per IP - monitor large numbers of outgoing messages and block accounts - limit the number of stanzas a client can send - watch my logs - react to abuse reports as fast as I can

Tobias: and you didn't even report it to me.

Ge0rG, will do the next time, I promise

Tobias: you still can report the message content that you received :P

Tobias: I know how annoying spam is, and it really really really makes me sad to learn about it in such a public-shaming way.

Tobias Guus is it possible to get thoses blacklists ?

edhelas, yes

Tobias: so what's the spam message you received, anyway? I'd like to improve my outbound filters.

Ge0rG, just going through my logs to find it

Tobias: and did you receive it from digital.advert307@yax.im or a different JID?

edhelas, I can look them up for you. Note that because _my_ users aren't appear to be talking to valid accounts on those domains, yours might. I'm not sure if a permanent blacklisting is approriate.

I also block on first offence. I am evil.

Tobias: so I've just deleted eight spammer accounts that connected through the same IP address. If you had told me the JID or the content of the message you received (did you receive one at all?), I possibly could have deleted more.

Ge0rG, does your server ping every s2s connection every minute? even if you don't send other messages over that connection for a longer time?

Tobias: it does ping other servers, but I'm not sure if it is actually set to one minute. Why?

wc -l blocklist/zash.dat 256 blocklist/zash.dat

Eeeeeeeevil

just looking at my log

Zash: how often will prosody 0.10 send whitespace via s2s? ✏

Ge0rG: We never got around to adjusting the timeout, so after 6h of silence.

mod_pinger should only ping c2s, from reading the source code.

dwd, I added ProtoXEP ISR to the proposed agendums [sic]

(cc @ Flow)

Zash: why is it sending two whitespaces per minute on an s2sin then? And also one ping per minute on s2sout?

Ge0rG: Are you using 3rd party plugins that I have no idea about what they are doing?

Zash: if by "3rd party" you mean "from prosody-modules", then yes.

Yes, I do

Zash: where is the code that sends the once-in-6hr whitespace?

Ge0rG: In mod_s2s, when the network backend invokes the "read timeout" handler.

Have you perhaps changed the read timeout setting?

Zash: I have. `network_settings.read_timeout = 840` - which is NOT 60 seconds.

Zash: also why should it send twice on an s2sin link?

Ge0rG: Duno. Bug?

just to pick a random s2s: Dec 05 10:18:52 s2sin98ca6d0 debug sending: Dec 05 10:18:52 s2sin98ca6d0 debug sending: Dec 05 10:18:52 s2sin98ca6d0 debug Received[s2sin]: <iq id='keepalive' type='result' to='yax.im' from='tengu.chat'> Dec 05 10:19:52 s2sin98ca6d0 debug sending: Dec 05 10:19:52 s2sin98ca6d0 debug sending: Dec 05 10:19:52 s2sin98ca6d0 debug Received[s2sin]: <iq id='keepalive' type='result' to='yax.im' from='tengu.chat'>

Ge0rG: Weird. Libevent?

Tobias: thanks for pointing out the bug. Were you able to find anything in your logs regarding that spammer you encountered?

not yet...will tell you in about half an hour

zinid, if you’re doing strict schema validation, how do you handle the lax order requirements of XMPP?

which cannot really be reflected in schemas?

(well, somebody did the work to reflect that in XEP-0030, but ...)

Sigh. Blocking messages from strangers is really annoying. I get a subscription request from a JID I don't recognize and I can't even ask them why they contact me without exposing my presence.

Oh, `admin@xmpp.wiki` is not actually an admin.

Does .wiki domain exist?

Looks like one of chatmes domains?

Didn't know that

Zash: yes, it is.

mod_block_registrations should also be required for IBR servers…

with admin, operator, and root banned

mathieui: right, that too.

block_registrations_users = { "abuse", "admin", "administrator", "hostmaster", "info", "news", "noc", "owner", "postmaster", "register", "registration", "root", "security", "service", "signup", "support", "sysadmin", "sysop", "system", "test", "trouble", "webmaster", "www", "xmpp", }

based on http://tools.ietf.org/html/rfc2142 and http://blog.postbit.com/reserved-username-list.html

added `operator` now.

Zash: ^

Ew, spaces for indentation!

Zash: what? I'm using tabs.

The module

wtf who uses tabs for indentation

This

No, TABS! Holy war!

oh no

maybe we should talk about tea instead

No, Coffee! Holy war!

nobody expects the xmpp inquisition

mathieui: nobody expects the whitespace inquisition?

Ge0rG, https://q.zash.se/a6db0f2a6dcf.txt

Tobias: that's been almost two weeks ago!?

yes?

Tobias: you didn't know that I'm the admin of yax.im?

i did

like I said, next time I'll report it to you before blocking your domain

Tobias: sorry, but I'm speechless.

is the account already deleted?

BTW, there is a dozen of accounts that used the same IP address.

Tobias: no. I'm not even sure why the message went out at all. It should be blocked for multiple reasons.

like high percentage of YELLING?

Tobias: I'm not measuring that. But multi-line messages to non-subscribers should be blocked.

jonasw: What order requirement can't be reflected?

Ge0rG: does `prosodyctl mod_firewall test` work?

jonasw: Either way ejabberd is using it's own schema format, not XSD.

Zash: I remember now. my `bodycheck` rule had to be disabled for outgoing messages because IN_ROSTER doesn't work in `::preroute`

Say what

jonasw, we don't use XSD, they are shit

Zash: what does IN_ROSTER return when applied in the ::preroute chain?

Ge0rG: Oh right, because it refers to the receivers roster, which wouldn't be available for a remote user

Zash: so how do I find out if the local sender is subscribed to the remote receiver?

jonasw, it has erlangish format, here it is: https://github.com/processone/xmpp/blob/master/specs/xmpp_codec.spec

Ge0rG: And the sender can just add whatever they want to their roster, so you'd need the thing, yes, subscription check

SUBSCRIBED Tests whether the recipient is subscribed to the sender, ie will receive presence updates from them. Note that this does work, regardless of direction and which chain is used, since both the sender and the recipient will have mirrored roster entries.

Ge0rG: SUBSCRIBED?

Right

is that what I need? Will it work as expected?

Ge0rG: Unless it has the same bug...

Zash: actually I need the opposite - whether the sender is subscribed to the receiver.

Ge0rG: Why?

Whoever invented asymmetric presence subscription.

Zash: because a spammer could pre-approve all receivers?

Ge0rG: But that's not implemented! :D

Zash: ah, it looks like `rostermanager.is_contact_subscribed` will actually check both directions.

Zash: thanks very much.

So now I can apply a `JUMP_CHAIN=user/bodycheck` to outgoing messages as well. But how can I add another action to inform the admin about local users spamming?

JUMP_CHAIN will BOUNCE when spam was found.

Tobias: deleted the twelve spammer accounts. Improved firewall rules. Sent abuse report to ISP. Thanks for reporting.

ta

Holger, in XMPP, the order of elements of different names (e.g. <{disco}feature/> and <{disco}identity/>) is irrelevant. this can be, but is very hard to, represent with XSD

but sure, if you use your own validation, that doesn’t affect you, zinid

this is not strictly speaking the validation, it's more like ASN.1 codec, which transforms XML into internal language structure

but validation is performed during decoding, yes

you have a weird obsession for ASN.1

which is fine, I guess

zinid: You do Erlang, too? xD

(just kidding)

tux, well, erlang knowledge is required for ejabberd development, most of the time 🙂

jonasw: <xs:complexType><xs:all><xs:element name="feature"/><xs:element name="identity"/></xs:all></xs:complexType> won't do the trick?

jonasw, ha, you didn't see my obsession regarding parsers yet 😀

Holger, afaik not

I.

jonasw, for example, from this ABNF file https://github.com/processone/xmpp/blob/master/c_src/uri.abnf the parser code is generated: https://github.com/processone/xmpp/blob/master/c_src/xmpp_uri.c

I think that enforces order between feature and identity

jonasw: No. That would be xs:sequence instead of xs:all.

Holger, tbh, I have not much of an idea about XSD, I tried to read the spec and I find it massively confusing

there have been some comments w.r.t. on standards@

Ah, the new rules absolutely pay off. Found another 18 spammer accounts.

zinid: yeah, that's true. 8)

jonasw: I'm not much into it either, dunno whether you can express all XMPP syntax weirdness with XSD.

inetnum: 176.126.252.8 - 176.126.252.15 netname: FVDE descr: Tor Exit Node Hosting

Whoops. Four of five IPs that injected spam today are exit nodes.

wow, suddenly

just ban all exit nodes 😀

Require all Tor users fill in a Google Captcha!

Ge0rG: good busy!

Okay, so I've deleted another 50 accounts

@iteam - our wiki claims "The XMPP Standards Foundation maintains a dedicated email list (muc@xmpp.org) about MUC" but there is no such list. Does anybody want one?

I’m pretty sure I was on that list.

it was deleted though

Subject: [MUC] deleting this list Date: 14.02.13 05:09 From: Peter Saint-Andre <stpeter@stpeter.im> To: muc@xmpp.org I just deleted the specialized bosh@xmpp.org list. I think it would be appropriate to do the same with the muc@xmpp.org list. Instead of having a specialized conversation here, we'll simply use the main standards@xmpp.org list. Any objections? Peter

the only reply to that was from someone who claimed to not have done anything with jabber for 3 years

How appropriate.

Hah

TBH, I don’t see a use-case for separate lists

Ge0rG, edit that out of the wiki maybe?

jonasw: Very active topics that drown out others?

jonasw: done

Zash, I find standards@ still comfortable to read

I think it's more about interests not overlapping than about traffic.

If only we had threaded email-like message support in XMPP.

It works like a sort of filter, but enforced by the sender instead of the receiver

If you expect the same people on both lists, it's not gaining much, but if you think lots of people care about MUC, but not other stuff on standards@ there's a point.

I don't think there is in this case.

Ge0rG, why aren’t you in jdev@?

jonasw: I'm not?

now you’re

jonasw: I had a power outage.

zinid, re your standards@ reply: > Well, yes, it will be timed out. I also don't think this is a > violation, because in this case we cannot block IQ from flooders for > example, as this is also a violation.

I would suggest to send some IQ type="error" back in these cases

and making IQ requests timeout is super-annoying

You can't type=error to a result or an error.

Kev, yes

this was in response to what he said about flooders

(which is why I didn’t reply on the standards@ thread)

jonasw, why would I send iq=errors to flood IQs? wtf?

what if this is a dos attack, I should do that too?

zinid, depends on how sure you are that it’s a genuine flood

ok, so if I want that much, then I can 🙂

it might also be a client which is confused or simply has a lot of things to do. sending proper type="wait" errors back seems more reasonable to me, until it becomes a burden. ✏

whatever, as daniel said any discussions are pointless

you already built up your mind, why bother

I'm also not sure why on earth I should route (or store) malformed packets

because you can’t be sure that they’re malformed

jonasw, I can if I have schema

okay, but then you have to consider how schemas are used in XMPP and acknowledge that elements and attributes may be added at any time, and must be ignored

well, that's the rule I don't like

I know that

but that’s how XMPP works and has always worked

(*always: to my knowledge)

and where is it now?

I don’t think that this specific rule is the cause of the low popularity of XMPP

it can be pretty much though

how?

what? lack of formal validation of packets/

?

for example, some people would rather use something more robust, like asn.1

you can formally validate what you know about, there’s no problem with that

no, if you’re going down the "no XML" route, what people nowadays want to use is JSON

but I know there is no <retry/> element in the schema

in which schema?

the one you made up, or the non-normative which may or may not be in the XEP?

ah, indeed, we don't even have schemas

exactly

The 'critical' property of things in ASN.1 sure seems nice

because the rule I mentioned earlier (unknown things need to be ignored, and also the order of things does not matter) is hard to codify in XMLSchema

Zash, google dropped "required" in protocol buffers 3.0

(critical is "you have to understand it", right?)

jonasw, yeah, and you resorted for postulate this ad-hoc

zinid, I don’t understand your statement, sorry, could you rephrase?

jonasw: Yes. If you don't understand something marked 'critical' then that's a fatal error and you should abort everything.

Much nicer than just blanked ignoring of everything not understood.

Zash, idneed

we could have that, if we wanted to

In XML? Hrrrm

but even then I’d find it questionable for a server to decide what a client understands

Zash, xmpp:critical="true", define xmpp prefix on <stream:stream/>, be done with it :-)

but SamWhited will kill me for that

It probably wouldn't have much use outside of initial stream negotation

possibly

Well. Depends

Altho that's working fine enough.

jonasw, nah, I'm pretty much bored

jonasw, I will anyway do what I think better

Basic XEP-0060 question. I don't get how the interaction of proper PubSub (non-PEP) nodes with presence is supposed to work.

E.g. pubsub#access_model=presence. This assumes the PubSub service has access to the node owner's roster data? The node owner might be a remote user, no?

yes, I think so

that access model probably simply doesn’t work for services where this isn’t true

Or pubsub#send_last_published_item=on_sub_and_presence (this is even the default).

This assumes the PubSub service will receive presence from the subscriber?

probably

It's like these parts of 0060 were written without federation in mind.

In theory, the service could send a presence request

Zash, so it’d have to regularly type="probe" the subscribers?

jonasw: if it's subscribed, it should get presence pushed to it, like any other contact

oh good point

you still need to send probes after restart

Sure, it would have to do the things the server does for users with rosters.

Zash, do servers do that on behalf of components?

argh

nevermind, I misread your sentence

Briefly looking over Ge0rG's email. Retraction... Wait there is a retraction feature in MIX? 😂

That makes me wonder what else is in there

daniel: that mail was shorter than I expected.

that mail is long enough to make kmail think for a moment before showing it. well done, Ge0rG) ✏

I wonder if Georg will be talking about the dish washing feature later in the email

:D

no, but there is a valet parking feature.

For scooter and cars?

daniel: there is both user and administrator retraction. Both can be configured on or off

for gas driven locomotives

Ge0rG: will add the valet parking feature in next update

Ge0rG: I plan to respond to your email later this week

Steve Kille: I think there is no need to hurry. It took me multiple weeks to read the XEP and write the mail, so take your time :)

Ge0rG, LMC should probably be changed to use origin-ids

Steve Kille: is this valet parking feature available for my office trip next week? :)

daniel: I think the stanza-id XEP should be changed to enforce a client setting message-id = origin-id.

intosi: only if you implement MIX 2.0 until then

Ge0rG, probably. but that's orthogonal to changing LMC et al to using origin-ids

daniel: changing LMC et all would be a breaking change.

and most sane implementations will already set message-id=origin-id

Ge0rG, sure

but it's a breaking change either way

daniel: it's not a breaking change to fix stanza-id ;)

yes?! but it's still orthogonal

daniel: you might resurrect the thread I had with Flow regarding that topic in October.

let's say its complementary.

even if you force origin-id to be the same as message-id. if lmc gives origin-id precedence over message-id that's still a breaking change

because that xep has to deal with the case where they are not the same

can't we just properly fix message-id, once and for all?

fix meaning?

mandate globally unique message IDs

and just plain reject messages that violate that rule

that doesn't prevent services from rewriting the id though

which is imho the bigger issue

daniel: stopping id rewriting would be the other part of fixing message-id

But is the outgoing message from a MUC really the same message as the incoming one?

Zash: if a tree falls in a MUC, and there are no participants, will there be a presence update?

Schrödingers presence update.

Zash: when you want to LMC a MUC message you just sent, will you use the sent-message-id, the reflected-message-id, the MAM id or the origin-id?

origin-id

(once the XEP says so)

Zash: is it the MUC service's task to track all message ID references it rewrites and fix them?

origin-id, stanza-id

you guys are reinventing version vectors and vector clocks

http://www.abdsphysics.com/uploads/6/5/0/9/65090265/4802565.png?540

I think that it's very typical for the XMPP standardization process that we end up with a message having three different IDs.

Why don't we stick a nonce in each message, apply canonicalization and declare that the message id is a hash of the canonical serialization of it?

fwiw Conversations already gives origin-id precedence over stanza-id when it comes to LMC, reciepts chat markers

/s ;)

daniel: please comment on the "UPDATED: XEP-0359 (Unique and Stable Stanza IDs)" thread.

daniel: thanks! :)