XSF Discussion - 2019-09-18

Daniel, fixed the 500

jonas’, thanks. i'm getting a 400 when trying to search

Daniel, can you extract the error?

because search works for me via curl

and in the web ui via browser

Daniel, the search API returns a JSON on 400, with a single "error" key

it says key keywords is required. but i'm 80% i'm setting that. also i haven’t changed the code since it was last working

Daniel, I’m not sure how that error can be caused except if "keywords" is actually missing

I haven’t touched that code since it was last working either, and it works fine locally with curl

$ curl -s \ --data '{"keywords": ["xsf"]}' \ -H 'Content-Type: application/json; charset=utf-8' \ https://search.jabbercat.org/api/1.0/search | jq -C .

lovetox_, > jonas the black banner looks a bit boring I agree, do you have ideas on how to fix that? :)

no i hate webdesign

lovetox_, let me rephrase: If you have a visual idea, I may be able to put it in css/html :) ✏

*cough* https://github.com/horazont/muchopper/issues/15 *cough*

Ge0rG: I'd make that search button same height as the input field

Ge0rG, I think it might be easy to overlook when integrated in the banner

jonas’: that's possible.

OTOH, havig two search boxes is slight overkill

but only slight!

most of the vertical space is taken up by meta.

the copy to clipboard will be the first piece of JS on the room list page, Ge0rG, I hope you feel guilty. ✏

jonas’, ok never mind. classic heisenbug. if i log the query beforing sending it it works

so it's not you

https://upload.yax.im/upload/kmJi9wW3ylSzcV0P/Screenshot_20190918-100107_Firefox.jpg

I wanted to paste that screenshot earlier, but yaxim was crashing due to my incorrect use of some StringUtils API

jonas’ how to whiteliste an avatar?

lovetox_, ask me

it’s documented in the For Owners documentation section

ok thanks i see you already whitelisted gajim

lovetox_, I had it whitelisted yesterday on launch already, it just took the bot a while to re-scan the gajim muc :)

damn, so now I need to load the prosody module after all.

And also set the ~horny pe~ yak logo

jonas’: great work!

Ge0rG, reload the page

and hover over an entry

link styling looks a bit weird

jonas’: nothing happens?

i would remove these dots

"hover" is not an option on mobile, but then you should probably have a handler for xmpp: there

Ge0rG, yeah, that was my line of thought

oh, I forgot to confirm the pull on the live instance. Ge0rG retry

jonas’: awesome! I'd just add a space before the clipboard character.

there is a space there?

maybe hard-reload your things

(it’s a margin-left: 0.5em)

would it not be better to copy the xmpp:xx?join link?

lovetox_, you can right-click -> copy link address for that one

lovetox_: no

jonas’: ah, I had some intermediate version

jonas’: after clicking the clipboard, it remains visible and marked until you click somewhere

intentional

to a certain extent

I want it to be visible while focused so that it can be reached and used with tab

that’s the price to pay for that, I guess

so what do I need on prosody 0.11 for MUC avatars?

Ge0rG, also, reload and retry that click thing

Ge0rG: There's a community module of course.

lovetox_: http://paste.debian.net/1101390/ :(

Tho technically possible to only use the included mod_vcard, but then it's read-only and you'd have to manually add the data.

jonas’: much better now

Zash: | /usr/lib/prosody/modules/mod_a community module.lua: No such file or directory

yeah Ge0rG i really dont have a solution for that yet

i want to parse validate jids in the xmpp lib

but on encountering invalid ones, im not sure what to do

lovetox_: don't crash the xml stream.

Ge0rG, wanna tell me what you want to have whitelisted?

jonas’: chat.yax.im

no

yaxim@chat.yax.im?

jonas’: the first four of https://search.jabbercat.org/search?q=chat.yax.im

maybe also test@chat.yax.im - if you dare

yeah Ge0rG but thats not so easy, on the other side, server should not send messages from invalid jids

🙂

the new copy button is super-convenient for building the white-list! :) ✏

lovetox_: 🤖 will disagree with you.

https://search.jabbercat.org/search?q=xmpp-de the second hit is amusing

lovetox_: from the stacktrace it seems that it's a JID in my roster, ♥@ツ.op-co.de - so that Gajim will never properly connect to my account again

are you saying this is a valid localpart?

dont think its the roster to be honest, its a stanza where the from attr has a invalid jid

roster pushes are from your server, not from contacts

Maybe it's presence unavailable then?

maybe, i will fix that later if i have time

lovetox_: I'm not saying that it's a valid JID (I don't know), but I'm saying that it's accepted by my server, and thus shouldn't crash my client.

but server should really get going with precis

Because it's a cheap DoS otherwise.

lovetox_: we've had that recently. It should be enforced by the server that's authoritative for the JID.

thats not an argument, rfc says jids MUST be validated by server

If it doesn't, you are rather out of luck

lovetox_: okay, but there are different unicode versions and different xmpp versions. And now everybody can crash your client by sending something from an illegal JID

this local part was allowed once? they really updated the rfc so that already existing accounts got invalid?

Where "illegal" depends on your python version, your server version and the position of the moon.

sounds weird

NodePrep and PRECIS aren't an exact match, so you are guaranteed to end up with illegal JIDs _somewhere_

yeah, i will try to fix that 🙂

It's likely that U2665 is undefined in Unicode 3.2 which NodePrep uses.

libidn defaults to allowing undefined characters, and thus Prosody does too.

Ge0rG keep that jid, when i have a fix then you can test if its working 🙂

lovetox_: feel free to add it to your roster

would need a client that allows that, Gajim does not 😃

lovetox_, https://mailarchive.ietf.org/arch/msg/xmpp/a-WhzOTyOq168GujQHgzQ1-DURI for an idea of how deep the mess is we’re in

no i write a test for the xmpp lib 🙂

genius all undefined characters are allowed 😃

lovetox_, not by the relevant PRECIS profile. I would suggest to remove the faulty stanza from the stream as first course of action

https://github.com/xsf/xeps/pull/827 is this some kind of joke? :x

yes, sorry I couldn't resist

pep.: Read the rest of the XEP

Zash: I know the xep, I implement it in my rot13 and b64 plugins for poezio.

and it still wasn't clear my comment was a joke? I should be more careful 🙂

“07:45:33 flow> Link Mauve, that C&C Renegage remake?”, yes, they are using XMPP for their lobby thing now.

Not for the actual game (yet?).

“18:47:16 Zash> Are there clients that still fail if you stop advertising <session/>?”, IIRC libpurple did.

That’s second-hand experience from IIRC Maranda.

excellent, so if we ever want to move past pidgin, that’s how.

Confirmed. "Error initializing session" it says.

It's sending the session thing even if it's not advertised.

When we added <optional> there were definitely other clients that failed too

I think one was Psi/Gajim, and probably some other that is extinct these days

and clients couldn't just drop it, because ejabberd had weird behaviour if you didn't do it

even though it is, and always was, a no-op in Prosody

yeah, weeiiiird behaviour

iirc it would let you log in, but timeout your session after some time

even though you were using it fine

I’m pretty sure I was unable to send stanzas

otherwise I would probably not have noticed when developing aioxmpp

unfortunately, that was back in 2015 where I wasn’t as good at commit message writing as I’m now, so the error behaviour is lost in history

i vaguely remembering not being able to send and receive message or something

more than a subtle 'will log you out at some point'

jonas’, could you also whitelist SVGs from chat.jabberfr.org on muclumbus?

Link Mauve, SVGs are not supported

Link Mauve, and I generally only white-list single rooms where I trust the owners or domains where only admins can create rooms (and I trust the admins to a certain extent)

jonas’: but then again, you trust *me*

jonas’: are the avatars going to be exposed over the api as well?

MattJ, Daniel, jonas': FWIW yes back then you got no error from ejabberd when sending random stanzas while it was waiting for `<session/>`, but you'd neither be able to send nor receive anything. You also shouldn't time out though as long as you do send random crap :-)

jonas’, why do you limit it to ¬SVG?

jonas’, once you start supporting SVG, you can whitelist the first 12 entries here: https://search.jabbercat.org/search?q=chat.jabberfr.org

And also the first four here: https://search.jabbercat.org/search?q=chat.khaganat.net

Hm, why don't we have 157 for individual rooms?

Why don't we have 157 for individual users?

Link Mauve, I limit it to PNG and JPEG to be precise

Any reason for that?

Link Mauve, rescaling

I heard SVG has trouble with that

It’s free to do in the browser with SVG.

and embedding SVG would require a separate domain to prevent any funny javascript attacks in SVG

or at least a CSP

jonas’, use <img/> instead of embedding it.

all things I don’t know about

Link Mauve, will that stop it from executing scripts?

Yes.

are you sure? if so, why?

Yes I am; because images are expected since about forever to not contain scripts.

yeah well, I expect the web to be sane since forever and it isn’t

And browsers would be very cautious to not open such a huge hole that can even do cross-site things.

they have traditionally done that, I don’t trust browsers

I was fairly confident SVGs even with image could execute javascript

For instance example.org has an <img src="https://evil.com/image.png"> on some page, now evil.com starts serving a SVG containing a script at this URL.

Link Mauve, am I still safe when I use the SVG as background-image?

jonas’, yes, it also is in an image context.

moparisthebest, do you have a link to an attack description?

jonas’, what isn’t would be to use <object/> or <embed/> or <iframe/>.

jonas’, https://www.w3.org/wiki/SVG_Security

MattJ, thanks

yea if browsers actually implement it like that it should be safe, neat

>For instance example.org has an <img src="https://evil.com/image.png"> on some page, now evil.com starts serving a SVG containing a script at this URL. Haven't you said earlier to use <img/>?

Seve, yes.

Ah you mean nothing would happen if that is the case

Yes, this story is about as obvious as it could be, and the reason why browsers took so long to accept SVG in image contexts.

jonas’ is still right though in that if someone links to https://somedomain.com/evil.svg that'll still execute javascript and be able to steal cookies and XSS from somedomain.com, so if you allow SVG uploads, you need different domain or CSP or similar

Right-click->View Image

Or a data: URI.

Or a Blob.

Or any of the various mechanisms that aren’t tied to the domain.

(CSP is very good.)

moparisthebest, thanks, good point

a CSP it is

the sane thing would be to disable all csript things for the avatar endpoint?

Probably also all external CSS.

everything

how do I do that?

Just burn the web. With napalm

At JabberFR, for our HTTP File Upload domain, we use: Content-Security-Policy: frame-ancestors 'none'; default-src 'none'; img-src 'self'; media-src 'self'; report-uri /report-csp-violation

I should add style-src 'self'.

Link Mauve: do you have a csp violation listener at /?

``` ~$ ssh snikket2 ~$ more /etc/nginx/snippets/csp-strict.conf add_header Content-Security-Policy "default-src 'none'; img-src 'self'; style-src 'self'; font-src 'self'"; ```

Ge0rG, yes.

For every domain.

for a site like yours jonas’ I'd probably just disable everything with CSP, then enable as you need

yeah

That’s default-src 'none'.

Link Mauve: what's the implementation you are using?

default-src 'none'; seems to still allow SVG to be rendered

Ge0rG, the one which uses a ton of memory.

It's funny when you disable the CSS in SVG.

Link Mauve: ejabberd?

Zash, yes, which is why you should also have style-src 'unsafe-inline'.

Ge0rG, ah no, Prosody.

With the mod_http_upload community module.

Link Mauve: prosody is a csp violation reporting tool?

Link Mauve, > frame-ancestors 'none'; default-src 'none'; style-src 'unsafe-inline';

does that sound good?

jonas’, sounds nice yeah.

You may want to add a listener so that you know what your users are being prevented from fetching, at least for a short while.

... nah :)

I only set it on the avatar endpoint

people will complain when their avatars aren’t working

Ge0rG, ah no, I’m using one which sends them to my JID.

Let me figure it out.

pep., do you remember where I put it?

Link Mauve, https://search.jabbercat.org/search?q=jabberfr

the others will be updated when the scanner gets to them

\o/

I should add the new muc#roominfo_webchat_url thing we are going to accept shortly. ^^

Since we also have that.

https://chat.jabberfr.org/ could use an emoji instead of a SVG now, for the join bubble.

indeed

thinking of directly embedding small avatars as data URIs for faster loading ✏

not sure how to write the query so that it only fetches the avatar from the database for the result list if it’s small enough

Indeed, on just chat.jabberfr.org it takes about 700ms for me to download every SVG.

to the SQL console!

ugh, the only standard-SQL-way of doing it might be with a self-join

hm, however, doing that would break caching

another column that's only non-null if small enough? :/

in postgres, I can do: select mime_type, case when length(data) < 16384 then data else null end from avatar;

but locally I’m testing with sqlite, soo....

oh, sqlite also has case

was going to say I feel like that should work in sqlite

and sqlalchemy supports it, too

now the question is, does it even make sense to embed the avatars?

right now, we send a 304 for avatars if they’re still cached on the client

we can’t do that for the result page

so we’d send all avatars of a page to the client, inefficiently base64-encoded

every time, since they can’t benefit from caching

you'd probably get more benefit from supporting http/2

tell that to apache

because then browser could request all the images over one connection at the same time etc

apache surely supports http/2 by now?

dunno

if it isn’t on by default, I don’t bother

moparisthebest, they already can with HTTP/1.1.

not in parallel though?

SVG support brought a few new avatars to the listing

HTTP/2 seems to be a clusterfuck regarding the parallelism anyways. no idea why they thought it’d be a good idea to re-implement multiplexing which we alreday have with TCP

jonas’, itym SCTP

moparisthebest, how does HTTP/2 do that?

maybe, I mean http/3 is coming, but regardless it's still way faster for browsers

or that

But we can't have SCTP because broken middleboxes

And NAT, and all the other things that ruin all nice things

Is MPTCP still alive?

Link Mauve, like jonas’ said they reinvented multiplexing etc etc

So instead we have WebRTC, which is like SCTP over RTP.

moparisthebest, but in parallel you mean over multiple TCP connections, or…?

http/3 over quic sounds pretty sweet though

no just one

it's probably just a config option to enable http2 jonas’ you should look at it

moparisthebest, what is the difference between that and HTTP/1.1 pipelining then?

I assume it's new enough because you support TLS 1.3 :) https://www.ssllabs.com/ssltest/analyze.html?d=search.jabbercat.org&s=2a01%3a4f9%3a2b%3a2c50%3a1010%3a1010%3a0%3a1&latest

Link Mauve, HTTP/2 introduces its own framing on top of TCP

so you’d be requesting resources A B C, and then receive chunks A1 B1 C1 A2 B2 C2 ...

also the server may push you resources of which it expects you to need some

https://stackoverflow.com/questions/34478967/what-is-the-difference-between-http-1-1-pipelining-and-http-2-multiplexing I guess, plenty of different search results

jonas’, oh, I see.

bottom line is, it's far far faster, so turn it on

it’s also much more broken

https://github.com/Netflix/security-bulletins/blob/master/advisories/third-party/2019-002.md

than ?

I’m going to wait until the HTTP/2 implementations settle down before deploying that and having yet another headache to deal with

http 1.1 pipelining is pretty broken too iirc :) everything is broken

The web especially.

https://github.com/Netflix/security-bulletins/blob/master/advisories/third-party/2019-002.md#attacks

plenty of good stuff in there

jonas’, do the compromise I did on prosody.im, only do HTTP/2 over IPv6

jonas’, do you have something setup for i18n for muclumbus yet?

Link Mauve, not really