XSF Discussion - 2019-09-18


  1. jonas’

    Daniel, fixed the 500

  2. Daniel

    jonas’, thanks. i'm getting a 400 when trying to search

  3. jonas’

    Daniel, can you extract the error?

  4. jonas’

    because search works for me via curl

  5. jonas’

    and in the web ui via browser

  6. jonas’

    Daniel, the search API returns a JSON on 400, with a single "error" key

  7. Daniel

    it says key keywords is required. but i'm 80% i'm setting that. also i haven’t changed the code since it was last working

  8. jonas’

    Daniel, I’m not sure how that error can be caused except if "keywords" is actually missing

  9. jonas’

    I haven’t touched that code since it was last working either, and it works fine locally with curl

  10. jonas’

    $ curl -s \ --data '{"keywords": ["xsf"]}' \ -H 'Content-Type: application/json; charset=utf-8' \ https://search.jabbercat.org/api/1.0/search | jq -C .

  11. jonas’

    lovetox_, > jonas the black banner looks a bit boring I agree, do you have ideas on how to fix that? :)

  12. lovetox_

    no i hate webdesign

  13. jonas’

    lovetox_, let me rephrase: If you have a visual idea, I may be able to put it in css/httml :)

  14. jonas’

    lovetox_, let me rephrase: If you have a visual idea, I may be able to put it in css/html :)

  15. Ge0rG

    *cough* https://github.com/horazont/muchopper/issues/15 *cough*

  16. pep.

    Ge0rG: I'd make that search button same height as the input field

  17. jonas’

    Ge0rG, I think it might be easy to overlook when integrated in the banner

  18. Ge0rG

    jonas’: that's possible.

  19. Ge0rG

    OTOH, havig two search boxes is slight overkill

  20. jonas’

    but only slight!

  21. Ge0rG

    most of the vertical space is taken up by meta.

  22. jonas’

    the copy to clipboard will be the first piece of JS on that page, Ge0rG, I hope you feel guilty.

  23. jonas’

    the copy to clipboard will be the first piece of JS on the room list page, Ge0rG, I hope you feel guilty.

  24. Daniel

    jonas’, ok never mind. classic heisenbug. if i log the query beforing sending it it works

  25. Daniel

    so it's not you

  26. Ge0rG

    https://upload.yax.im/upload/kmJi9wW3ylSzcV0P/Screenshot_20190918-100107_Firefox.jpg

  27. Ge0rG

    I wanted to paste that screenshot earlier, but yaxim was crashing due to my incorrect use of some StringUtils API

  28. lovetox_

    jonas’ how to whiteliste an avatar?

  29. jonas’

    lovetox_, ask me

  30. jonas’

    it’s documented in the For Owners documentation section

  31. lovetox_

    ok thanks i see you already whitelisted gajim

  32. jonas’

    lovetox_, I had it whitelisted yesterday on launch already, it just took the bot a while to re-scan the gajim muc :)

  33. Ge0rG

    damn, so now I need to load the prosody module after all.

  34. Ge0rG

    And also set the ~horny pe~ yak logo

  35. Ge0rG

    jonas’: great work!

  36. jonas’

    Ge0rG, reload the page

  37. jonas’

    and hover over an entry

  38. lovetox_

    link styling looks a bit weird

  39. Ge0rG

    jonas’: nothing happens?

  40. lovetox_

    i would remove these dots

  41. Ge0rG

    "hover" is not an option on mobile, but then you should probably have a handler for xmpp: there

  42. jonas’

    Ge0rG, yeah, that was my line of thought

  43. jonas’

    oh, I forgot to confirm the pull on the live instance. Ge0rG retry

  44. Ge0rG

    jonas’: awesome! I'd just add a space before the clipboard character.

  45. jonas’

    there is a space there?

  46. jonas’

    maybe hard-reload your things

  47. jonas’

    (it’s a margin-left: 0.5em)

  48. lovetox_

    would it not be better to copy the xmpp:xx?join link?

  49. jonas’

    lovetox_, you can right-click -> copy link address for that one

  50. Ge0rG

    lovetox_: no

  51. Ge0rG

    jonas’: ah, I had some intermediate version

  52. Ge0rG

    jonas’: after clicking the clipboard, it remains visible and marked until you click somewhere

  53. jonas’

    intentional

  54. jonas’

    to a certain extent

  55. jonas’

    I want it to be visible while focused so that it can be reached and used with tab

  56. jonas’

    that’s the price to pay for that, I guess

  57. Ge0rG

    so what do I need on prosody 0.11 for MUC avatars?

  58. jonas’

    Ge0rG, also, reload and retry that click thing

  59. Zash

    Ge0rG: There's a community module of course.

  60. Ge0rG

    lovetox_: http://paste.debian.net/1101390/ :(

  61. Zash

    Tho technically possible to only use the included mod_vcard, but then it's read-only and you'd have to manually add the data.

  62. Ge0rG

    jonas’: much better now

  63. Ge0rG

    Zash: | /usr/lib/prosody/modules/mod_a community module.lua: No such file or directory

  64. lovetox_

    yeah Ge0rG i really dont have a solution for that yet

  65. lovetox_

    i want to parse validate jids in the xmpp lib

  66. lovetox_

    but on encountering invalid ones, im not sure what to do

  67. Ge0rG

    lovetox_: don't crash the xml stream.

  68. jonas’

    Ge0rG, wanna tell me what you want to have whitelisted?

  69. Ge0rG

    jonas’: chat.yax.im

  70. jonas’

    no

  71. jonas’

    yaxim@chat.yax.im?

  72. Ge0rG

    jonas’: the first four of https://search.jabbercat.org/search?q=chat.yax.im

  73. Ge0rG

    maybe also test@chat.yax.im - if you dare

  74. lovetox_

    yeah Ge0rG but thats not so easy, on the other side, server should not send messages from invalid jids

  75. lovetox_

    🙂

  76. jonas’

    the new copy button is super-convenient! :)

  77. jonas’

    the new copy button is super-convenient for building the white-list! :)

  78. Ge0rG

    lovetox_: 🤖 will disagree with you.

  79. jonas’

    https://search.jabbercat.org/search?q=xmpp-de the second hit is amusing

  80. Ge0rG

    lovetox_: from the stacktrace it seems that it's a JID in my roster, ♥@ツ.op-co.de - so that Gajim will never properly connect to my account again

  81. lovetox_

    are you saying this is a valid localpart?

  82. lovetox_

    dont think its the roster to be honest, its a stanza where the from attr has a invalid jid

  83. lovetox_

    roster pushes are from your server, not from contacts

  84. Ge0rG

    Maybe it's presence unavailable then?

  85. lovetox_

    maybe, i will fix that later if i have time

  86. Ge0rG

    lovetox_: I'm not saying that it's a valid JID (I don't know), but I'm saying that it's accepted by my server, and thus shouldn't crash my client.

  87. lovetox_

    but server should really get going with precis

  88. Ge0rG

    Because it's a cheap DoS otherwise.

  89. Ge0rG

    lovetox_: we've had that recently. It should be enforced by the server that's authoritative for the JID.

  90. lovetox_

    thats not an argument, rfc says jids MUST be validated by server

  91. Ge0rG

    If it doesn't, you are rather out of luck

  92. Ge0rG

    lovetox_: okay, but there are different unicode versions and different xmpp versions. And now everybody can crash your client by sending something from an illegal JID

  93. lovetox_

    this local part was allowed once? they really updated the rfc so that already existing accounts got invalid?

  94. Ge0rG

    Where "illegal" depends on your python version, your server version and the position of the moon.

  95. lovetox_

    sounds weird

  96. Ge0rG

    NodePrep and PRECIS aren't an exact match, so you are guaranteed to end up with illegal JIDs _somewhere_

  97. lovetox_

    yeah, i will try to fix that 🙂

  98. Zash

    It's likely that U2665 is undefined in Unicode 3.2 which NodePrep uses.

  99. Zash

    libidn defaults to allowing undefined characters, and thus Prosody does too.

  100. lovetox_

    Ge0rG keep that jid, when i have a fix then you can test if its working 🙂

  101. Ge0rG

    lovetox_: feel free to add it to your roster

  102. lovetox_

    would need a client that allows that, Gajim does not 😃

  103. jonas’

    lovetox_, https://mailarchive.ietf.org/arch/msg/xmpp/a-WhzOTyOq168GujQHgzQ1-DURI for an idea of how deep the mess is we’re in

  104. Ge0rG &

  105. lovetox_

    no i write a test for the xmpp lib 🙂

  106. lovetox_

    genius all undefined characters are allowed 😃

  107. flow

    lovetox_, not by the relevant PRECIS profile. I would suggest to remove the faulty stanza from the stream as first course of action

  108. pep.

    https://github.com/xsf/xeps/pull/827 is this some kind of joke? :x

  109. sonny

    yes, sorry I couldn't resist

  110. Zash

    pep.: Read the rest of the XEP

  111. pep.

    Zash: I know the xep, I implement it in my rot13 and b64 plugins for poezio.

  112. sonny

    and it still wasn't clear my comment was a joke? I should be more careful 🙂

  113. Link Mauve

    “07:45:33 flow> Link Mauve, that C&C Renegage remake?”, yes, they are using XMPP for their lobby thing now.

  114. Link Mauve

    Not for the actual game (yet?).

  115. Link Mauve

    “18:47:16 Zash> Are there clients that still fail if you stop advertising <session/>?”, IIRC libpurple did.

  116. Link Mauve

    That’s second-hand experience from IIRC Maranda.

  117. jonas’

    excellent, so if we ever want to move past pidgin, that’s how.

  118. Zash

    Confirmed. "Error initializing session" it says.

  119. Zash

    It's sending the session thing even if it's not advertised.

  120. MattJ

    When we added <optional> there were definitely other clients that failed too

  121. MattJ

    I think one was Psi/Gajim, and probably some other that is extinct these days

  122. MattJ

    and clients couldn't just drop it, because ejabberd had weird behaviour if you didn't do it

  123. MattJ

    even though it is, and always was, a no-op in Prosody

  124. jonas’

    yeah, weeiiiird behaviour

  125. MattJ

    iirc it would let you log in, but timeout your session after some time

  126. MattJ

    even though you were using it fine

  127. jonas’

    I’m pretty sure I was unable to send stanzas

  128. jonas’

    otherwise I would probably not have noticed when developing aioxmpp

  129. jonas’

    unfortunately, that was back in 2015 where I wasn’t as good at commit message writing as I’m now, so the error behaviour is lost in history

  130. Daniel

    i vaguely remembering not being able to send and receive message or something

  131. Daniel

    more than a subtle 'will log you out at some point'

  132. Link Mauve

    jonas’, could you also whitelist SVGs from chat.jabberfr.org on muclumbus?

  133. jonas’

    Link Mauve, SVGs are not supported

  134. jonas’

    Link Mauve, and I generally only white-list single rooms where I trust the owners or domains where only admins can create rooms (and I trust the admins to a certain extent)

  135. Ge0rG

    jonas’: but then again, you trust *me*

  136. Daniel

    jonas’: are the avatars going to be exposed over the api as well?

  137. Holger

    MattJ, Daniel, jonas': FWIW yes back then you got no error from ejabberd when sending random stanzas while it was waiting for `<session/>`, but you'd neither be able to send nor receive anything. You also shouldn't time out though as long as you do send random crap :-)

  138. Link Mauve

    jonas’, why do you limit it to ¬SVG?

  139. Link Mauve

    jonas’, once you start supporting SVG, you can whitelist the first 12 entries here: https://search.jabbercat.org/search?q=chat.jabberfr.org

  140. Link Mauve

    And also the first four here: https://search.jabbercat.org/search?q=chat.khaganat.net

  141. Zash

    Hm, why don't we have 157 for individual rooms?

  142. Ge0rG

    Why don't we have 157 for individual users?

  143. jonas’

    Link Mauve, I limit it to PNG and JPEG to be precise

  144. Link Mauve

    Any reason for that?

  145. jonas’

    Link Mauve, rescaling

  146. MattJ

    I heard SVG has trouble with that

  147. Link Mauve

    It’s free to do in the browser with SVG.

  148. jonas’

    and embedding SVG would require a separate domain to prevent any funny javascript attacks in SVG

  149. jonas’

    or at least a CSP

  150. Link Mauve

    jonas’, use <img/> instead of embedding it.

  151. jonas’

    all things I don’t know about

  152. jonas’

    Link Mauve, will that stop it from executing scripts?

  153. Link Mauve

    Yes.

  154. jonas’

    are you sure? if so, why?

  155. Link Mauve

    Yes I am; because images are expected since about forever to not contain scripts.

  156. jonas’

    yeah well, I expect the web to be sane since forever and it isn’t

  157. Link Mauve

    And browsers would be very cautious to not open such a huge hole that can even do cross-site things.

  158. jonas’

    they have traditionally done that, I don’t trust browsers

  159. moparisthebest

    I was fairly confident SVGs even with image could execute javascript

  160. Link Mauve

    For instance example.org has an <img src="https://evil.com/image.png"> on some page, now evil.com starts serving a SVG containing a script at this URL.

  161. jonas’

    Link Mauve, am I still safe when I use the SVG as background-image?

  162. Link Mauve

    jonas’, yes, it also is in an image context.

  163. Link Mauve

    moparisthebest, do you have a link to an attack description?

  164. Link Mauve

    jonas’, what isn’t would be to use <object/> or <embed/> or <iframe/>.

  165. MattJ

    jonas’, https://www.w3.org/wiki/SVG_Security

  166. jonas’

    MattJ, thanks

  167. moparisthebest

    yea if browsers actually implement it like that it should be safe, neat

  168. Seve

    >For instance example.org has an <img src="https://evil.com/image.png"> on some page, now evil.com starts serving a SVG containing a script at this URL. Haven't you said earlier to use <img/>?

  169. Link Mauve

    Seve, yes.

  170. Seve

    Ah you mean nothing would happen if that is the case

  171. Link Mauve

    Yes, this story is about as obvious as it could be, and the reason why browsers took so long to accept SVG in image contexts.

  172. moparisthebest

    jonas’ is still right though in that if someone links to https://somedomain.com/evil.svg that'll still execute javascript and be able to steal cookies and XSS from somedomain.com, so if you allow SVG uploads, you need different domain or CSP or similar

  173. MattJ

    Right-click->View Image

  174. Link Mauve

    Or a data: URI.

  175. Link Mauve

    Or a Blob.

  176. Link Mauve

    Or any of the various mechanisms that aren’t tied to the domain.

  177. Link Mauve

    (CSP is very good.)

  178. jonas’

    moparisthebest, thanks, good point

  179. jonas’

    a CSP it is

  180. jonas’

    the sane thing would be to disable all csript things for the avatar endpoint?

  181. Link Mauve

    Probably also all external CSS.

  182. jonas’

    everything

  183. jonas’

    how do I do that?

  184. Ge0rG

    Just burn the web. With napalm

  185. Link Mauve

    At JabberFR, for our HTTP File Upload domain, we use: Content-Security-Policy: frame-ancestors 'none'; default-src 'none'; img-src 'self'; media-src 'self'; report-uri /report-csp-violation

  186. Link Mauve

    I should add style-src 'self'.

  187. Ge0rG

    Link Mauve: do you have a csp violation listener at /?

  188. Zash

    ``` ~$ ssh snikket2 ~$ more /etc/nginx/snippets/csp-strict.conf add_header Content-Security-Policy "default-src 'none'; img-src 'self'; style-src 'self'; font-src 'self'"; ```

  189. Link Mauve

    Ge0rG, yes.

  190. Link Mauve

    For every domain.

  191. moparisthebest

    for a site like yours jonas’ I'd probably just disable everything with CSP, then enable as you need

  192. jonas’

    yeah

  193. Link Mauve

    That’s default-src 'none'.

  194. Ge0rG

    Link Mauve: what's the implementation you are using?

  195. Ge0rG is asking for a friend

  196. jonas’

    default-src 'none'; seems to still allow SVG to be rendered

  197. Link Mauve

    Ge0rG, the one which uses a ton of memory.

  198. Zash

    It's funny when you disable the CSS in SVG.

  199. Ge0rG

    Link Mauve: ejabberd?

  200. Link Mauve

    Zash, yes, which is why you should also have style-src 'unsafe-inline'.

  201. Link Mauve

    Ge0rG, ah no, Prosody.

  202. Link Mauve

    With the mod_http_upload community module.

  203. Ge0rG

    Link Mauve: prosody is a csp violation reporting tool?

  204. jonas’

    Link Mauve, > frame-ancestors 'none'; default-src 'none'; style-src 'unsafe-inline';

  205. jonas’

    does that sound good?

  206. Link Mauve

    jonas’, sounds nice yeah.

  207. Link Mauve

    You may want to add a listener so that you know what your users are being prevented from fetching, at least for a short while.

  208. jonas’

    ... nah :)

  209. jonas’

    I only set it on the avatar endpoint

  210. jonas’

    people will complain when their avatars aren’t working

  211. Link Mauve

    Ge0rG, ah no, I’m using one which sends them to my JID.

  212. Link Mauve

    Let me figure it out.

  213. Link Mauve

    pep., do you remember where I put it?

  214. jonas’

    Link Mauve, https://search.jabbercat.org/search?q=jabberfr

  215. jonas’

    the others will be updated when the scanner gets to them

  216. Link Mauve

    \o/

  217. Link Mauve

    I should add the new muc#roominfo_webchat_url thing we are going to accept shortly. ^^

  218. Link Mauve

    Since we also have that.

  219. Link Mauve

    https://chat.jabberfr.org/ could use an emoji instead of a SVG now, for the join bubble.

  220. jonas’

    indeed

  221. jonas’

    thinking of directly embedding small avatars as data URLs for faster loading

  222. jonas’

    thinking of directly embedding small avatars as data URIs for faster loading

  223. jonas’

    not sure how to write the query so that it only fetches the avatar from the database for the result list if it’s small enough

  224. Link Mauve

    Indeed, on just chat.jabberfr.org it takes about 700ms for me to download every SVG.

  225. jonas’

    to the SQL console!

  226. jonas’

    ugh, the only standard-SQL-way of doing it might be with a self-join

  227. jonas’

    hm, however, doing that would break caching

  228. moparisthebest

    another column that's only non-null if small enough? :/

  229. jonas’

    in postgres, I can do: select mime_type, case when length(data) < 16384 then data else null end from avatar;

  230. jonas’

    but locally I’m testing with sqlite, soo....

  231. jonas’

    oh, sqlite also has case

  232. moparisthebest

    was going to say I feel like that should work in sqlite

  233. jonas’

    and sqlalchemy supports it, too

  234. jonas’

    now the question is, does it even make sense to embed the avatars?

  235. jonas’

    right now, we send a 304 for avatars if they’re still cached on the client

  236. jonas’

    we can’t do that for the result page

  237. jonas’

    so we’d send all avatars of a page to the client, inefficiently base64-encoded

  238. jonas’

    every time, since they can’t benefit from caching

  239. moparisthebest

    you'd probably get more benefit from supporting http/2

  240. jonas’

    tell that to apache

  241. moparisthebest

    because then browser could request all the images over one connection at the same time etc

  242. moparisthebest

    apache surely supports http/2 by now?

  243. jonas’

    dunno

  244. jonas’

    if it isn’t on by default, I don’t bother

  245. Link Mauve

    moparisthebest, they already can with HTTP/1.1.

  246. moparisthebest

    not in parallel though?

  247. jonas’

    SVG support brought a few new avatars to the listing

  248. jonas’

    HTTP/2 seems to be a clusterfuck regarding the parallelism anyways. no idea why they thought it’d be a good idea to re-implement multiplexing which we alreday have with TCP

  249. Zash

    jonas’, itym SCTP

  250. Link Mauve

    moparisthebest, how does HTTP/2 do that?

  251. moparisthebest

    maybe, I mean http/3 is coming, but regardless it's still way faster for browsers

  252. jonas’

    or that

  253. Zash

    But we can't have SCTP because broken middleboxes

  254. Zash

    And NAT, and all the other things that ruin all nice things

  255. Zash

    Is MPTCP still alive?

  256. moparisthebest

    Link Mauve, like jonas’ said they reinvented multiplexing etc etc

  257. Link Mauve

    So instead we have WebRTC, which is like SCTP over RTP.

  258. Link Mauve

    moparisthebest, but in parallel you mean over multiple TCP connections, or…?

  259. moparisthebest

    http/3 over quic sounds pretty sweet though

  260. moparisthebest

    no just one

  261. moparisthebest

    it's probably just a config option to enable http2 jonas’ you should look at it

  262. Link Mauve

    moparisthebest, what is the difference between that and HTTP/1.1 pipelining then?

  263. moparisthebest

    I assume it's new enough because you support TLS 1.3 :) https://www.ssllabs.com/ssltest/analyze.html?d=search.jabbercat.org&s=2a01%3a4f9%3a2b%3a2c50%3a1010%3a1010%3a0%3a1&latest

  264. jonas’

    Link Mauve, HTTP/2 introduces its own framing on top of TCP

  265. jonas’

    so you’d be requesting resources A B C, and then receive chunks A1 B1 C1 A2 B2 C2 ...

  266. jonas’

    also the server may push you resources of which it expects you to need some

  267. moparisthebest

    https://stackoverflow.com/questions/34478967/what-is-the-difference-between-http-1-1-pipelining-and-http-2-multiplexing I guess, plenty of different search results

  268. Link Mauve

    jonas’, oh, I see.

  269. moparisthebest

    bottom line is, it's far far faster, so turn it on

  270. jonas’

    it’s also much more broken

  271. jonas’

    https://github.com/Netflix/security-bulletins/blob/master/advisories/third-party/2019-002.md

  272. moparisthebest

    than ?

  273. jonas’

    I’m going to wait until the HTTP/2 implementations settle down before deploying that and having yet another headache to deal with

  274. moparisthebest

    http 1.1 pipelining is pretty broken too iirc :) everything is broken

  275. Link Mauve

    The web especially.

  276. jonas’

    https://github.com/Netflix/security-bulletins/blob/master/advisories/third-party/2019-002.md#attacks

  277. jonas’

    plenty of good stuff in there

  278. Zash

    jonas’, do the compromise I did on prosody.im, only do HTTP/2 over IPv6

  279. Link Mauve

    jonas’, do you have something setup for i18n for muclumbus yet?

  280. jonas’

    Link Mauve, not really