XSF Discussion - 2017-11-06


  1. Guus

    Holger: done. Check your mail.

  2. zinid

    can I have an account there too?

  3. zinid

    zinid - xramtsov@gmail.com

  4. Kev

    Done.

  5. zinid

    thx

  6. jonasw

    SamWhited, you might want to update https://wiki.xmpp.org/web/Sam_Whited_for_Council_2017#Better_recommendations

  7. Kev

    jonasw: Actually, the deadline for applications has passed, so we shouldn't be editing our applications really.

  8. jonasw

    Kev, I know. I just wanted to point it out for Sam to judge, given that the content is obviously deprecated and the thing which is deprecated has changed before the deadline

  9. Guus

    Happy to see that we at least have a full complement of candidates. I was worried there for a bit.

  10. jonasw

    Kev, also, MattJs application for board is still null

  11. Kev

    Don't think that's true.

  12. jonasw

    Kev, then at least it’s not properly linked

  13. Kev

    Have you tried clicking it? :)

  14. Guus

    there's a page for Matt, but the link shows red for me. Some weird caching issue in mediawiki that I though was fixed.

  15. jonasw

    no. stupid caches.

  16. jonasw

    the three issues in computer science. Cache invalidation and Off-by-one errors.

  17. Guus

    I like how 50% of the applications were made on the day of the deadline.

  18. jonasw

    I don’t like that.

  19. Guus

    you're right, it's not ideal, but it's funny to see how procrastination is at work each time. At least, I'm hoping that it's just that.

  20. jonasw

    I too do hope that

  21. intosi

    Usually is for me when I'm late at putting up my reapplication ;)

  22. jonasw

    is there a summary on how the voting for Board & Council works for those who find the Bylaws hard to read?

  23. jonasw

    is it that you vote for each individual yes/no and each individual has to be elected by majority to be part of that group?

  24. jonasw

    if so, what happens if n>m (where m being the size of the group) individuals get elected?

  25. Guus

    I'm just hoping that procrastination combined with wiki account loss didn't prevent people from applying.

  26. Guus

    jonasw: I don't know, not do I think your explanation was clearer than the bylaws 😉

  27. jonasw

    I’m sorry :P

  28. goffi

    Guus: I was considering applying but I gave up as I've already too much work and I can't take any more engement this year.

  29. goffi

    commitment*

  30. Kev

    jonasw: Top five, basically.

  31. jonasw

    top five with respect to what?

  32. jonasw

    do we get one yes/no vote per candidate?

  33. Kev

    ISTR we pick (up to) our chosen five, those count as yes, others count as no.

  34. Guus

    istr istr istr

  35. Guus

    I...

  36. Guus

    it stands to reason?

  37. Guus

    goffi: totally understandable. I'm just hoping that there we not people that did want to apply, but ended up missing the deadline because the forgot about / were not aware of the loss of wiki accounts

  38. Guus

    (we had a couple of requests tonight to recreate accounts, which made me wonder)

  39. Kev

    I seem to recall

  40. jonasw

    Guus, I’m confident that those people would step up if that was the only reason

  41. jonasw

    in which case I’m sure that we could extend the period retrospectively since this was an issue outside of their control

  42. Guus

    I kind of disagree with the 'outside of their control' classification, but would be in favor of accepting late candidacies.

  43. Kev

    If anyone had said they needed an account, and didn't get one, yesterday that'd be one thing, but if anyone comes in today and says they want to apply, they obviously missed it.

  44. Guus

    There's the "ah, I couldn't sign up and was confident that requesting a new account wouldn't get me a new account before the deadline" argument. If we would be extending the deadline (which I don't think we are), there's not much reason to accept one type, but not another.

  45. Kev

    I suggest we stop debating what to do in a situation we don't have :)

  46. jonasw

    :)

  47. jonasw

    it’s monday morning, don’t judge people for a desire to distract themselves with irrelevant scenarios

  48. edhelas

    Kev +1

  49. edhelas

    by the way, is there people that are interested to come to T-DOSE this month ? https://wiki.xmpp.org/web/T-DOSE_2017

  50. Guus

    I'll be at T-Dose! :)

  51. Guus

    let me put out a tweet for that event

  52. Guus

    edhelas: we should prepare for some demos and the like. Thoughts?

  53. Guus

    edhelas: would you mind drafting a blogpost that announces our presence there?

  54. edhelas

    I don't have much time for that atm sorry

  55. goffi

    Guus: by the may, I need to recreate account too

  56. goffi

    no that you're talking about it :p

  57. goffi

    now*

  58. Guus

    goffi: desired nickname and email address please

  59. Ge0rG

    It looks like we finally have a situation where wiki account creation is almost instant.

  60. goffi

    Guus: answered in P.V.

  61. edhelas

    Ge0rG API over XMPP MUC

  62. jonasw

    XML-RPC!

  63. goffi

    ad-hoc commands

  64. goffi

    that works great

  65. goffi

    I love this XEP

  66. edhelas

    if only it was more user friendly

  67. goffi

    why would it not be ? It's a client thing to make is user friendly

  68. edhelas

    http://www.t-dose.org/node/1063

  69. Holger

    Guus: Got the account email, thank you.

  70. Guus

    Holger: yw

  71. zinid

    isn't it easier to add <subscribe/> to MUC XEP instead of writing this MIX stuff?

  72. Guus

    It's certainly easier to write that, yes.

  73. Ge0rG

    Zash: tell us about minimix

  74. zinid

    Guus: so what's the problem? we can create those pusbus nodes inside a muc room

  75. zinid

    *pubsub :)

  76. moparisthebest

    ha I like pusbus better we should rename it

  77. zinid

    no objection

  78. Ge0rG

    I vote for pup-soup. 🐶🍲

  79. zinid

    also, assuming a user's server should know about MIX is a bad idea

  80. zinid

    another issue: > To achieve this, the client will query the user's own MAM archive using Message Archive Management (XEP-0313) [3], with the query filtered by the channel JID. > The only exception to this is when a user wishes to access message history in the channel prior to when the user joined the channel. To achieve this, the client will use MAM to retrieve message history directly from the MAM Archive of the MUX channel.

  81. zinid

    why a client cannot request mam archive from MIX channel right away?

  82. Ge0rG

    zinid: you should read up the previous discussions on standards@

  83. zinid

    which ones?

  84. Ge0rG

    the ones on MIX

  85. zinid

    there are tons of them

  86. Ge0rG

    Yes.

  87. zinid

    no thanks

  88. zinid

    I just did some search and didn't find any relevant info inside those discussions

  89. Alex

    memberbot is up and council for the board & council election

  90. jonasw

    Alex, thanks :)

  91. jonasw

    even though I assume it’s up and running, and not up and council :-)

  92. jonasw

    Alex, you included ralphms application despite it being late?

  93. jonasw

    (I’m not saying that we should not, but I think that some people have strong opinions on that)

  94. Alex

    yes I did

  95. Arc

    """Your problem is so terrible, I worry that, if I help you, I risk drawing the attention of whatever god of technology inflicted it on you."""

  96. jonasw

    :D

  97. Guus

    please don't walk in the sea.

  98. Guus

    into*

  99. Arc

    I LOL'd so hard reading that

  100. Zash

    Arc: That sounds like how helping people with old code makes you the maintainer.

  101. Guus

    Yeah, I get funny looks whenever reading his book

  102. jonasw

    The What-If book?

  103. Guus

    yeah

  104. Guus

    he might have more, unsure :)

  105. Zash

    Is that from What-If?

  106. jonasw

    no

  107. Guus

    xkcd

  108. jonasw

    it’s todays xkcd, Zash

  109. Zash

    Bunneh: xkcd

  110. Bunneh

    Thermostat https://xkcd.com/1912/

    Thermostat
    Thermostat
  111. jonasw

    Guus, I learnt in that book that I both want and totally not want a wall with the periodic table of elements :-)

  112. Guus

    I learnt from that book that I snort when snickering

  113. jonasw

    :D

  114. Arc

    im digging through my old code with mod_xmpp now, doing a major overhaul. its actually not terrible, just a bit ... spaghettified

  115. Arc

    i was actually closing in on a "good" solution but was one step away. instead of chopping up stanzas by the outside, i needed to chop them from the inside.

  116. jonasw

    what’s mod_xmpp?

  117. Guus

    "not terrible"

  118. Guus

    (just a bit spaghettified)

  119. Arc

    jonasw: it started as an Apache module to do XMPP over Websockets proxying to an xmpp server over C2S

  120. jonasw

    ah

  121. Arc

    Guus: i know I'm not unique in, when you work on code you havent touched by a year, you start hating your younger self

  122. Guus

    Arc: I totally bypass that by simply forgetting that I touched that code. As long as I don't use git blame, I can hate the random anonymous dev that did that terrible thing.

  123. Arc

    heh

  124. Guus

    Or, as one of my code-workers used to say: "I must've been drunk."

  125. jonasw

    Guus, +1 for "I must’ve been drunk."

  126. Arc

    or high.

  127. jonasw

    it’s extra funny since I don’t drink alcohol or so at all.

  128. Guus

    ... that you remember ...

  129. jonasw

    right.

  130. jonasw

    I also find that this varies greatly by language.

  131. Arc

    so, I'm looking at chopping up the XML stream by the inner part of the stanza, and parsing/serializing the outer part of the stanza. does that seem sane?

  132. jonasw

    I’m not sure what "chopping" means and why you need to do it

  133. jonasw

    (also, jdev@ maybe)

  134. Arc

    obviously open to edge cases, but I'm still using expat with this, and I'll move to libexi in a later version meaning it'll do a full parse->serialize during the process

  135. Arc

    jonasw: outer part, eg, <message from="" to="" id="">, vs inside part eg <body xmlns=""> etc

  136. jonasw

    I guessed that much -- what kind of chopping?

  137. Arc

    oh man, i havent been in jdev in forever

  138. jonasw

    (I may be missing context on how websockets work)

  139. Arc

    jonasw: its semi related to websockets.. chopping actually in regard to APR bucket brigades. so Apache operates streams as a linked list of buffers. when the expat parser hits certain markers, I record the point, and then can recreate/remove part of the stream, then grab the buffer until a certain termination point, passing that along verbatim

  140. jonasw

    okay, what

  141. Arc

    one of the bugs in mod_xmpp has always been that, contrary to spec, it hasn't included xmlns="jabber:client" with every stanza

  142. jonasw

    huh, is that a MUST or SHOULD?

  143. Guus

    didn't you get all that from 'spaghettified code', jonasw? ;)

  144. Guus

    namespace can be either on the stream or on each stanza, iirc

  145. jonasw

    my personal style of test-driven development (which is much less strict than what people probably usually advocate, I don’t know, I’m self-taught) has stopped me from writing spaghettified code

  146. Arc

    its a MUST because XMPP over WebSockets isn't within a root <stream:stream> element anymore. each stanza is a whole and complete XML document

  147. jonasw

    I think so too, Guus

  148. jonasw

    Arc, okay

  149. jonasw

    I was scared there for a second

  150. Guus

    https://xmpp.org/rfcs/rfc6120.html#streams-ns-content

  151. Arc

    most javascript code implementing xmpp over websockets doesn't test for the xmlns="jabber:client"

  152. Zash

    I'm not so sure that doing that wrapping thing was the best idea

  153. jonasw

    because I’m pretty sure that aioxmpp doesn’t include xmlns="..." on each stanza

  154. Guus

    ah, websockets

  155. jonasw

    Arc, most javascript code doesn’t give a f..thing about namespaces.

  156. jonasw

    and if you try to do, you run into all kinds of funny browser bugs.

  157. Guus

    it's annoying to have to move stanzas from a c2s stream to a s2s stream, for the difference in namespace

  158. jonasw

    Guus, I agree that using different namespaces there was a weird choice

  159. Zash

    All that because nobody wanted to write a SAX parser for browsers

  160. jonasw

    asm.js + libxml2?

  161. jonasw

    or libexpat

  162. Guus off to watch some House of Cards, for board election winning tips.

  163. Zash

    jonasw: but now it's set in stone forever and ever

  164. jonasw

    Zash, until XMPP 2.0 comes around or so...

  165. Zash

    maybe a "just plain XMPP over websockets, no fancy framing" spec?

  166. jonasw

    wouldn’t that break due to lack of SAX parsers?

  167. Arc

    i dont think the namespace actually breaks anything in javascript

  168. Zash

    jonasw: I mean as a separate thing

  169. jonasw

    more separate things?

  170. Arc

    but yea in WebSockets the second example in guus's URL, "prefix-free canonicalization", is what websockets stanzas SHOULD look like.

  171. Zash

    The current XMPP-over-WS RFC is basically "XMPP over WS for the Web"

  172. jonasw

    Zash, is there a use-case for WB not over the Web?

  173. jonasw

    Zash, is there a use-case for WB not for the Web?

  174. jonasw

    Zash, is there a use-case for WebSockets not for the Web?

  175. jonasw

    damnit

  176. Zash

    The dark future where only port 443 can be used?

  177. jonasw

    Zash, don’t support that dark future

  178. jonasw

    (and don’t walk into the sea)

  179. Zash

    And where you can't tunnel whatever over TLS on 443?

  180. Zash

    I don't

  181. Zash

    It's of course inevitable tho :(

  182. jonasw

    Zash, stop saying that

  183. jonasw

    you make me sad

  184. jonasw

    I don’t want to be sad.

  185. Arc

    in that particular example, xmpp connect might actually be faster than starttls

  186. jonasw

    Arc, even faster than XEP-0386?

  187. Arc

    jonasw: no, because you have the HTTP upgrade handshake

  188. jonasw

    wait that number is wrong

  189. Arc

    i know what you mean tho

  190. Arc

    and i love it.

  191. jonasw

    XEP-0368 (SRV records for XMPP over TLS)

  192. Arc

    I mean its a bit rough around the edges, i wish it wasnt needed, that xmpp were to default over TLS

  193. Arc

    zash is right tho, there's already networks that block all communication that's not on an "approved" protocol on its "accepted" port

  194. Arc

    HTTPS must be accepted.

  195. jonasw

    Arc, I’m not sure pushing that fight further up in the ISO/OSI layers will help

  196. Arc

    we've had some form of XMPP over HTTP proxy to deal with those kinds of networks

  197. Zash

    It's all just moving negotiation around the layers

  198. jonasw

    Arc, at some point, either breaking of TLS by those firewalls will becmoe standard or other means of guessing the type of traffic will be used.

  199. Arc

    at some point those firewalls will also include ALPN sniffing

  200. Zash

    Arc: Implying that they don't already?

  201. Arc

    point

  202. Arc

    i still have a WRT54GL router setup exclusively to proxy all IP traffic over HTTPS because most school districts have only 80 and 443 open, and block *MOST* websites on both. Luckily I own an IP address not currently included in any blocking blacklist

  203. Arc

    the DC Public Library system was setup similarly. good luck getting a video conference to work

  204. Arc

    it was otherwise impossible to do after school programming

  205. moparisthebest

    > jonasw‎: Zash, is there a use-case for WebSockets not for the Web?

  206. moparisthebest

    unfortunately yes: https://github.com/moparisthebest/WebSocketSocket

  207. moparisthebest

    for a brief period my work mitm'd all TLS and I had to tunnel TLS over websocket over mitm'd TLS

  208. moparisthebest

    luckily I did not write a XEP :P

  209. Arc

    Guus: also man, given the attendance record for the last board, i dont think you have much to worry about. really.

  210. Arc

    we clearly need new, energized blood

  211. Arc

    (even if its me that's out the door for next year)

  212. Guus

    I'm not worried either way, just wanted to make a House of Cards reference.

  213. Arc

    heh

  214. Guus

    because i think that's hillarious.

  215. Guus

    there.

  216. Arc

    well, no HoC reference is complete without accusations of gay rape

  217. moparisthebest

    might be nice to link to https://wiki.xmpp.org/web/Board_and_Council_Elections_2017 in topic

  218. Arc

    I pledge that I only sexually harass men when asked to by women who feel harassed by those same men. :-P

  219. Arc

    PyCon before the CoC was a very dark time.

  220. Arc

    hey Kev how solid is MIX at this point?

  221. moparisthebest

    pretty solid if you mean too dense to read :P

  222. Arc

    is there a reference implementation yet?

  223. Arc

    moparisthebest: did you ever see or read my presentation on xmpp microservices?

  224. zinid

    Arc: I read your EXI xep

  225. moparisthebest

    no, got a link to read?

  226. Arc

    zinid: that's impossible, i havent written it yet

  227. zinid

    moparisthebest: just read his EXI xep to get the idea ;)

  228. Arc

    the only exi xep that exists now is garbage

  229. Guus

    Arc: there's a MIX implementation for Openfire being worked on by Surevine - not sure in what state of completeness it is.

  230. moparisthebest

    I'm not positive EXI has a purpose outside tiny embedded devices really

  231. Arc

    the XEP as it stands does not sync the grammar to be used, it relies on the server taking several schemas and compiling a grammar itself, which depending on implementation may or may not match

  232. moparisthebest

    like mobile phones handle XML just fine

  233. Arc

    moparisthebest: yes, it absolutely does. the problem is imlementation

  234. Zash

    Or lack thereof?

  235. moparisthebest

    why would a phone app want to implement something other than XML though is the question?

  236. Arc

    of course it does. mobile devices can handle XML, but the overhead is immense. to say "hi" takes 200+ bytes

  237. zinid

    Arc: ah, you're not the author, ok

  238. moparisthebest

    certainly no memory or speed reasons anymore

  239. moparisthebest

    in a world where node.js is a thing, what's 200+ bytes?

  240. moparisthebest

    a hello world webpage is ~4mb

  241. Zash

    Bunneh: do #'<message to="arc@example.com" type="chat"><body>hi</body></message>'

  242. Bunneh

    Zash: 67

  243. zinid

    Arc: but EXI is a terrible way to fix this issue, at least current XEP is pure shit

  244. zinid

    abstracting from XML is a way to go

  245. Arc

    zinid: yes, it absolutely is. which is why i dont want to fix it. i want to write a new one

  246. Zash

    zinid: Separation of the data and its encoding?

  247. zinid

    Zash: yeah...

  248. Arc

    what I'm missing is a manner for the client to transmit the grammar to the server when the server doesn't already have it. there isnt a standard encoding for this, and it must be implementation agnostic

  249. moparisthebest

    and sounds like a nightmare security-wise, probably

  250. Arc

    i dont think so. why would it?

  251. zinid

    why would you need to transfer schemas? do you know any asn.1-base implementation transfering asn.1 definitions?

  252. jonasw

    I wonder if EXI grammars can be used to create exponential costs (they can contain regexes, right?)

  253. Arc

    the current XEP does something, security wise, awful in having the server fetch grammar files from arbitrary HTTP URLs. that's begging to be used as a DDoS amplication attack

  254. moparisthebest

    I vaguely recall discussing this before, I think you said the server would cache these or whatever

  255. moparisthebest

    or does the client transfer all it's going to use every session?

  256. moparisthebest

    because then you don't save bytes

  257. Flow

    Arc, is it so important to do that? I've heard that EXI works reasonably efficient even when not used in schema-informed mode

  258. jonasw

    moparisthebest, I think the grammars would be keyed by a cryptographic hash sum

  259. Zash

    What if each party says which namespaces they have schemas of, and then you fall back to some inefficient generic encoding for everything not in the union of known schemasq

  260. Zash

    s/q$/?/g

  261. Arc

    Flow: reasonably is relative. you have to transmit all your string tables

  262. Arc

    Zash: aka non-strict encoding.

  263. moparisthebest

    so can evil client fill up that cache and/or boot out other in-use ones?

  264. Arc

    moparisthebest: SHA256 should reasonably cover this, unless you think SHA256 is weak. and the server could have a finite cache.

  265. moparisthebest

    my point is EXI seems possibly useful in a iot network of trusted super low resource clients or whatever

  266. moparisthebest

    and useless for desktop or modern phones

  267. moparisthebest

    (which are now, indistinguishable resource-wise, right?)

  268. Arc

    you cannot trust iot devices. thats where terrible iot security comes from.

  269. moparisthebest

    so can't I just evict your useful grammar files by sending you a load of useless ones?

  270. Flow

    moparisthebest, mobile devices may have similar computing power than desktops, but they have other constraints too

  271. moparisthebest

    even if you do everything correctly and sha256 is secure

  272. Zash

    LRU cache?

  273. Arc

    yea

  274. jonasw

    moparisthebest, use a very limited per-account cache, plus a global cache for shasums which are used by multiple accounts.

  275. jonasw

    in the worst case the global cache is filled with garbage, but the account-local caches for well-behaving accounts will still work as they should.

  276. moparisthebest

    jonasw, so then I just connect 2 evil clients and still evict from global?

  277. Zash

    moparisthebest: Think of EXI as better compression that is safe from https://blog.thijsalkema.de/blog/2014/08/07/https-attacks-and-xmpp-2-crime-and-breach/

  278. Flow

    Zash, are you 100% sure that EXI isn't vulnerable to similar form of attacks?

  279. moparisthebest

    ah yea I think we talked about that too and weren't entirely sure it was safe in all modes

  280. Arc

    regex btw is an argument against transmitting schemas, its primarily used when defining constrained character sets, and the regex in question isn't a full regex implementation but rather a list of character ranges

  281. Zash

    Flow: I'm not 100% sure of anything

  282. Zash

    s/safe/safer/ probably

  283. moparisthebest

    in fact I think Arc said it was vulnerable in some modes

  284. moparisthebest

    it's been awhile

  285. Arc

    moparisthebest: there's vulnerabilities in all XML libraries.

  286. Flow

    Zash, ok, let me rephrase: Do you expect that EXI is not vulnerable to similar things like CRIME/BREACH?

  287. moparisthebest

    Arc, sorry I meant vulnerable agaist crime-like compression attacks

  288. Zash

    Flow: I expect that kind of attack to not be effective against something that roughtly boils down to byte-packing of Enum-like fields

  289. Arc

    well EXI includes the option for including DEFLATE

  290. Arc

    there are 4 modes; bitpacked, byte aligned, pre-compression, and DEFLATE

  291. Arc

    the client chooses.

  292. Arc

    honestly ive found bitpacked to work the best in almost every case

  293. Arc

    in bitpacked mode there's no huffman table or similar to exploit. it doesn't compress text really at all, only XML structure.

  294. Arc

    if there's a potential attack on DEFLATE i'd be personally satisfied in including it in a Security section of the XEP with an advisement against using it.

  295. Zash

    If you make sure that any fields that an attacker can put stuff into are treated as text then it should be more resistant

  296. Arc

    *nod*

  297. Arc

    I believe I remember seeing a case of ascii-only 6-bit encoding, and UTF32, intermixed.

  298. Arc

    but that's about the only text compression you're going to find

  299. Zash

    5 bits should be enough for everyone! :)

  300. Arc

    until its not. :-P

  301. Arc

    insert unicode emoticon.

  302. moparisthebest

    I wonder size-wise how it compares to that xml->json thing, was that a xep?

  303. Flow

    Arc: It the wrong usage of DEFLATE that opens the side-channel. If the attacked endpoint performs a full flush, i.e. drops the dictionary, on every "channel" change, then it should be safe

  304. Arc

    Flow: interesting. well, that's something we could address. XML fragments are included, i honestly dont remember how compression was supported.

  305. Zash

    Flow: I think that should have had a protocol break

  306. Arc

    i remember i talked to sam a lot about using framing with it, and each stanza a full and complete xml document

  307. Flow

    Zash, why?

  308. Zash

    Flow: How does one end know that the other end is doing it Right?

  309. moparisthebest

    yep, no way to tell, no way for server configs to forbid insecure clients

  310. Arc

    the core of the argument for mobile comes down to this tho, the reason Google dropped XMPP support, according to one of the guys on the Hangouts team, was the massive bandwidth and processing overhead of XML vs binary.

  311. moparisthebest

    so XML was an engineering problem google couldn't tackle?

  312. Zash

    Arc: Is that why they pushed for HTTP/2 to be binary?

  313. moparisthebest

    because marketing sounds far more likely

  314. Arc

    moparisthebest: XML has a very high processing overhead, and bandwidth overhead, and those are things that can't be just "tackled"

  315. Arc

    remember when i accidentally crashed gtalk?

  316. Zash

    Text based protocols do have other valuable properties tho

  317. moparisthebest

    google engineers didn't know about EXI or some other encoding?

  318. Flow

    Zash, the other end being the other end of the stream or the xmpp communication?

  319. Zash

    Flow: Yes

  320. Zash

    :D

  321. moparisthebest

    I just find it hard to believe the reason wasn't "we want to lock people into our walled garden"

  322. Flow

    I'm not sure if in the c2s case, the client needs to know that the server also does full flushes on channel changes

  323. Flow

    Isn't it possibly sufficent if the client does the right thing?

  324. Arc

    that was due to an optimization in their UTF8 to protobufs handling. a shortcut was taken, the message i sent jumped their double null termination and propigated.

  325. Zash

    Flow: How do I, the server, know that you do the right thing if the protocol is identical?

  326. Flow

    Zash, you don't, but do you care as server if, for the example, the client authenticated your TLS cert?

  327. Arc

    𐑓𐑳𐑒 𐑿

  328. Arc

    that exact message was all it took.

  329. Zash

    Flow: just <method>zlib-but-better</method> is what I mean

  330. Zash

    -demoji Arc> 𐑓𐑳𐑒 𐑿

  331. Bunneh

    Zash: Arc> 𐑓𐑳𐑒 𐑿

  332. Zash

    Arc: hexdump?

  333. Arc

    not emoji zash. its shavian https://en.wikipedia.org/wiki/Shavian_alphabet

  334. Flow

    Zash, sure why not, but you could also do the better part as client with <method>zlib</method>

  335. Flow

    The client has the incentive to do the right thing, the server doesn't really care, that is what I mean :)

  336. Zash

    Flow: You think the server should you just let you shoot yourself in the foot? :)

  337. Arc

    it reads "F-U-K YEW" which is what I wrote in jdev several years ago, when gtalk was brand new, to a google dev who argued that there's "no difference between characters and bytes, thats why we use UTF8"

  338. jonasw

    what

  339. Flow

    Zash, the server will happily route my root password in <body/> to your JID, won't it?

  340. Arc

    whatever optimizations they used, the space there caused their parser to jump the terminating null plus the two "safety nulls" they had in the protobuff reader and cause every gtalk server processing it to crash

  341. Zash

    Flow: You think the server should just let old non-fixed versions shoot themselves in the foot?

  342. Arc

    google devs found it as the last message in the queue in every affected server, and they "decoded" the phonetic english

  343. jonasw

    "safety nulls"

  344. jonasw

    amazing

  345. Flow

    Zash, valid point. So you prosody implement zlib-but-better?

  346. Arc

    this was a long, long time ago, but yes. their "optimized" xml/utf8 decoder had two "safety nulls" to ensure that this wouldn't happen. they didnt expect 4-byte unicode

  347. Flow

    s/you/would?

  348. jonasw

    nobody does expect 4byte unicode (*glances at mysql*)

  349. Arc

    one of the guys from the gtalk team shared that bit with me a long time after it happened.

  350. Flow

    as mod_compression_safe ;)

  351. Zash

    Flow: Maybe, if it get's properly XEP'd, but no promises

  352. Arc

    anyway the biggest issue for EXI right now is *how* to communicate the grammar.

  353. Flow

    Wasn't there even a TLS compression extension for CRIME or something?

  354. Flow

    Arc, bytestreams?

  355. Flow

    Or what exactly is the issue? That there is no mechanism defined?

  356. Zash

    Does it really need to be communicated at all?

  357. Flow

    My question exactly

  358. Arc

    Flow: no, no XML schema or otherwise that Ive ever seen.

  359. Arc

    Flow: the gains for it are huge, especially for initial connection.

  360. Flow

    I'm sorry but I don't follow. It is not required to exchange grammar to the other endpoint for EXI do work, but it would improve things, right?

  361. Flow

    Then why not: 1. authenticate 2. upload grammer via base64 encoded stanzs. 3. activate exi 4. bind

  362. Flow

    Or is the grammar byte format not well defined?

  363. Arc

    Flow: because you don't want to require the client support text-mode XML. especially with IoT

  364. Flow

    Arc, ahh ok, Smack's EXI protype would always work on XML, so that is what I thought would everyone do

  365. jonasw

    Arc, 1. activate exi, 2. upload grammar via exi-encoded stanzas, 3. use grammar, […]?

  366. Flow

    What is the other thing besides text mode XML? binary XML?

  367. Arc

    Flow: https://www.w3.org/TR/exi/#informedGrammars

  368. edhelas

    regarding the Styling XEP proposal, XMPP is a "protocol", this means it has to stay in the backend on my app, telling it what is received and what to send, XMPP is NOT a protocol that enforce how my app should look like, with Markdow, if I want to display my messages without formating I'd have to remove manually all those ugly ~ and *

  369. Arc

    Flow: EXI uses a lot less code to implement. so yes, text XML vs EXI

  370. Flow

    Maybe I'm a bit inflexible, but I can't think how a XMPP client/library/server would work with pure non-XML EXI exclusively

  371. jonasw

    Flow, _xmppexi-server._tcp SRV :-)

  372. jonasw

    Flow, _xmppexi-client._tcp SRV :-)

  373. Flow

    It sure is possible

  374. Zash

    jonasw!

  375. jonasw

    Zash!

  376. Zash

    jonasw: I was just typing that

  377. Arc

    Flow: client connects and sends a EXI header, specifying the schemaId as sha256, if server doesn't support it it'll respond with a default EXI grammar specifying this, client sends a new header to transmit the grammar

  378. Arc

    it adds a handshake if its unsupported

  379. Arc

    the grammar can be informed by a schema but includes weights. I might be wrong, and i'd love to be wrong, but I am not aware of an implementation-independent way to specify weighted options in an EXI schema

  380. Arc

    the grammar is a tree

  381. Arc

    the tree is scoped by where you are, and the options available at each point. more common options use fewer bits, or even only one bit. eg, end element is commonly transmitted with the first bit

  382. Arc

    in non-strict encoding there are options at every step, even for elements which have no attributes, child elements, or content

  383. Arc

    tho that can be transmitted with a single bit, end-element, or "other"

  384. Arc

    ive skimmed a few other EXI libraries for other languages and they all represent this slightly differently.

  385. moparisthebest

    just, if _xmppexi-client._tcp becomes a thing make it '368 style direct-tls please :)

  386. Arc

    moparisthebest: you have my whole-hearted agreement there

  387. Arc

    I do not like the idea, tho, of having to invent a XML schema to represent the grammar. because that has to be documented, and it'd be complex.

  388. goffi

    edhelas: please post your remark on the @standard. The worst with the current proposal, is that you can't even know if you have to remove those ugly ~ and *

  389. moparisthebest

    You can choose

  390. moparisthebest

    It basically describes what most clients do anyway

  391. Zash

    Which "most clients"?

  392. moparisthebest

    Thunderbird, Gmail?, Hexchat, every IRC client I've *ever* used, people writing text from the beginning of writing text when nothing parsed that except people, gajim

  393. moparisthebest

    I'm missing a ton surely

  394. moparisthebest

    Point is, parsed or not, it's well understood by anyone reading it

  395. Zash

    If everyone understands it already, then do we need to do anything?

  396. moparisthebest

    Nope that's the beauty of it

  397. moparisthebest

    You don't have to do anything

  398. goffi

    would be fun to post "ls `date +%Y-%m-%d`-*.xml" in a shell@ MUC room with some of client using this XEP some others not using it.

  399. Zash

    goffi: I recently learned about `date -uI`

  400. goffi

    Zash: easier to remember :)

  401. Zash

    oui

  402. moparisthebest

    goffi: so highlight but keep characters?

  403. daniel

    moparisthebest: that's probably for the best

  404. SamWhited

    I think keeping the characters vs. hiding them is a client decision FWIW, but I really like keeping them (eg. use https://simplemde.com/ for a while, it's very pleasant)

  405. goffi

    let's add one more different way of rendering

  406. daniel

    I just changed my implementation to keep the characters bit display them with 50% opacity

  407. daniel

    *but

  408. moparisthebest

    Gajim keeps them

  409. moparisthebest

    Have to check others...

  410. daniel

    Yes I'm starting to think that keeping them is for the better. And maybe the xep should specify that (the characters have to be kept)

  411. SamWhited

    I would be happy to change that to say that you SHOULD keep them, I only didn't do that because I assumed people would complain if I did.

  412. daniel

    I guess you can never fully avoid false positives

  413. goffi

    daniel: yes, by properly marking when you use a rich syntax and when you are not

  414. daniel

    If we decide to keep them we should specify if the style should include the keyword

  415. daniel

    I opt for not

  416. daniel

    Because it looks better

  417. SamWhited

    daniel: I'm not sure what you mean, do you just want to make eg. the * bold but not the word?

  418. moparisthebest

    goffi: then rewrite all e2e xeps, carbons, and come up with some nightmare to check if the content sent 2 different ways matches in content meaning

  419. moparisthebest

    Or, keep. It. Simple.

  420. daniel

    SamWhited, if *bold* will render to <b>*bold*</b> or *<b>bold</b>*

  421. daniel

    and i think the later looks better

  422. edhelas

    :')

  423. Zash

    <b>*bold</b>*

  424. SamWhited

    daniel: Ah, right. Tentatively I *think* I agree with you.

  425. moparisthebest

    Good compromise

  426. edhelas

    we all agree that next to the type="markdown" content we will have an unformated classic <body> tag ?

  427. moparisthebest

    I think gajim bolds the asterisks too, not on it now though

  428. daniel

    especially if you then go and display the * with 60% opacity

  429. daniel

    moparisthebest, yes it does

  430. daniel

    this is just coming from my personal preference on what i think looks better. not what other clients do

  431. moparisthebest

    I think it actually doesn't matter but the xep should recommend

  432. SamWhited

    edhelas: no, I think that's exactly what many people here are trying to avoid

  433. edhelas

    sic…

  434. edhelas

    can I also invent my own markup for Post content published in Pubsub ? like in Microblog then ?

  435. edhelas

    something that is like Markdown but with my own personnal syntax

  436. goffi

    moparisthebest: no all e2e XEP, OTR and OMEMO, and we already complained about that before. OX is done the right way.

  437. edhelas

    then people can embed videos and centered texts, but without using XML

  438. goffi

    moparisthebest: and you'll have to rewrite RFCs if you don't want different contents.

  439. edhelas

    I'm half serious here, or you go full Markdown or you do nothing, because I don't think developpers with love to write their own parsers again

  440. goffi

    but at least if the XEP mention a MUST (and not a SHOULD) keep formatting characteres, I would be more OK, as I could safely just ignore it.

  441. moparisthebest

    goffi: so I'll just use the clients that implement ox then, oh wait that's none...

  442. daniel

    i can get on board with a MUST. i like strict XEPs anyway

  443. goffi

    daniel: with a MUST I don't see any issue right now (beside your pasted code being ugly on some client, but that's their choice)

  444. daniel

    just updated Conversations master in case someone wants to see how this looks like

  445. goffi

    actually forcing formatting characteres would not work with escaping. So this would mean removing escaping

  446. moparisthebest

    I think daniel already said that and I agree

  447. moparisthebest

    Removing escaping that is

  448. SamWhited

    Yah, removing escaping seems fine to me

  449. SamWhited

    I'll add that to my TODO list assuming no serious counterpoints are brought up in the list discussion

  450. goffi

    I've mentionned this in a new message, I'm done with standard@ flooding for today :)

  451. goffi

    or we are already tomorrow, I can flood again

  452. goffi

    oh*

  453. goffi

    (this joke only work in CET timezone)

  454. Arc

    SamWhited: your XEP looks good to me

  455. SamWhited

    Arc: thanks! Any oddities you find, things that aren't clear, etc. please let me know!