XSF Discussion - 2017-03-02

  1. Ge0rG

    Damn, you've got me. I type my gpg password rather often. I can look up the other things for mutt tomorrow if you are interested

  2. Zash

    I'd rather know how to not quit mutt by accident all the time

  3. Ge0rG

    Unbind the Q key

  4. Zash

    Whos brilliant idea was it to put quit and 'go back' on the same key anyways?

  5. Ge0rG

    Zash: it's a sensible idea in general. Except when you want to "leave" a limit filter

  6. SamWhited

    I have my password saved in a GPG'ed file; mutt unlocks GPG on start to get the password, which also keeps the GPG agent unlocked for 15 minutes or whatever, which works pretty well.

  7. Ge0rG

    Zash: for incoming mail, you can set pop_pass and imap_pass in imap, or even bind a key to a macro like "cimaps://user@domain:password@server/INBOX\n"

  8. Zash

    https://tools.ietf.org/html/rfc6778 and https://tools.ietf.org/html/rfc7017

  9. Ge0rG

    That's so meta.

  10. Zash


  11. Ge0rG

    Zash: you might want to tell the XSF why you are pasting all the URLs in here.

  12. Zash

    Ge0rG: I'm sleep-pasting URLs I think

  13. Ge0rG

    Zash: time to get coffee, then.

  14. Ge0rG

    I've had my first coffee of the day at 0430 local time.

  15. Zash

    Anyways, the IETF seems to have gone through the process of figuring out better ways to access mailing list archives, so I'm trying to nudge people towards looking at the work they did.

  16. Ge0rG

    Zash: I'm not sure how IMAP is going to help in that regard. It sounds to me like a mix of NNTP nostalgia and nerd cred.

  17. Ge0rG

    Zash: I'd like to have a feature where you can search the ML by affected XEPs. So a kind of tagging.

  18. Ge0rG

    And people write the craziest things into the Subject:, so you can't just /~s XEP-0123

  19. Ge0rG

    if we could add XEP-xxxx tags post-factum, it would be great.

  20. Zash

    Makes archives predating your subscription accessible.

  21. Ge0rG

    Zash: last time I needed that (and it was to correctly reply-to to a mail), I just downloaded the .mbox. I think that the number of people who care about that, outside the IETF, is small.

  22. Ge0rG

    Zash: and the set of people who fail to import an .mbox into their MUA, but manage to connect to an anon IMAP is probably very small.

  23. Zash

    The underlying point is to look at what a similar organization did about pretty much the same problem.

  24. Ge0rG

    Okay, I can buy into that

  25. Zash

    They did end up with a pretty nice search thingy.

  26. Ge0rG

    Zash: I hope you don't mean "connect with imap, use your MUA search" approach

  27. Zash

    Ge0rG: https://mailarchive.ietf.org/arch/

  28. Ge0rG

    Zash: it looks like a web MUA to me. I searched for "xmpp" and wasn't impressed with the results too much

  29. Ge0rG

    OTOH, it looks like a MUA.

  30. Ge0rG

    Oh Android. If you register your app as an Intent handler, older versions use "{handler_title}" as the display text, and newer versions use "Open with {handler_title}". I'm pretty sure I'm not the only one to find "Open with Add contact" a strange wording.

  31. Ge0rG isn't awake either, yet. Just misread the last members@ thread as "XSF Bored Meeting Minutes".

  32. jonasw

    ah, that ietf-mailarchive-thing is nice

  33. jonasw

    seen it a couple of times

  34. jonasw

    I’m starting the writeup of the XEP-115 (Entity Capabilities) replacement. I have a few questions: 1. I would like to acknowledge waqas work and the work of the authors of XEP-115. How do I do that appropriately? The XEP-Template doesn’t have an acknowledgements section, but seeing that XEP-115 (and others) have one, I assume that’s an appropriate way to do it. Correct? 2. In the examples I will need a namespace. Where will I source it from? Should I use a namespace under my own control and the editor will choose a different one when the XEP is accepted as experimental?

  35. Kev

    Is this a replacement of 115, or an update to 115?

  36. daniel

    jonasw: there is no formal way for acknowledgements. Most authors just dedicate an entire section to it

  37. jonasw

    Kev: replacement, you can probably work your way from http://logs.xmpp.org/xsf/2017-02-28/#19:49:01 upwards to see the discussion around that.

  38. Kev

    Just re-using 115 seems appropriate to me, you're not in need of drastically changing the protocol, are you?

  39. Kev

    (I note that other things like pubsub have dependencies on 115, so if you write a whole new XEP you're looking at patching a *lot* of XEPs to update those dependencies)

  40. daniel

    That's probably true

  41. jonasw

    interesting point, noone seems to have thought about that the other day

  42. jonasw

    a namespace bump for 115 would be less intrusive probably

  43. Kev

    A namespace bump, if needed, or maybe a backwards-compatible update (if possible) seem reasonable to me. But keep in mind it's not coffee-o'clock yet, and I don't even drink coffee.

  44. jonasw

    backwards-compatible won’t happen. the algorithm (and I’m not talking about sha1 or something) is broken and in need of fixing for eight years.

  45. Kev

    I'm not utterly convinced that means it can't happen (forwards-compatible can't happen, certainly), but I'm not convinced it can, either.

  46. jonasw

    i should probably announce coffee-o-clock now.

  47. jonasw

    in my opinion, xep 60 doesn’t have a dependency on 115, but on 30. it’s just worded badly.

  48. jonasw

    or rather, "in my reading" than "in my opinion"

  49. jonasw

    from the amount new work I’m doing for it, an update to 115 feels more appropriate than a new xep, too

  50. Flow

    Kev, Steve Kille: Would MIX be interested in an atomic CAS for PubSub. For example to race-free replace the subject/topic/... of a node. I'm considering writing a CAS add-on XEP for PubSub.

  51. jonasw

    what is CAS?

  52. Flow always wonders why there is no CAS for PubSub

  53. jonasw

    (I only know Computer Algebra System, which I assume you don’t mean)

  54. Flow

    jonasw: compare-and-swap

  55. jonasw


  56. jonasw

    makes sense.

  57. jonasw officially announces coffee-o-clock!

  58. jonasw

    (or rather, tea-o-clock)

  59. Ge0rG had two cups of coffee yet. Time to get a new one.

  60. jonasw

    Flow: I feel that CAS will be hard to implement server-side. when do two XML subtrees compare equal?

  61. Flow

    jonasw: by node id

  62. Flow

    err item id

  63. Tobias


  64. jonasw


  65. Tobias


  66. jonasw

    Flow: CAS would be useful for data storage in PEP nodes, too

  67. Flow

    jonasw: It would be useful everywhere where PubSub/PEP is used

  68. jonasw

    mostly everywhere :)

  69. jonasw

    but yes.

  70. Flow

    and where you want to avoid accidentially deleting existing data because of a race condition

  71. jonasw

    there are usecases where you add data instead of replacing by item id :)

  72. Tobias

    I wonder why 115 didn't just use Canonical XML standard for c14n of disco to later hash it https://www.w3.org/TR/2008/REC-xml-c14n11-20080502/

  73. jonasw

    Tobias: I was wondering about that, too, but I think canonical XML is strict with the relative ordering of elements

  74. jonasw

    also I‘m not sure how many xml libs support c14n; considering that there are *still* some in use which don’t do namespaces properly

  75. Tobias

    could be, yeah

  76. Tobias

    jonasw, you're aware of this thread, right? https://mail.jabber.org/pipermail/standards/2011-August/025011.html

  77. jonasw

    not yet

  78. Flow

    jonasw: which usecases are that?

  79. jonasw

    Flow: microblogging-ish :)

  80. Flow

    ahh right

  81. Tobias

    jonasw, it discusses a lot issues with current XEP-0115, that should be solved in a new version

  82. jonasw

    Tobias: thanks!

  83. jonasw

    I’m looking into it

  84. Flow

    jonasw: Also https://wiki.xmpp.org/web/XEP-Remarks/XEP-0115:_Entity_Capabilities

  85. jonasw

    I was also planning to ask standards@ for input when I have a first draft

  86. Tobias

    Flow, what? the IANA has two registries for hash names?

  87. Flow

    Tobias: Yep

  88. jonasw

    that’s a good point; the one we currently use doesn’t list sha3 for example

  89. Flow

    I discovered that when searching for a registry for ISR-SASL2

  90. Tobias

    Flow, einmal mit profis :P

  91. Flow

    Tobias: Hehe, to be fair, that could happen to the XSF too :)

  92. Tobias

    Flow, nah...we'll only ever have XEP-0300, which can be updated relatively easy

  93. Tobias

    i think IANA stuff requires lots of time and process

  94. Flow

    If someone knows if and whom we should tell about this within the IETF/IANA, then please do so/tell me.

  95. Flow

    Link Mauve: BTW, SASL2?

  96. jonasw

    does anyone know the rationale for querying a specific disco-node containing the hash in the verification procedure xep 115?

  97. Tobias

    jonasw, what exactly do you mean?

  98. jonasw

    example 3 here: https://xmpp.org/extensions/xep-0115.html#discover

  99. jonasw

    node='http://code.google.com/p/exodus#QgayPKawpkPSDYmwT/WM94uAlu0=' instead of simply querying without node. is the idea to avoid races with changing capabilities?

  100. jonasw

    hm, it mentions "backwards-compatibility"

  101. jonasw

    for avoiding races it seems helpful, why was it abandoned?

  102. jonasw

    (even though races wouldn’t be harmful here)

  103. Flow

    jonasw: so that you get the result of that very same hash?

  104. jonasw


  105. Flow

    that approach seems sensible to me

  106. Tobias

    could also help with server side caching i suppose

  107. jonasw

    Flow: I don’t like the approach though, from an implementers point of view

  108. Flow

    e.g. Smack also responds to the last 10 hashes

  109. Flow

    jonasw: I do like the approach from an implementers point of view

  110. Tobias

    Flow, you keep a history what sets of features the last 10 smack releases supported?

  111. Flow

    Tobias: No, disco features are dynamic, not tied to a smack release

  112. jonasw

    Flow: there is no harm in a race here, because if you get a race with an unknown hash (if you know the hash, you don’t care) you simply get the updated disco#info and discard the hash.

  113. Flow

    so the last 10 features of the connection

  114. Tobias

    that yoo, yeah

  115. Flow

    jonasw: true, no race here, but it helps with other things, like tobias said, server side caching, and I think it's the cleaner approach

  116. jonasw

    how does it help with server-side caching?

  117. Flow

    jonasw: The server can cache the response

  118. jonasw

    hm okay

  119. Flow

    and send it instead of forwarding the request to the queried client

  120. Tobias

    jonasw, the server doesn't need to forward the IQ to the to-JID if it knows the from-JID just wants the disco#info for a hash

  121. jonasw

    makes sense

  122. Tobias

    it could reply directly

  123. jonasw

    seems like using a different format for these nodes would be great though: '{ecaps2-namespace}#{hash-algo}.{hash-value}' or something along those lines to make it easily recognizable

  124. jonasw

    right now a server needs to track the 'node' exported in <{caps}c/> to know whether a disco-node is a caps hash

  125. jonasw

    *belongs to a caps hash

  126. jonasw

    is there an element I can use to link to another section in a XEP?

  127. jonasw

    except <link url='#anchor'/>

  128. dwd

    IANA has *no* registry for hash names. IANA has several protocol registries to cover parameters for hashes, some of these are strings.

  129. jonasw

    dwd: that makes sense and explains the odd titles for those registries.

  130. Flow

    like "Named Information Hash Algorithm Registry"

  131. dwd

    We co-opted one for our purposes in XEP-0300, but it's originally for PKIX, so it contains OIDs as well.

  132. dwd

    Maybe we should also allow urn:oid:2.16.840. for SHA-256?

  133. jonasw


  134. jonasw

    no no no no

  135. Tobias

    dwd, although that one hasn't been updated since 2000something

  136. jonasw

    oids are a mess.

  137. dwd

    jonasw, How can you say that? They're terribly convenient stable identifiers. Even if Surevine only has one OID arc (Isode has two - snazzy).

  138. jonasw

    ugh, the names in xep-0300 are longer than some base64-encoded hash values themselves…

  139. Tobias

    jonasw, what names?

  140. jonasw

    dwd: as long as you don’t need to parse them semantically, it’s fine probably, like urns

  141. jonasw

    Tobias: <var> <name>urn:xmpp:hash-function-text-names:md5</name> <desc>Support for the MD5 hashing algorithm</desc> <doc>XEP-0300</doc> </var>

  142. Tobias

    yeah...that's so people don't used md5 :P

  143. dwd

    jonasw, Oh, the feature names.

  144. Tobias


  145. jonasw

    well, close. >>> len(base64.b64encode(hashlib.sha256().digest()).decode("ascii")) 44 >>> len("urn:xmpp:hash-function-text-names:sha-256") 41

  146. dwd

    jonasw, Well, that's a reason to use SHA3-512, then.

  147. jonasw

    my python cannot into sha3

  148. jonasw

    hm, 3.6 can’t either…

  149. Tobias


  150. jonasw

    but that looks like a configuration problem; it also doesn’t have BLAKE2b512 which is available in 3.5 here

  151. mathieui

    jonasw, 3.6 can do sha3 just fine

  152. jonasw

    Tobias: did you mean <sad/>?

  153. jonasw

    mathieui: yes, it appears to be a problem with my python3.6.0a3 probably sourced from debian/experimental

  154. Tobias

    jonasw, nah..i mean the trumpish hashtag sad ;)

  155. Tobias

    jonasw, so 3.6 doesn't have blake2 but 3.5 has?

  156. jonasw

    Tobias: or rather xep-14 <x xmlns="jabber:x:tone">sad</x>? :>

  157. jonasw

    Tobias: as I said: it’s most likely an issue with my local setup, the documentation says it is there:

  158. jonasw


  159. mathieui

    Tobias, 3.6 has blake2 as well

  160. Tobias


  161. jonasw

    meh, short names for the functions in xep-0300 would be great

  162. jonasw

    or am I just missing those?

  163. dwd

    jonasw, The long names are only used in the disco#info, right?

  164. jonasw

    dwd: it apperas so

  165. dwd

    jonasw, The actual use in protocol are short names, like "md5".

  166. jonasw

    dwd: but there doesn’t seem to be a registry or source to refer to on which short name to use for which function.

  167. Tobias

    jonasw, table 1 has short hash function names

  168. jonasw

    for some, yes.

  169. Tobias

    see the sentence before the table

  170. jonasw

    it is lacking sha3-{224,384} for example

  171. jonasw

    even including that sentence

  172. Tobias

    well yeah..didn't see much sense in those intermediate values

  173. jonasw

    fair point

  174. jonasw

    re-using 0300 makes a lot of sense

  175. Tobias

    the standard should probably be 256bit ones, and if you need more security, might as well go to 512 bit then

  176. jonasw

    hm, would making new hash functions mandatory trigger a bump on the <hash/> element…?

  177. jonasw

    that sounds like a *lot* of fallout.

  178. Flow

    jonasw: why should it trigger a (namespace?) bump?

  179. jonasw

    Flow: I don’t know. I’m asking.

  180. Guus

    *couch*Flow logo*couch*

  181. jonasw

    Kev: out of curiousity, what software are you talking about in your mail from 09:57+01:00?

  182. Tobias

    i just assumed that mail was some weird welsh humor :)

  183. dwd

    jonasw, I suspect it's mailman...

  184. Guus

    as we're all here: Does any more need to be discussed regarding https://github.com/xsf/xmpp.org/pull/269 ?

  185. Guus

    or rather: my merging of it?

  186. Tobias

    dwd, a new version of mailmain you mean?

  187. Tobias

    or the current mailman?

  188. dwd

    Tobias, No, I think it's just whatever we're using now. I suspect there might - might - be sarcasm at play here.

  189. jonasw

    Guus: FWIW, github has a review feature, and it may make sense to have one or two eyes confirm that they took a close look on the changes, possibly leaving comments.

  190. Tobias

    dwd, never seen him use that before though

  191. dwd

    Tobias, No, it's unusual in those who are cursed by not being English.

  192. Tobias

    dwd, you misspelled 'blessed' there

  193. jonasw

    I had to change my editors dictionary to en_US (from en_GB) to write XEPs :<

  194. dwd

    What? Why?

  195. jonasw

    because XEP-0134 (or -0001?) says so.

  196. dwd

    Sounds like a candidate for a PR, then. :-)

  197. jonasw


  198. Guus

    jonasw: I don't disagree, but as far as I know, that feature is not used by XSF. We could, sure. I don't feel that there's a need for it here (the consequences of missing something in a PR review are very unlikely to be catastrophic for our website, and I prefer a continuous release cycle), but I accept that others think differently.

  199. jonasw

    Guus: it’s really low-entrance-barrier though (if you’re a github user), and I don’t mean that it should be *mandatory*.

  200. Guus

    jonasw: I'm using it for other projects. Not knowing when to use it appears to be my problem. :) I thought your PR was fine.

  201. jonasw

    have you checked I didn’t slip in a try: shutil.rmtree("/") except: pass in? :)

  202. Guus

    I am assuming that you thought so, because you PR'ed it in the first place.

  203. jonasw

    I’m new in the XSF, my word shouldn’t count a thing when I add code to servers.

  204. Guus

    Oh, you could have slipped in things. I recognized your name, I glanced at the code, I ran it locally, it had the desired effect and did not delete my root partition. That combined made merging the PR an acceptible risk for me.

  205. jonasw


  206. jonasw

    I’m just saying that I completely understand the point of people asking for thorough reviews. I would do the same if it was my infrastructure.

  207. Guus

    Who am I to object to thorough reviews?

  208. Guus

    I think mine was thorough enough by my standards, but I am fully aware that others have different standards.

  209. Kev

    I think there's a significant difference between 'updating text on the website', which I'm fine with people generally having access to do. And "running code on our servers", for which most people don't have rights.

  210. Tobias

    Guus, i agree though that i should probably have left a note in the PR that I was planning to review it soonish

  211. Kev

    Running code that people thought was fine, but wasn't sensibly vetted caused us to not take part in GSoC last year, and huge amounts of wasted effort for me in the process, not to mention the downtime of the server so the XSF couldn't fulfil its primary purpose for a day.

  212. dwd

    FWIW, the pelicanconf.py file (the only one, as I understand it, that is executed on the server) looks perfectly safe to me and adequately simple.

  213. dwd

    It also looks clearly bounded, in as much as I can solve the halting problem in my head.

  214. Tobias

    dwd, as far as I know https://github.com/xsf/xmpp.org/blob/master/buildCompleteWebsite.sh is run to build the whole website on the server

  215. Kev

    I think the more crises someone has been through with production servers, the less blazé they get about deployment :)

  216. Tobias

    because pelican has very limited capabilities

  217. Kev

    Anyway, I don't object to the PR based on the description, I just don't want any code deployed on XSF servers that hasn't been reviewed by iteam.

  218. jonasw

    Tobias: it can do anything python can if you put it in the pelicanconf :>

  219. Guus

    Kev: I've been a production herding developer, professionally, for 10+ years.

  220. Kev

    Guus: And how many times has pushing something without checking it caused a day's worth of downtime for you? ;)

  221. Tobias

    jonasw, probably

  222. Guus

    including websites that have significant amount of views (millions, monthly)

  223. Guus

    Kev: I did check.

  224. dwd

    Kev, I think you may mean blasé, rather than blazé.

  225. Kev

    dwd: I very much do.

  226. dwd

    Although there's an argument for either.

  227. Kev

    Guus: Then I have no objection. Your original comment didn't mention that you'd reviewed the code, just run it locally.

  228. dwd

    jonasw, So, not threading, then? :-)

  229. Kev

    Well, I still have an objection in principle, because I think the server admins should get to review the code too, but I'm happy in this instance if you've reviewed the code.

  230. Guus

    Kev: I'm pretty sure I did not review it up to your standards. I'm also not worried by that.

  231. jonasw

    dwd: depends on the specific python implementation and the specific task. Python can very much thread in the sense that C extensions which are called from python code from different threads may in fact run in parallel. It is just pure python code which, on CPython at least, isn’t run in parallel. :)

  232. dwd

    Kev, This is build-time code, incidentally, not runtime code. So I'd hold it to lower standards.

  233. Kev

    jonasw: "Python can totally thread, as long as you code in C instead of Python"? :)

  234. dwd

    jonasw, Yeah, I'm only too aware...

  235. jonasw

    Kev: pretty much

  236. Kev

    dwd: When it's run on the server, I'm not sure the standards need to be much lower. If it's malicious, same effect, if it manages to resource-starve and bring down the server, same effect. There are some runtime cases (resource-heavy, but not resource-starving) that don't apply, but the standard's still pretty high.

  237. jonasw

    actually, this is why in the organisations I use pelican, the build system and the contents are separate repositories. The build system repository has strict review requirements, content lesser so.

  238. jonasw

    (although, fun fact: pelican lets you write to arbitrary files from the content files alone :-))

  239. jonasw

    (well, the current master branch doesn’t anymore)

  240. Tobias

    jonasw, templates probably still can though, right?

  241. jonasw

    not sure about that, but I don’t consider templates content.

  242. Kev

    Anyway, my opinion isn't going to matter for long. My new games PC has just arrived, and Cath is going to kill me as soon as she gets home and sees the den.

  243. Tobias

    heh :)

  244. Guus

    You have time for a games PC? *envy*

  245. Kev

    Sure. It just sits there, it doesn't need much time.

  246. Kev

    Now playing games, that would take more time...

  247. jonasw

    I would like to re-ask my question now that more people are active. When writing a new XEP, in the examples and specification I will need a namespace. Where will I source it from? Should I use a namespace under my own control and the editor will choose a different one when the XEP is accepted as experimental?

  248. Guus

    which of both is what will get you killed later today?

  249. Kev

    jonasw: It's easiest for the Editors if you use an appropriate NS from the start, although technically IIRC the Editors should pick one.

  250. jonasw


  251. Kev

    Stripping out your NS to replace it with an xmpp one at publication time is mostly busy-work.

  252. Kev

    And while the other Editors are much less lazy than me, stil ... :)

  253. jonasw


  254. jonasw

    just wanted to make sure that I don’t overstep any boundaries by suggesting a namespace from the xmpp-urn-namespace

  255. Kev

    jonasw: It's slightly tweaking the process, but it's the sensible thing to do, and what everyone else does.

  256. Kev

    Guus: The mess, and that I'm not intending getting rid of my old games PC, but running both in parallel both run the risk of death-by-spouse.

  257. Guus

    Kev: in which case, I am glad I had the chance to meet you in person at FOSDEM, before your premature death.

  258. jonasw

    is there any precendent to form arbitrary (i.e. entity controlled) disco#info nodes from an urn:xmpp:-namespace? so for http://… namespaces it’s obvious to use # as a separator, is there any precedent what to use with urn:xmpp:-namespaces?

  259. Kev

    I'm afraid I'm too stupid to understand the question.

  260. Tobias

    jonasw, so you want to have dynamic namespaces, not previously defined in a XEP or registry?

  261. jonasw

    not namespaces, but disco#info node names

  262. jonasw

    nah, I’m too stupid to formulate it clearly. see in https://xmpp.org/extensions/xep-0115.html#discover <query xmlns='http://jabber.org/protocol/disco#info' node='http://code.google.com/p/exodus#QgayPKawpkPSDYmwT/WM94uAlu0='/> the node there is composed of a URL base and a hash value.

  263. jonasw

    I don’t see the point of using some client-provided string as a prefix so I would like to use the namespace of the XEP as prefix. what kind of separator makes sense between the prefix and the hash info? Is there a precedent for that?

  264. jonasw

    ah yes, it appears so

  265. jonasw

    xep 290 also uses #

  266. arc

    the argument that won me over on not allowing clients to dictate their resource was that of distributed hosting routing

  267. Tobias

    you mean clustering?

  268. arc

    sure, whatever term you want to have for a @server hosted by multiple servers. and sorry i completely misread the conversation above, so that statement was kinda out of the blue

  269. Ge0rG

    I'm still not convinced of that clustering use case. "Google does it this way" doesn't cut it for me.

  270. arc

    Ge0rG: we're going to need it for IoT

  271. Kev

    Ge0rG: Well, I guess it'd be interesting if you could explain how you solved it in your clustered server, to persuade the other clustered server vendors that it's easy?

  272. Ge0rG

    Kev: wait, let me fire up a bunch of dockers.

  273. arc

    right now prosody can effectively handle 40k concurrent users on an average AWS instance last i ran the brute force test. in order to scale to the size that some of these IoT manufacturers want you need multiple servers, ideally geographically distributed

  274. Ge0rG

    arc: what about running different per-region domains?

  275. arc

    the last sit-down I had with an IoT manufacturer they said 10m units is what they consider base level, and any solution they consider should be able to scale to ten times that

  276. Ge0rG

    are there any xmpp installations handling north of 1m connections? I only remember WhatsApp's we-are-awesome post in that regard.

  277. Tobias

    really wonder if all those IoT devices need permanent connections

  278. SamWhited

    per-region domains is changing the security model. Also, it means if I live in the US, but I travel to China, I'm still connecting to my server in the US (or whatever domain I registered on). We were talking about single domains, multiple-domains is a completely different thing.

  279. Ge0rG

    Tobias: of course they do!

  280. arc

    Tobias: for receiving input, yes. though they're not very active.

  281. SamWhited

    Ge0rG: I can't give exact figures (and don't know them anyways), but I'm pretty sure we (HipChat) are.

  282. SamWhited

    (and we also use the server-assigned-resource-part-for-routing solution, FWIW)

  283. MattJ

    FWIW Prosody's clustering will use the resource for internal routing purposes

  284. arc

    in one case a device wanted to send a "heartbeat" with 12 bytes of data every 6 seconds (1/10th of a minute)

  285. Ge0rG

    arc: that's a very intensive use case

  286. arc

    Ge0rG: yes, and each device having a retail price of around $15

  287. arc

    that's the future we face and have to plan for

  288. Kev

    I like that Arc has such a high opinion of our maths that he had to explain that 6 seconds was 1/10 of a minute :D

  289. arc

    Kev: sorry i haven't had my tea yet lol

  290. jonasw

    tea <3

  291. Guus

    I for one wonder how many seconds 2/10 of a minute is.

  292. arc

    I will readily admit that a 100m service blitzed my brain out. I mean, sure we can toss around big numbers like its nothing, but that's actually some significant engineering challenges.

  293. Kev

    arc: It undoubtedly is.

  294. arc

    at that rate you need dedicated S2S routers. and questions like where are the heartbeats routing to

  295. Ge0rG

    I could also imagine that a 100m IoT deployment has different requirements than a public chat service

  296. Ge0rG

    (and also probably different sysop challenges, where having a resource string as a debug tag is less useful)

  297. arc

    absolutely. from the EXI side those stanzas are extremely small. as long as the 12 bytes of data are encoded in int or float attributes within their custom schema, the whole stanza could be around 16 bytes. and since the devices will be communicating with a finite number of other devices, mostly on the same LAN..

  298. arc

    my recommendation was embed their XMPP server in their 802.15.4 to wifi gateway module, to keep a majority of the traffic local and reduce their service end traffic as a first point. which i think is what they're doing

  299. MattJ

    Ge0rG, client-provided debug tags aren't guaranteed to be unique, I'm really unconvinced by your argument

  300. SamWhited

    I've also come to the conclusion that agreeing to compromise on that basis was a mistake… if you were using the resource part as a debug tag you were using a quick hack; if that's a thing we want, we need a real solution, we don't need to make a part of the JID more complicated just so someone can see sometihng in existing logs.

  301. SamWhited

    Adding stuff to the JID that isn't related to routing is changing the purpose of JIDs, and that feels like a bad idea.

  302. Ge0rG

    MattJ: a properly implemented client can provide sufficient uniqueness.

  303. MattJ

    Ge0rG, you're not a server developer, clearly :)

  304. SamWhited

    As a general rule of thumb I don't think we should ever have to rely on a "properly implemented" client.

  305. MattJ


  306. arc

    SamWhited: from the EXI side it doesn't matter. the entire JID is one string in the string table. i think having a human readable (aka designed for the UI) resource after the # makes some sense. though, that could also be done through pep

  307. Ge0rG

    MattJ: but I know a little bit about client development

  308. jonasw

    at this point I tend to agree with SamWhited. for debugging, there really should be something else, like an additional optional stream header which can be used for debugging, or a stream feature to attach a debug identifier to a stream or use <identity/> as soon as it’s available.

  309. MattJ

    Ge0rG, it's a nice idea, for you, with your client. But in the real world, on a real server, we can't depend on every client being Yaxim

  310. arc

    wouldn't this make sense to attach to PEP?

  311. MattJ

    I totally get why you want a debug tag, and let's do that. But I think it's separate to the resource

  312. SamWhited

    Or just some form of fingerprint the server constructs (so that the client doesn't have to do anythihng), eg. maybe it queries the client for its disco#info, and then hashes that along with the JID and any other info it can get and uses that to track sessions

  313. MattJ

    arc, no, because PEP is per user, not per client

  314. arc

    MattJ: couldn't the PEP .. sorry still early .. list a resource to human readable lookup?

  315. jonasw

    SamWhited: for a single session, a server can just roll a random number.

  316. Ge0rG

    MattJ: let me rephrase your suggestion: let's create a nice perfect future debug tag sometime in the remote future, and remove the existing and working debug tag right now.

  317. SamWhited

    jonasw: ah, yah, I guess this is about tracking clients, not sessions. oops.

  318. SamWhited

    The existing and working debug tag that breaks more critical parts of the system and makes everything more complicated.

  319. jonasw

    for clients use <identity/> as soon as its available and log it to associate the identity with the session nonce in the logs.

  320. jonasw

    identity + bare jid probably

  321. SamWhited

    And requires that clients do a specific thing which they may or may not actually do.

  322. MattJ

    Ge0rG, given that you're currently the only person I've seen suggesting that the resource string can and should be used this way, I don't think we're anywhere near your ideal being reality either

  323. MattJ

    i.e. other clients don't use the resource this way, you do. You'll update to use the debug tag, they won't

  324. SamWhited

    I remember at summit people complained that identity couldn't be used for this, but I don't remember why? What jonasw suggested sounds sensible, and works today.

  325. jonasw

    I see that the resource is *currently* a nice way to track a client in debug logs; but BIND 2.0 won’t be there tomorrow. There’s plenty of time for server devs to adapt. This could easily be part of the UX considerations for sysops in BIND 2.0

  326. jonasw


  327. MattJ

    I'd be fine (and glad) to include some kind of unique client identifier in bind2

  328. Ge0rG

    MattJ: I don't know how many sysops of public servers are active in this MUC

  329. jonasw

    or even include a "debug identifier" in Bind 2.0 which is never ever exposed to anything but server logs. although I think a stream header would be nicer because it allows tracking even before authentication succeeded.

  330. jonasw

    ha, MattJ beat me to it

  331. MattJ

    and with bind1 clients, use their provided resource as a cookie, and then use something else for the actual resource

  332. Zash

    What is it with you and writing lots of text while I'm out on a walk?

  333. MattJ

    (sorry, cookie == debug tag in my mind)

  334. jonasw

    MattJ: makes sense

  335. SamWhited nods

  336. jonasw

    sounds like a very useful way forward

  337. MattJ

    Zash, you should take your phone, to make sure you never miss a message!

  338. Zash

    I did, for photos of all the snsow

  339. Zash

    I did, for photos of all the snow

  340. Ge0rG

    MattJ: I want to be able to easily grep my logs for certain things, and to get all traffic exchanged with a given client instance (including re-auth and 0198 resumption)

  341. jonasw

    (this discussion also pins me to a chair in a waiting room where I wanted to leave 20 minutes ago, but whatever)

  342. arc

    phone? he should have always wear Glass so this room is constantly flowing above his eyeball

  343. Ge0rG

    MattJ: or to get all traffic exchanged with a certain client software.

  344. Tobias

    Zash, how much ❄?

  345. jonasw

    Ge0rG: I think you actually want structured logs

  346. MattJ

    I want to submit pull requests to all other clients to change their default resource string to "yaximXYZ"

  347. jonasw

    cramming all those criteria in a single string isn’t doing any good

  348. Zash

    My position on resource selection is that the rules in xmpp-core are fine and don't need changing.

  349. Zash

    I agree with SamWhited that something else ought to be used for this kind of tracking and debugging.

  350. Zash

    Ge0rG: Would it satisfy you if we returned the log tag in the handshake somehow?

  351. Ge0rG

    Zash: the rules in xmpp-core are sufficient indeed. As long as the server doesn't override what the client sends ;)

  352. arc

    the more i think about it, the less i think about this as an issue of debugging, but more of the use case where you want your contacts to be able to specifically reach you on your laptop vs phone vs whatever

  353. arc

    that was brought up at the summit, i dont remember by who

  354. SamWhited

    The rules in xmpp-core would be fine, except that if you let clients "set" a thing, they're going to stop reading the RFC at that point and assume that's the JID they get. In my mind the rules should be "the server sets the resource part, it's opaque to clients, and the clients get no say in it"

  355. Zash

    arc: That is doable via disco#info

  356. jonasw

    Ge0rG: what about the following: 1. bind 2.0 allows for a "debug tag" 2. servers are strongly encouraged (via UX considerations in the bind 2.0 xep) to include that debug tag to every log message related to that client ?

  357. Zash

    SamWhited: The client gets to make a suggestion, but the server decides. Similar to how extensions and stuff work in TLS.

  358. SamWhited

    Because it's for *routing* which is strictly a server concern.

  359. SamWhited

    Zash: Yah, I wouldn't mind that, except it seems to be a source of bugs because clients don't actually pay attention to the servers decision

  360. SamWhited

    Or at least, that's what it sounded like at summit.

  361. arc

    SamWhited: most client authors AFAICT don't write to the rfc, they use it as a rough guide and really write to a server

  362. Ge0rG

    SamWhited: there is still no consensus on whether that _routing_ info should be persistent for a given client instance or not.

  363. Zash

    arc: And that's how we get "but it works in Internet Explorer".

  364. SamWhited

    Ge0rG: Sure, but that's orthogonal (and probably up to the server / service)

  365. SamWhited

    arc: Indeed :(

  366. Ge0rG

    SamWhited: actually it's related, because the client is the only one that knows its identity on a reconnect

  367. jonasw

    there should be a way to pain to those who do that, arc

  368. Zash

    Ge0rG: Have you thought about my suggestion of including a namespaced attribute on the stream header? That's greppable in logs, which gives you the sessions log tag, which you can then grep for.

  369. arc

    jonasw: a network testing script which tests a client or service for compliance

  370. arc

    starting with "fun" things like sending <stream:stream version="2.0">

  371. Zash

    Are there any security issues with using the stream ID as tag in logging?

  372. SamWhited

    Ge0rG: Ah, yah, fair enough, I guess you can't really separate that from the clients control.

  373. arc

    and using custom prefixes.

  374. Ge0rG

    Zash: I want to reduce the number of IDs, not increase it.

  375. arc

    just basically go through the RFC for every MUST and SHOULD, write a test for that case, and MUSTs show up as red, while SHOULD appears in yellow - any client failing to (eg) accept a different resource than requested by the server would show up this way

  376. arc

    and if you provide it, and its something client authors can find, they will almost certainly use it.

  377. Ge0rG

    Sorry, I'm in a meeting currently, and I'm heavily sleep-deprived. Can't focus on the discussion here.

  378. Zash

    arc: FWIW I don't think the client needs to know its own resource in that many cases.

  379. arc

    sure but can you think of a case where a client not understanding its resource correctly would cause a fault that you could test for on the server side?

  380. Zash

    Strip out the 'to' attribute on everything you send, see how the client reacts.

  381. jonasw

    as a client, I don’t care about the to a server sends me

  382. arc

    yea isnt it legal to do that?

  383. Zash

    No 'to' attribute is supposed to be semantically equivalent to to=full JID

  384. arc

    i mean i guess you could test an iq ping addressed to nobody, to the client by a random resource, to the client's requested resource, and to the client's given resource

  385. Zash

    Or the bare JID in the other direction

  386. arc

    replying to a ping that's misaddressed should at least be a warning, tho in that case it'd often be hard to say whether it was understanding its resource correctly or not

  387. arc

    but if it only replied to its requested resource but not its given resource..

  388. Zash

    Isn't that an error on the servers part?

  389. arc

    Zash: test servers must send bad data. thats the point.

  390. Zash

    There's been a bunch of security issues related to not validating the 'from' on certain stanzas, like roster requests and such.

  391. arc

    the point of a test suite isnt to test whether a client behaves correctly with typical data to a properly functioning xmpp server. the point is to test whether it behaves according to the RFC, so in many cases the client would - i assume - need to close the connection and reconnect.

  392. jonasw

    yeah, but from is not to

  393. arc

    or send an <iq type='error'> or etc

  394. arc

    i mean i above proposed one of the first tests would be <stream:stream version='2.0'> to check that the client is actually parsing the stream version according to the RFC. it should reject the connection, right there

  395. Zash

    arc: https://modules.prosody.im/mod_conformance_restricted.html may be of interest to you

  396. arc

    Zash: i'll look at it

  397. arc

    but does it send intentionally bad data to test?

  398. arc

    I have a utf8 test suite I'd *love* to see how both clients and servers respond to

  399. Zash

    Yes, sends XML things forbidden by the RFC

  400. jonasw

    sending PIs is bad data i guess :-)

  401. jonasw

    damn i need tobunload csi

  402. jonasw

    *to unload

  403. arc

    Zash: have you tested for UTF-8? what happens when NULL is in the middle of a stanza, say in the <message><body>? or ending a <message><body> with a chr(148) followed by </body>

  404. Zash

    arc: Have we had the conversation about IDNA versions and PRECIS and how the only reasonable thing to do is crawl down under ones desk and cry?

  405. arc

    Zash: no but it sounds like a conversation id love to have ;-)

  406. SamWhited

    Heh, this is true.

  407. Ge0rG

    arc: yay! please tell me if Unicode Robot Face (🤖 U+1F916) is a legal resource character

  408. SamWhited

    and Unicode, and UTF-8, and natural languages

  409. arc

    Ge0rG: I don't know but i'd love to find out!

  410. SamWhited

    I'm almost certain it is; I can go check if you really want.

  411. arc

    i discovered that GNU Screen has some deep UTF8 issues, as does Synergy

  412. arc

    I started digging in and found lower level libraries were at fault

  413. arc

    GNU Screen only handles 1 and 2 byte unicode

  414. arc

    internally it was using UCS2

  415. Zash

    Like how MySQL has something called "utf8" which only supports up to 3 byte UTF-8 sequences?

  416. arc


  417. SamWhited

    Yup, it's valid

  418. arc

    i think SamWhited cheated

  419. Zash

    arc: GNU libidn and IBM ICU behave differently when given Unicode outside of Unicode 3.something or whatever was state of the art at the time. One accepts. One rejects. Much fun.

  420. SamWhited


  421. SamWhited

    That JID implementation is using a very well tested PRECIS implementation that's built with Unicode 9

  422. arc

    Mr Miller *IS* in the DC area, we're setting up a time for coffee

  423. MattJ


  424. Ge0rG

    Now I wish I could have Robot Face as a sRVname SAN in a LE cert

  425. Zash

    Ge0rG: Nice things, they are unobtainable.

  426. Ge0rG

    Zash: like Unobtainium?

  427. SamWhited

    Oh no, Unobtanium is much more attainable than nice things.

  428. Ge0rG


  429. Ge0rG

    BTW, why is the Board Meeting over now?

  430. Zash

    It was the board meeting to end all board meetings

  431. Ge0rG

    Zash: I think it only ended three of them.

  432. arc


  433. arc

    so today's joy on the FLOSS Foundations mailing list is the announcement of the new Open Fashion Foundation, quote, "to disrupt fashion industry with lessons learned from computing industry."

  434. Zash

    Aaaawhat who let this override browser shortcuts?!

  435. SamWhited

    So they're going to spend all their time adding new features to cloths and ignoring the fact that the cloths are unraveling and falling off?

  436. Zash throws things at LE's discuss thing

  437. arc

    SamWhited: lol

  438. arc

    this is one I won't even be synical about. Its just a pure bundle of joy that someone out there has made FOSS licensed fashion a personal mission in their life

  439. SamWhited

    ahem, yes, sorry about that. I mean, "good for them" :)

  440. arc

    can you imagine a fashion show hosted by this organization?

  441. Zash

    The latest in beard and ribs fashion?

  442. arc

    "This piece by Manuel Debrough, available under the Apache 2.0 license from github..."

  443. arc

    Zash: oh no, dollars to donuts I'm willing to bet a fabulous gay man is behind this.

  444. SamWhited

    heh, I have a bit of a guilty pleasure in that I really enjoy fashion stuff (even though I know nothing about it, which is probably obvious if you've ever seen the way I dress), so that actually sounds pretty nifty

  445. SamWhited

    But I do enjoy seeing the things people come up with

  446. arc

    actually I can see them trying to QueerEye geek's tshirt and jeans

  447. SamWhited

    aww yeah, I'm gonna be fashionable for once

  448. arc

    the rugby club I started 4 years ago in DC just raised over $2500 in one night hosting a drag show.

  449. arc


  450. arc

    when I mentioned this on IRC, one of my friends with the Gnome foundation immediately said they needed to run a drag show, and had people volunteering. The thought of that alone is priceless.

  451. arc

    so yea I can see a geek fashion show, especially in san francisco

  452. arc

    they could raise thousands for charity too

  453. dwd

    I can see "designer-stained t-shirts" and "artful crumpling" becoming a thing.

  454. SamWhited

    Hah, indeed. I'm going to start a new line: "morning coffee spill"

  455. dwd

    "Bob wears jeans (model's own) and a t-shirt (free from some conference)"

  456. arc

    dwd: have you ever watched queer eye?

  457. dwd

    arc, Can't say I have.

  458. moparisthebest

    gah I hate that, I have jeans with holes worn in them by myself by working before that was in fashion, and now I don't want to wear them for fear of people thinking I'm trying to be fashionable...

  459. arc

    https://www.youtube.com/watch?v=g5dZ4QG7dW0 most of the men they makeover are shaggy geeks. they turn them metro. in almost every case the man starts with tshirt and jeans, and they end up posh with a new haircut, product, etc - also with their house/office made over.

  460. SamWhited

    hipster moparisthebest was into jeans and t-shirt's before they got all popular

  461. moparisthebest


  462. dwd

    moparisthebest, You're way older than I thought, then. I recall holes in jeans being fashionable, and that was when my mum bought me clothes.

  463. moparisthebest

    I seriously still wear the same jeans and t-shirts I wore when I was 18 and stuff, my wife tries to throw them away all the time lol

  464. dwd

    arc, See, I don't need that. I *can* dress up. I just usually *don't*.

  465. moparisthebest

    dwd, oh maybe it went out of style and back in, or I just didn't know about it, I'm 31 :P

  466. arc

    https://youtu.be/g5dZ4QG7dW0?t=11m25s is where they bring this one guy to buy fashonable denim to replace his "jeans"

  467. arc

    dwd: nor I. but its a great visual

  468. arc

    this is more like my husband and I: https://www.youtube.com/watch?v=kbf_nFtA8YQ

  469. dwd

    moparisthebest, Yeah, I'll be 43 soon, and I suspect my mother was telling me ripped jeans should just be replaced at about the time you were born, then...

  470. arc

    its funny, i have a tshirt and jeans policy - and have gotten a lot more traction with it than otherwise.

  471. arc

    also the beard. the bigger the beard, the more they think you know. John "Maddog" Hall taught me that trick

  472. jonasw

    that’s some unexpected backlog

  473. Ge0rG

    so much text, so laggy connection.

  474. jonasw

    Ge0rG: barely worth it if you’re not into fashion. most likely not worth it on your 30% loss link there.

  475. Ge0rG

    the link already feels like 20%. Looks like it's improving. I even have sub-second latency.

  476. Ge0rG

    Maybe I should fire up Gajim to see how it behaves with MSN and high-latency links.

  477. arc


  478. arc

    If I have the joy of reading about Open Fashion Foundation today so should all of you ;-)

  479. jonasw

    is there a section in a usual XEP where I can put notes on alternative variants I considered but eventually decided against? much like PEPs have, for example here: <https://www.python.org/dev/peps/pep-0448/#variations>? otherwise I might add a Design Considerations section…

  480. Ge0rG

    jonasw: +1 for Design Considerations

  481. moparisthebest

    that sounds right to me

  482. Zash

    # requirements it needs to do the thing # discussion we could do something, but that has these problems we colud do something else, which seems pretty good, so the rest of the spec is about this

  483. Ge0rG

    I think that every XEP should contain its rationale.

  484. Zash


  485. jonasw


  486. jonasw

    Zash: hm, PEPs do it differently: requirements, then spec, then other variants. I actually like that, because when I implement something, I don’t need to read the other variants. If I want to know why the other variants were rejected, I can skip to that section. thoughts?

  487. Zash

    No it should start with the schema! :)

  488. jonasw

    ah, I wish one could rely on schemas in XEPs.

  489. moparisthebest

    my ideal documentation would just start with already written code :)

  490. jonasw

    moparisthebest: no.

  491. moparisthebest

    in the language I'm using

  492. moparisthebest

    and it has to magically know that beforehand

  493. moparisthebest

    yea I'm joking sorry :)

  494. jonasw


  495. moparisthebest

    I agree with you about that PEP order jonasw

  496. Zash

    Language Specification: What the code does is correct. EOF

  497. moparisthebest

    right :)

  498. jonasw


  499. moparisthebest

    if you think you found a bug you are mistaken, it's actually a feature

  500. jonasw


  501. moparisthebest

    and it's apparantly worked for xep115 for 10 years right?

  502. Zash

    Is fine, don't worry

  503. moparisthebest

    ... why did I automatically read what Zash just said with a russian accent?

  504. jonasw

    https://www.youtube.com/watch?v=rp8hvyjZWHs (Trust me, i’m an engineer !)

  505. Ge0rG

    Hm. I need to youtube-dl that so I can watch it. ETA: 12:51

  506. jonasw


  507. moparisthebest

    some of those things are actually awesome

  508. moparisthebest

    the backhoe rowing the boat for example

  509. Ge0rG

    jonasw: alternatively, you could stream it to the MUC with libcaca and LMC.

  510. jonasw

    Ge0rG: my client cannot into LMC

  511. Ge0rG

    I'm sure mathieui would be glad to provide a video streaming plugin for poezio :D

  512. jonasw

    Tobias: you mentioned earlier that a server could cache xep115 responses for those specific disco#info nodes.

  513. jonasw

    I wonder whether that’s a great idea after all.

  514. jonasw

    I was wondering whether it has any privacy implications for a client.

  515. jonasw

    (on behalf of whom the server is answering)

  516. Zash

    jonasw: You may be able to guess that the server has seen a disco#info before through timing

  517. jonasw

    Zash: well, yes, but lets assume that a server has seen that disco isn’t revealing anything, for example because all servers use the capsdb.

  518. jonasw

    I wonder whether it would be okay for a server to reply on behalf of a client if the client is not actually online. While that would prevent any unintended presence leaks if the server answers for a resource which would by itself not have answered to that specific asker, it has the downside that stuff may be confused if a server answers a request for a resource which isn’t even online.

  519. Tobias

    jonasw, as long as you have not an extremely user specific client feature set, that shouldn't be a an issue

  520. SamWhited

    I don't think it's a problem because it's generally up to the server to enforce permissions / decide who can query what anyways, not the client.

  521. SamWhited

    So your server SHOULD be taking precautions to prevent presence from leaking anyhow

  522. SamWhited

    (or whatever is being queried)

  523. jonasw

    what are the criteria for an xsd to appear here? <https://xmpp.org/schemas/>

  524. Ge0rG

    NoooOooOOOooo! [download] 87.6% of 3.35MiB at 45.18KiB/s ETA 00:09ERROR: unable to download video data: [Errno 104] Connection reset by peer

  525. jonasw

    Ge0rG: youtube-dl can resume :)

  526. MattJ

    Ge0rG, it supports resum...

  527. MattJ


  528. MattJ

    I should know. Are you using my wifi by any chance? :)

  529. Zash

    MattJ: You have wifi?!

  530. MattJ

    Too many complaints from "smart"phone users in the house to resist any longer

  531. Ge0rG

    MattJ: free WiFi on a rowded train, moving at 200km/h

  532. Ge0rG

    MattJ: free WiFi on a crowded train, moving at 200km/h

  533. moparisthebest

    kind of amazing that works at all

  534. arc

    jonasw: https://youtu.be/rp8hvyjZWHs?t=2m37s has got to be the best hack I've seen in a long time

  535. jonasw


  536. moparisthebest

    arc, the rowing backhoe? yea that impressed me the most

  537. arc


  538. moparisthebest

    there is no arguing with that one, boat motor breaks, have a backhoe on board, it's ingenious

  539. arc

    i thought my use of a toilet fill valve in a bucket for plant watering was good

  540. Zash

    I don't usually have a backhoe on board

  541. arc

    this is a whole new level

  542. dwd

    Zash, So what do you do if your motor breaks?

  543. moparisthebest

    probably something boring like an oar

  544. Zash

    I guess I would have to convert it into a putt putt boat

  545. Zash

    I would also have to get a boat and a motor...

  546. arc

    what if you had a car onboard and could get it up on jacks

  547. moparisthebest

    change out the wheels for paddles like an old river boat?

  548. dwd

    arc, If he doesn't even have a boat he's got worse problems.

  549. SamWhited

    arc: Like this (sort of)? https://www.youtube.com/watch?v=dyBl9vf8Td0

  550. arc

    thats true. Zash how will you hack up a boat to start with?

  551. Zash

    But why would I have a boat? Not really a water person.

  552. Zash

    I'd rather have cabin in the woods and some potatoes. Backhoe would come in handy then.

  553. arc

    Oh, I *really* doubt that you want to have cabin in the woods

  554. arc


  555. ThurahT

    true, there are nicer things than a portal to a demi-god-demon

  556. Zash

    Can't be worse than the mosquitoes

  557. arc

    i'll take mosquitos over the horrific monsters they send to kill you

  558. arc

    and what rises if EVERYONE fails

  559. moparisthebest

    I think I'd prefer the things I could kill with guns

  560. arc

    I think the scene of the japanese school children circling around and dispelling the demon is the best

  561. arc


  562. arc

    "The spirit of the demon will now live in the happy frog!" ... "How hard is it to kill a group of 9 year olds?"

  563. moparisthebest

    this has been an odd day in the xsf, went from talking about fashion, to boats rowed by backhoes, to demons in cabins in the woods

  564. arc

    blame me.

  565. moparisthebest

    with some xmpp sprinkled in :)

  566. arc

    yea there's XMPP involved, that's all that matters. That means we can charge lunch to the corporate card right?

  567. Ge0rG blames arc.

  568. arc

    after all the work I did I realized this morning that the hash function isnt likely all that useful for embedded systems, and in 95%+ of the cases won't even get included in the binary

  569. arc

    embedded xmpp is unlikely to include text xml.

  570. Ge0rG

    It ain't no fun with the lags.

  571. arc

    the hash function is used pretty much, if not entirely exclusively for hashing text strings in order to find a cooresponding match on the string table

  572. arc

    anyone else have a problem that you dig too deep into a problem that you lose sight of the big picture?

  573. SamWhited

    oh yes… frequently.

  574. Ge0rG

    When I dig too deep into a problem I always encounter sub problems to which there is no documented solution on the Internets, but often many people having the same issue.

  575. MattJ

    Don't get me started, today has been one of those

  576. arc

    i hate that.

  577. Zash

    I still got some glibc in my eye from yesterday.

  578. arc

    or you dig deep enough that you realize its a problem caused by the language you're using that can't be fixed, just.. worked around

  579. MattJ

    e.g. the moment when I realised (after putting log statements all over the place) that the testing tool I was using was broken, and connecting to the wrong server

  580. MattJ

    (in production)

  581. Zash

    Why isn't getrandom() in glibc until like the latest bleeding edge version nobody has?

  582. MattJ

    and the rabbit hole just goes deeper

  583. MattJ

    and now I'm just looking for some utility that will read lines from stdin and send them somewhere as UDP packets

  584. MattJ

    and trying to pretend I don't need to write my own

  585. Zash


  586. MattJ

    netcat failed on the "line" part

  587. arc

    my first "in office" job had two charming things; 1) a ban on coffee in the office (only green tea, because of management philosophy hogwash), and 2) "Eat Me" cookies in a sealed container in the break room for when you get trapped too deep in a rabbit hole

  588. arc

    it took me far too long to realize the reference

  589. Zash


  590. arc

    a also found that for every schema i could think of, bitpacked EXI is better, faster, and smaller binary than compressed EXI

  591. arc

    i didnt expect that.

  592. moparisthebest

    that's just a type of compression though isn't it?

  593. Zash

    What's compressed EXI?

  594. moparisthebest

    like it'd probably be equally susceptible to CRIME / BREACH type attacks?

  595. arc

    I guess you could call bitpacking a form ofcompression..

  596. arc

    Zash: so there's 4 modes for EXI; bitpacked, simple byte-aligned, pre-compression, and compression.

  597. arc

    byte-aligned is essentially the same as bitpacked but always padded to byte alignment, obviously

  598. Zash

    Can you explain them in terms of ASN.1 encoding schemes? :)

  599. arc

    compression is pre-compression plus DEFLATE

  600. Zash

    (that was a fun rabbit hole too)

  601. moparisthebest

    so which ones are secure under encryption? only pre-compression?

  602. arc

    pre-compression is byte-aligned, but with similar types of data grouped together on the stream. so eg all int values are together, all string values together, etc

  603. arc

    i wouldn't propose to know the answer to that moparisthebest

  604. jonasw

    MattJ: socat READLINE: UDP:?

  605. moparisthebest

    arc, probably should have someone figure it out before starting to use/promote it though?

  606. arc

    but the idea with pre-compression is that some form of compression will be applied on, eg, the TLS layer

  607. MattJ

    jonasw, I saw that, but READLINE seems to actually involve the readline library, i.e. it's intended for human input, not piping from another program

  608. jonasw

    MattJ: and STDIN doesn’t do the trick? :/

  609. jonasw


  610. moparisthebest

    arc, I think most if not all TLS libs removed support for TLS level compression because it's woefully insecure

  611. arc

    moparisthebest: i can *barely* hold enough of the EXI specification in my head to work on it. i don't have room for encryption on top of it.

  612. Ge0rG

    New personal record. Sigh: 64 bytes from icmp_seq=9 ttl=53 time=377539 ms

  613. MattJ

    jonasw, only if they split on lines (which I see no indication of)

  614. jonasw


  615. moparisthebest

    arc, and you shouldn't have to consider it at all as long as you don't do anything that makes it insecure, compression being one of those things

  616. Ge0rG

    MattJ: I'd write a small loop with scapy.py

  617. MattJ

    I found a utility, it just needs the correct command-line arguments

  618. arc

    but i would assume if you consider compression insecure, eg DEFLATE, Brotli, etc, then you would prefer bitpacked over all options

  619. MattJ

    lua -e'u=require"socket".udp() for line in io.lines() do u:sendto(line, os.getenv"HOST",os.getenv"PORT") end'

  620. jonasw

    python3 -c 'import socket; s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM); while True: s.write(s.stdin.readline().rstrip("\n"))'?

  621. jonasw


  622. MattJ

    Lua wints ;)

  623. MattJ

    Lua wins ;)

  624. arc

    at this point my primary concerns are the size of the embedded image. cutting text-domain XML out reduces the binary size of the library in about half. removing compression library is a pretty big win too.

  625. jonasw

    moparisthebest: CRIME and BEAST are based on the fact that the packet size changes depending on previously sent content, I doubt that this is the case with bit-packed, from the sound of the name :)

  626. jonasw

    but I haven’t looked into it, at all

  627. arc

    wolfssl is pretty small

  628. jonasw

    soo… now I have that xep-ecaps2.xml here, let’s check out xep-0001.xml on what I need to do next.

  629. moparisthebest

    arc, well compression is insecure because if an attacker can add the string "ar" to the payload and the size doesn't increase, then add the string "arc" and it still doesn't change, and build up from there, it can figure out what's under the encryption

  630. moparisthebest

    so if bit packing works in a similar way, it's equally insecure

  631. Zash

    jonasw: Print it on paper, fold a paper airplane and aim for SamWhited :)

  632. moparisthebest

    right jonasw I don't know either, just saying it's probably something that should be determined

  633. arc

    moparisthebest: hmm. no i don't think so. so the only way you could reverse engineer it would be exploiting the string table.

  634. moparisthebest

    like it'd be another useless thing to work on if it was proven as insecure as compression arc , idk

  635. jonasw

    moparisthebest: it also probably does not matter much for IoT-thing <-> gatewaything.

  636. moparisthebest

    yea it's pretty obvious security doesn't matter when it comes to IoT haha

  637. jonasw

    like Ge0rG quoted yesterday: "The S in IoT is for security"

  638. SamWhited

    It would actually be pretty awesome if XEPs were submitted that way…

  639. arc

    moparisthebest: ok, so string values are stored in the string table. this refers to whole strings only, but eg a JID you're communicating with would be added to the string table and referenced by id.

  640. SamWhited

    Please change the font to OCR-A or something first so I can scan it back in though.

  641. jonasw

    SamWhited: because you would not have to do any work, as paper planes don’t travel several thousand km?

  642. SamWhited

    jonasw: Says you; that just means you're not building a big enough paper airplane!

  643. jonasw

    SamWhited: we could also try XMPP over RFC 1149

  644. SamWhited

    heh, indeed

  645. SamWhited

    My favorite part is that there are Errata for that one.

  646. jonasw

    there was an actual implementation

  647. moparisthebest

    ok arc so the full payload size would increase with the strings you added, say "arc" would increase it 3 bytes, *unless* that FULL string was already in there, then it wouldn't decrease at all? if I understood you correctly

  648. moparisthebest

    that at least would not let you incrementally guess strings like 'a' then 'ar' then 'arc' etc etc

  649. arc

    moparisthebest: so if you can send a string value containing a 3rd party JID that you want to know if that agent is already communicating with, AND you know the schema being used, then you can determine whether that agent has communicated with that JID already.

  650. moparisthebest

    like you I don't know enough to say without a doubt that makes BREACH or CRIME not a problem, but it seems better to me...

  651. arc

    moparisthebest: yes. I do not recall a method for partial or combined strings

  652. jonasw

    assuming you can observe the network traffic between those entities, which may only be within the local wifi

  653. Ge0rG

    RFC1149 would be faster than my current link.

  654. arc

    i'm still loading the spec back into my head. but i remember that as a fault.

  655. arc

    one of my criticisms of EXI actually is the lack of a "list" type

  656. moparisthebest

    I'd feel better if someone like xnyhps said they'd reviewed it and it looked good to them :)

  657. SamWhited

    eeew, I just decided I should actually print EXI and read it… but it goes on forever.

  658. arc

    this comes up in some XML schemas such as SVG, where paths are made of collections of floats, ints, and characters separated by spaces

  659. Zash

    Hm, I should look at what a printer costs

  660. arc

    SamWhited: yea its not light reading. I recommend https://www.w3.org/TR/exi-primer/ to start with

  661. SamWhited

    arc: Thanks

  662. jonasw

    is there an email-adress where XEPs are supposed to go? the http://xmpp.org/xmpp-protocols/xmpp-extensions/submitting-a-xep/ page linked in XEP-1 404s

  663. arc

    that gives a very nice overview without sucking you into the details

  664. SamWhited

    jonasw: You can submit a PR on GitHub

  665. jonasw

    Zash: nothing, just "google" for one and ask the owner kindly to send you the printouts :)

  666. Ge0rG

    jonasw: you can make a PR of the XEP in inbox/

  667. jonasw

    SamWhited: which puts my xep in the inbox/ dir?

  668. jonasw


  669. SamWhited

    jonasw: Yup

  670. arc

    you're younger than me, you might be able to handle it better, but ive had to segmentize the details so i dont get overwhelmed. its a lot to hold in your head at once

  671. SamWhited

    oh I doubt that; if you can't hold the entire spec in your head I doubt I have any chance

  672. arc

    that's flattering but I doubt its true. age wears down your memory

  673. SamWhited

    jonasw: See the other XEPs in there for naming, I *think* you don't want it to start with xep- for reasons that I can't remember… something, something tooling.

  674. arc

    I'm turning 38.

  675. jonasw

    yeah, figured that much

  676. moparisthebest

    speaking of the inbox, some of those things are *ancient*, does or should it ever be cleaned out?

  677. jonasw

    am I the only one *always* falling for the delay github has with showing the "you have pushed to branch X n minutes ago, do you want to pull request?", hitting F5, seeing it appear before the page has reloaded, click compare & pull request and then the page reloads and you’re back to square one?

  678. SamWhited

    I think the editor readme says it never gets cleaned out. We don't want to break old pages.

  679. SamWhited

    Oh yah, I do that all the time

  680. moparisthebest

    break pages? do they get rendered?

  681. moparisthebest

    or you just mean links to the xml ?

  682. SamWhited

    moparisthebest: they get rendered on the site, just like actual XEPs

  683. moparisthebest

    I didn't know that

  684. jonasw

    SamWhited: https://github.com/xsf/xeps/pull/440 consider yourself paperplaned also: https://www.youtube.com/watch?v=Co452wJ-3Lg (Long Distance Calling - Black Paper Planes) (Music)

  685. moparisthebest


  686. moparisthebest


  687. SamWhited

    moparisthebest: Also, ¿Porque no los dos?

  688. SamWhited

    (I couldn't find the adorable little girl gif to send, so you just get text)

  689. arc

    given the current status of IoT I think I might actually focus for a few weeks on *just* the schema compiler and get a XEP out for it. the one thing im missing for the XEP is a definition for the schema of the schema

  690. SamWhited

    > the schema of the schema

  691. SamWhited

    I'm so sorry…

  692. jonasw

    that meta

  693. jonasw

    arc: schemas like in XML Schemas for XEPs?

  694. jonasw

    how are you going to deal with the mostly incorrect or inaccurate schemas out here in XEPs?

  695. jonasw

    well, probably not mostly.

  696. jonasw

    but they’re not normative, I’ve been told once.

  697. arc

    yea, in order for a client to transfer to the server the schema that it wants to use, which the server doesnt already have, it needs to be able to dump the EXI-encoded schema to the server. and that needs to be defined since every client and server needs to be able to understand it

  698. arc

    so the EXI schema for the EXI schema needs to be defined in the XEP

  699. jonasw

    that’s meta.

  700. arc

    its why I havent touched the XEP yet.

  701. arc

    but it needs to happen, and sooner the better

  702. jonasw hands arc a large bag of tea.

  703. moparisthebest

    sounds like he needs something harder to me

  704. arc

    i havent written a line of code in a month. i'm up for it.

  705. moparisthebest

    maybe 160+ proof

  706. jonasw

    there are too many movies showing that coke doesn’t end well.

  707. jonasw


  708. jonasw


  709. Zash

    160+ proof tea?

  710. arc

    oh I have a copeous amount of cannabis

  711. arc

    there's a "Balmer limit" to cannabis too, though.

  712. jonasw


  713. arc

    er "Ballmer Peak" https://xkcd.com/323/

  714. jonasw


  715. dwd

    jonasw, That your protoxep? ecaps2?

  716. jonasw

    dwd yes

  717. arc

    though its more a cliff. more is better, to a point, and then rapid degeneration. its around the point that you start feeling like time is on a bungee chord

  718. dwd

    jonasw, I think you win the prize for using every obscure separator character in the ASCII subset.

  719. jonasw

    dwd: thanks :D

  720. jonasw

    they were barely enough, I was worried I’d also need EOT

  721. dwd

    jonasw, Can those appear in XML?

  722. jonasw

    dwd: no.

  723. jonasw

    XML forbids control characters except htab, newline and carriage return

  724. jonasw

    (those between 0x00 and 0x20 at least)

  725. Ge0rG

    Hm. Thereis an IoT thread going on with me in Cc. I wonder who deemed me so important and why.

  726. dwd

    jonasw, Perfect. Nicely done.

  727. jonasw

    dwd: thanks! :)

  728. arc

    Ge0rG: you are the chosen one for IoT. you must lead the way, because everyone knows nobody else knows it

  729. dwd

    arc, IoT is different and special from everything else.

  730. Ge0rG

    arc: this must be a SCAM.

  731. arc

    I'm humored by these IoT "Meetups" full of VCs who think IoT means a standalone device that communicates solely with their service, like a modern wifi-connected thermometer that you can control with your phone through their online service

  732. Zash

    jonasw: "Cabability"

  733. arc

    in that ideology things like protocol standards don't matter. they mostly use a HTTP ReST API between the device an their service

  734. dwd

    arc, The sad thing is that most of these devices are going that way.

  735. jonasw

    Zash: that’s only because you cannot use entities in <dt>! thanks, fixed locally, waiting for more of these stupid typos before I push another commit.

  736. arc

    dwd: only because of the novelty of it. we need to catch up to steer course

  737. dwd

    arc, And worse, those that aren't suffer - my iKettle, for instance, is controlled locally, but people want to integrate - and they have to integrate via cloud services now.

  738. Zash

    dwd: Like the e-reader thing requiring an account with some online service to display text?

  739. arc

    why does your .. what i assume is a water kettle.. need remote access?

  740. arc

    that's my other IoT rant that I won't get into. not everything needs a chip in it. bloody Target selling basketballs with a chip in it to count bounces and report them to your phone via bluetooth

  741. dwd

    arc, So I can set it to boil from my desk, and - more importantly - so I get a notification on my smartwatch when it does.

  742. arc

    my basketball does not need bluetooth.

  743. dwd

    arc, I understand. You're wanting it to use zigbee instead?

  744. arc

    dwd: lol

  745. jonasw


  746. arc

    dwd: you're doing well roleplaying an IoT VC!

  747. dwd

    arc, I'm just like a VC, except without the money.

  748. arc

    oh so you're homeless? ;-)

  749. Zash

    dwd: My water boiler has this amazing wireless notification protocol called "loud click and the sound of boiling water slowly fading away"

  750. moparisthebest

    a basketball with a bounce counting chip?

  751. SamWhited

    mine makes a sort of loud whistling noise when the water is ready

  752. dwd

    Zash, Well. I can actually hear the kettle from my desk, in fairness.

  753. moparisthebest

    I'd think you were joking if I didn't know better

  754. xnyhps

    moparisthebest: I didn't read much of the backlog, but the DEFLATE option for EXI very likely is vulnerable, without, probably.

  755. Zash

    Weren't there baseballs with accelerometers in them to measure how hard they got hit?

  756. SamWhited

    it sounds vaguely like air being forced through a small round opening

  757. moparisthebest

    maybe I will move to a cabin in the woods like Zash :)

  758. jonasw

    Zash: uh, I once had an oven which had the protocol of "if you don’t take care the water boils over the pots edge and flows down the sides into the oven tripping the RCA and thus cutting of your power"

  759. arc

    we have a bluetooth enabled pressure cooker. it has a bluetooth range of maybe 8 feet, 10 if you're lucky. the app you need to communicate with it has basically a clone of the physical interface on the machine

  760. moparisthebest

    xnyhps, yea any compression like deflate/brotli/etc would be, the question was whether the 'bitpacking' optimization without compression would be

  761. moparisthebest

    or, without what we normally call compression

  762. moparisthebest

    I suck at wording

  763. Zash

    moparisthebest: Call it PER

  764. arc

    xnyhps: DEFLATE only or newer methods like Brotli too

  765. dwd

    moparisthebest, EXI in bitpacking mode doesn't have back-references, which is the basic issue.

  766. arc

    but there is the string table, which I think would argue could have issues, and that's in all modes.

  767. arc

    your own JID, for example, will be on the string table. so if someone could send you a jid as an attribute value, i believe it could under specific conditions, confirm if that is your JID or not.

  768. Zash

    jonasw: 'the i;octet' intentional or typo?

  769. arc

    or if your device is communicating with a server, and they know which IP you're communicating with but not the specific hostname..

  770. dwd

    Zash, ACAP Comparator. Not a typo.

  771. jonasw

    Zash: that’s how it’s called. not my idea :/

  772. dwd got his ACAP server compiling again the other day because someone actually wanted to use it.

  773. Lance

    jonasw: ^5 on the XEP, this looks awesome

  774. jonasw

    Lance: thanks

  775. arc

    but given what moparisthebest described earlier I think that's a lot less of a security risk, since you couldn't pull out substrings to progressively reverse engineer, and the specific conditions are more difficult to otherwise achieve

  776. Zash

    jonasw, dwd: Well it could also have been an artifact of the conversion to epub I did

  777. moparisthebest

    right I think you couldn't progressivly build up by guessing 1 character at a time that way arc

  778. dwd

    Zash, RFC 4790, now, extracted from ACAP. My mistake, I'm behind the times.

  779. xnyhps

    If you're not compressing the password anyway, the thread model becomes rather vague.

  780. moparisthebest

    which again sounds better/more secure to me, but probably not as secure as not being able to guess at all? I'm sure someone could come up with an attack

  781. xnyhps

    Finding out someone's JID requires quite a lot of access for not that much information.

  782. arc

    yea no the string table refers to whole qnames and string values

  783. xnyhps

    Or who someone is talking to, etc.

  784. arc

    yea its not exposing, say, integer values coming from a sensor

  785. dwd

    Surely you'd need to address things to them? So at best, you're able to try to guess if someone who you already know by IP address, who is also in a chatroom with you, is the Jid you think they are?

  786. xnyhps

    dwd: Yeah, and you probably have much easier ways to do that.

  787. arc

    well it only confirms that they're a JID in your string table. it wouldnt expose, necessarily, if they were that JID vs had talked to that JID

  788. arc

    EXI doesn't "understand" XMPP beyond the schema you provide it.

  789. arc

    i think it might be possible under certain conditions for an IoT vendor to craft an insecure schema tho

  790. arc

    for example sensor data should be fixed length

  791. dwd

    TBH, I don't think that the use of deflate in XMPP is a general problem anyway. In extremely high-risk cases, perhaps, and if you're dumb enough to use PLAIN and TLS compression.

  792. arc

    in that way EXI is more secure than text xml in that integer should be a fixed length, where a string representing an integer is not

  793. jonasw

    Zash: do you have a diff of xep 369 from 0.8 to 0.8.1 from your fancy difftool at hand?

  794. arc

    for security any XEP for sensor data, it should be actually put in the security section that float and integer values should be zero-padded to their maximum value to decrease risk of data leakage

  795. jonasw

    zero-padded to their maximum value? how does zero-padding to maximum value work?

  796. arc

    if all the stanzas from a device are the same except being X length, X+1, X+2, X+3, etc based on the scale of a specific integer value, you can determine whether that value is 0-9, 10-99, 100-999, etc

  797. Zash

    Don't EXI basically work like if you were to generate optimal C structs for all the things, then send that down the wire?

  798. arc

    so if the maximum value is, say, 255, it should send as '001' '002' etc

  799. jonasw

    arc: wait, leading zeros are encoded?

  800. arc

    jonasw: for text xml

  801. moparisthebest

    if it's a string it has to be

  802. arc

    what i was saying is this is a weakness in text XML that EXI doesn't have

  803. moparisthebest

    but even if not most things send integers in a set number of bytes

  804. arc

    sure, but eg, a light sensor could flip between 0 and 100, and that would make it obvious what the state was

  805. jonasw

    ah I thought you were talking about EXI already, of which I assumed that it encodes it as binary integer

  806. arc

    people do not generally encode 0 as <light value="000"/>

  807. arc

    yea it encodes as a binary integer

  808. jonasw

    is it a variable-width encoding?

  809. arc

    i would have to look that up again, i havent touched that part in awhile

  810. arc

    i know you can constrain the range of most values

  811. jonasw

    that would have the same issue then, and it cannot be worked around with leading zeros

  812. arc

    well i don't believe its variable width per value, i think its only variable width by schema. if the schema says the integer value of a given attribute is 0 to 127, it'll do the right thing.

  813. arc

    i havent touched that since november tho, id have to read up on it again

  814. jonasw

    no worries

  815. arc

    but im like 98% certain that an integer, float, etc value is fixed width from stanza to stanza

  816. moparisthebest

    the question is if an integer can be 0 to 65535, it obviously encodes 60000 as 2 bytes, but does it encode 120 as 1 byte or 2 ?

  817. moparisthebest

    that'd be a type of compression too

  818. arc

    i believe that if an integer is a short it will always be a short.

  819. moparisthebest

    could leak something, idk

  820. arc

    you're right it could. but i dont think it does that.

  821. moparisthebest

    that's how everything I can remember seeing works yea

  822. arc

    and when we draft EXI 2.0 that is something that should be definitely put on the table as a concern

  823. moparisthebest

    in general it seems like most things pre-2013 kind of took security as an after thought and might need to be revisted today

  824. arc

    so far the only thing I would like to add to EXI is being able to encode a delineator-separated sequence like is used in SVG

  825. arc

    if we had that, the SVG world would be all over it

  826. arc

    being able to encode paths more efficiently would be a major breakthrough.

  827. Zash

    jonasw: You happen to know which revisions that correspond to?

  828. jonasw

    Zash: nevermind, I diffed it locally

  829. arc

    my initial interest in EXI came from getting tired of hearing about why X chat system doesn't use XMPP, but a binary protocol, for efficiency on mobile / etc

  830. arc

    and the same is true for SVG vs proprietary vector formats

  831. jonasw

    I’m going to bring up the <feature xmlns="…" /> stuff on standards@ again.

  832. moparisthebest

    my complaint about SVG is that most things just arbitrarily execute javascript from them

  833. moparisthebest

    not a great security feature

  834. Ge0rG

    I wish I'd get some more insight from The Elders on carbonated body-less normal messages...

  835. arc

    moparisthebest: the same is true for XHTML-IM

  836. moparisthebest

    yep arc

  837. jonasw

    script content is not allowed in XHTML-IM…

  838. moparisthebest

    but like on my discourse instance I enabled common image format uploads, for example png, jpg, gif, and svg

  839. jonasw

    (reminds me, I wanted to polish up my XSLT which strips off anything not allowed as per xep 71)

  840. moparisthebest

    then, luckily it was a friend, uploaded an svg with some XSS javascript to steal cookies and showed me :)

  841. jonasw

    are there any xslt/xhtml wizards here?

  842. moparisthebest

    I'd assume this is where the xslt wizards live :) not me though

  843. Lance

    jonasw: stuff like <a href="javascript:alert(1)"> can still exist even without allowing <script> elements

  844. jonasw

    Lance: haven’t thought of hrefs, good point

  845. jonasw

    but that is usually easily filtered depending on the webview used

  846. dwd

    Lance, Dependsing on CSP.

  847. moparisthebest

    a blacklist would be a never ending hole

  848. jonasw

    moparisthebest: that’s why I’m using the whitelist from the XEP.

  849. dwd

    moparisthebest, No, I mean Content Security Policy stuff would prevent inline javascript from working.

  850. moparisthebest

    I'm not positive you can do that kind of thing with xslt

  851. moparisthebest

    yea dwd, not sure how you get/set that with something like xhtml-im

  852. moparisthebest

    surely if there was a handy .noJavascript() method they would have called it

  853. arc

    XSLT could do it. You shouldn't do this with XSLT.

  854. arc

    no matter how hard you try it will always leave a hole

  855. jonasw

    arc: what exactly?

  856. arc

    jonasw: filtering XML/HTML

  857. jonasw


  858. jonasw

    how else are you going to do it?

  859. jonasw

    also, I think that this should be pretty sound: https://github.com/horazont/aioxmpp/blob/devel/data/xhtml-im-sanitise.xsl (leaving aside the @href issue)

  860. arc

    I'm in the camp for saying XHTML-IM shouldn't be supported

  861. arc

    I wasn't. now I am.

  862. moparisthebest

    I agree

  863. jonasw

    arc: I also do not like XHTML-IM.

  864. jonasw

    but then again, there are people who want rich text in their IM clients.

  865. Zash


  866. moparisthebest

    you can have rich text without html

  867. jonasw

    moparisthebest: is there a XEP for that?

  868. moparisthebest

    not that I know of :)

  869. arc


  870. moparisthebest

    someone was advocating markdown somewhat recently

  871. jonasw

    (actually, a body type="text/markdown" or type="text/rst" would be great; just make sure your markdown/rst doesn’t pass through HTML…)

  872. moparisthebest

    right :) or it starts all over

  873. Zash

    Wasn't Markdown is defined as a HTML superset?

  874. jonasw

    yes, Zash

  875. arc

    i dont think thats still a complete solution.

  876. Zash

    Nice things, you can't have them

  877. arc

    the <a href="javascript:"> links will leak through

  878. moparisthebest

    well as Zash said bbcode it is then

  879. Lance

    plus the issues with multiple flavors of markdown, etc

  880. moparisthebest

    I'm sure there are plenty of libraries already ready to use

  881. moparisthebest

    in php...

  882. jonasw

    gah, bbcode is annoying too.

  883. Zash

    There can be only one! (And it is pandoc)

  884. Zash <3 pandoc

  885. moparisthebest

    as the saying goes annoying or insecure pick one

  886. moparisthebest

    I probably just made that saying up

  887. arc

    Lance: btw one thing i love is the stream framing from websockets? the added overhead for jabber:client namespaces is completely eliminated in EXI

  888. Lance


  889. arc

    if back then when that was being vexed over, if someone had said "in 5 years that won't be an issue anyway because EXI" it would have made the decision much easier

  890. Flow

    jonasw: I do think that xep115 has hash agility, and signalling the caps using a second hash algo wouldn't require a ns bump

  891. moparisthebest

    re: markdown only one markdown I know has a defined spec, http://commonmark.org/

  892. arc

    good lord, i cant even use libxml2 anymore. its just painful.

  893. jonasw

    Flow: there was some mailing list post where people discussed otherwise, in the thread Tobias linked I think

  894. arc

    schema-based xml coding makes so much more sense

  895. moparisthebest

    so I think if you mandated commonmark with the exception of no support for http://spec.commonmark.org/0.27/#html-blocks it might be easier, would need more thought

  896. Flow

    nothing prevents clients from using a second hash mech, as long as they still send the mandatory to implement one

  897. Zash

    Flow: You mean sending multiple <c> elements?

  898. Flow

    Zash: yep

  899. Zash

    Flow: Doesn't fix the algorithm for producing the hash tho

  900. Flow

    Zash: Right

  901. Flow

    But I don't aggree with the statement that the change of the hash function of xep115 requires a namespace bump in ecaps2

  902. Flow

    jonasw: Any particular reason for going with a new xep instead of updating xep115?

  903. jonasw

    Flow: I asked here, and people suggested that a clean new xep is the better way to go.

  904. Flow

    jonasw: i see

  905. Lance

    IIRC, it was so we could flag 115 as obsoleted by the new one

  906. Kev

    jonasw: Well, I think I suggested that a new XEP was the wrong way to go, and updating 115 was preferable :)

  907. Lance

    as an encouragement to devs to upgrade

  908. jonasw

    Flow: to be clear, I’m happy to drop -xxxx and merge the changes into 115 if council prefers that.

  909. Ge0rG

    also to prevent people from doing some compat with the old stuff badly.

  910. jonasw

    but considering that it were council people who suggested to go with a new xep, I followed that suggestion.

  911. Flow

    pfff, council people are not always right ;)

  912. Lance

    not even close :)

  913. jonasw

    Flow: they’re, from my understanding, those who decide whether a patch to XEP-115 will be accepted though.

  914. Kev

    I think it led to the wrong outcome in this case, but I can't fault the logic of taking advice from Council in general.

  915. Flow

    Sure, asking for feedback is always a good idea.

  916. SamWhited

    It seems like a good idea to me to go with a new XEP in this case just to encourage people not to try and have backwards compatibility with the old one (which rather defeats the purpose of having a new one), but I don't feel strongly about it and could be convinced either way.

  917. jonasw

    in any case, I’m off for tonight. may read the backlog if highlighted

  918. SamWhited

    defeats the purpose in this case, I mean, since it's a security issue. Backwards compatibility is sometimes a good idea.

  919. Kev

    SamWhited: 115 is a core dependency of a *lot* of XEPs. I don't think replacing it is warranted in this case.

  920. SamWhited

    yah, that is tricky, not sure what to do about that. Either way tough we'd have to solve that problem and I suspect the two will have to coexist for a while.

  921. Flow

    Kev: The question is: Is xep115 is dependency or xep115 *and* the current namespace of xep115?

  922. Kev

    Well, at least for the dependency, it's straightforward, as the dependency is just on the latest version of 115.

  923. Kev

    Whether it should be or not is another matter, of course.

  924. Flow

    This is a fundamental question as we will find ourselves in the situation more and more in the future. For example with the XEPs depending on xep300

  925. Lance

    Yeah, aside from PEP, most of the "dependency" for these XEPs is just the fact that it optimizes the true dependency on disco#info

  926. Zash

    Do we need a BCP kind of thing?

  927. Flow

    Do we want to update all consumers of xep300 if it receives an incompatible update?

  928. Flow

    Or do we want to sepcify a dependency as xep number *and* "namespace", and update the consumers one after another?

  929. Flow

    Lance: Well said. I hate that some XEPs give you the impression that xep115 is an alternative to xep30

  930. Flow

    Zash: BCP?

  931. Zash

    Flow: IETF thing, like a pointer to the latest RFC on some specific topic.

  932. Lance

    Best Current Practices

  933. Flow

    ahh berst current practice

  934. Zash

    Flow: RFCs never change, but a BCP may be changed to point to a new RFC

  935. Flow

    isn't the the opposite what XEP do?

  936. Flow

    i.e., they do change, so we need a pointer to a fixed revision of a xep

  937. Flow

    (which we have in our attic btw)

  938. Zash

    Final XEPs are probably the closest to how RFCs work

  939. Flow


  940. Flow

    ahh, enough DNSSEC fun for today. I follow jonasw to the realm of sweet dreams where everthing is like it should be

  941. arc

    its too bad SRV records don't allow additional information

  942. Ge0rG

    Flow: and dreem of jumping and colliding SHA1eep?

  943. Lance

    arc: what kind of additional info?

  944. arc

    i havent touch DNS resolution in awhile, can you send a single request for multiple SRV records?

  945. arc

    Lance: for example, the server capability, protocol version, etc

  946. Zash

    arc: Multiple how?

  947. moparisthebest

    with the same name sure

  948. Lance

    arc: whether or not to start with EXI, hrm?

  949. arc

    Lance: yes, or TLS, or etc

  950. moparisthebest

    I suppose that'd be what TXT records are for arc

  951. arc

    yes I know there's a XEP for TLS

  952. moparisthebest

    or encode TLS or not TLS in the name like I did haha

  953. moparisthebest

    that would easily explode though if you try to encode more

  954. arc

    moparisthebest: yes, but doesnt that require multiple lookups? or can the two alternative names be requested at once?

  955. moparisthebest

    now we have _xmpp-client, and _xmpps-client, we don't want _xmppse-client and _xmppe-client for exi for example too, probably

  956. Zash


  957. arc

    yea. so.. part of EXI is the first byte of an EXI stream is never a valid text unicode string by any enconding

  958. moparisthebest

    yea arc that's 2 seperate lookups

  959. arc

    one way is SRV records. the other way is to just punch EXI at the server, and it either responds with EXI or not

  960. moparisthebest

    arc, uh what about ALPN I think that neatly solves your problem?

  961. arc


  962. moparisthebest

    tls extension, tells it the protocol(s) you'd like to speak

  963. moparisthebest

    Application Layer Protocol Negotiation ?

  964. moparisthebest

    http2 uses it

  965. arc

    oh, yes that could work

  966. arc

    ive seen this before, just forgot about it

  967. moparisthebest

    xep-0368 uses it too, but optionally

  968. arc

    yea i saw this mentioned somewhere about http2 awhile ago. so, what does the payload look like

  969. Zash

    A text string in a TLS extension

  970. Ge0rG

    a byte array.

  971. Ge0rG

    because text strings are imPRECISe

  972. arc

    ok so we could define a meaning for that which is extensible to other things

  973. moparisthebest

    yea Ge0rG is more correct it's a precisely defined sequence of bytes

  974. arc

    the key is it must be possible to use EXI without support for text XML

  975. moparisthebest

    so basically an EXI xep could depend on xmpps-* records from xep-0368, and send it's own custom ALPN protocol sequence

  976. moparisthebest

    or optionally, both xmpp-client and xmpp-exi-client or whatever

  977. moparisthebest

    and server would say I can speak X

  978. moparisthebest

    at which point you'd proceed or try next SRV record

  979. arc

    i *hope* that server support would be well deployed before its an issue

  980. arc

    oh interesting. it doesnt look like Contiki OS supports ALPN

  981. Lance

    arc: also, once the EXI XEP is decent, I'd be happy to help with making a proper xmpp-exi websocket binary subprotocol

  982. arc

    Lance: absolutely. but lets get a javascript library for it first ;-)

  983. arc

    from the times its been brought up i think the right path is to kill 0322 and start fresh. the one up there is utter nonsense from an implementers point of view

  984. arc

    50% of the document is re-implementing EXI header format in a less compact form

  985. arc

    and it doesn't even really get into how to handle a "pure" EXI stream (not starting with text XML)

  986. arc

    the mechanism I think is best is this: 1) Client sends EXI header with <open> framing. in the header, the schemaId field contains a hash identifier for the schema it wants to use, generally in sha256: URI format, but this allows future hash values to be used

  987. arc

    2) if server doesn't already have that schema, it responds with EXI header for a "default" stream using the schema-schema, and gives an error that the requested schema must be provided

  988. arc

    3) if client receives such an error, it will restart its EXI stream with the same schema and transfer that schema 4) server responds with the hash as it understands it wishes the client to use in the future (generally, sha256: URI) 5) stream restarts (or continues after step 1, if server responded with the EXI header for the same schema) normally

  989. arc

    the error-restart method should only be needed after a server is wiped, upgraded, or the first time a client of a specific version connects to it. sha256 is suggested to minimize this (large servers will already have the schema on file) but can be boosted in the future

  990. arc

    it otherwise uses the same framing as websocket.

  991. arc

    vs XEP 0322 it removes the issues with asking the server to download schemas from a HTTP resource (eg, using XMPP servers to multiply ddos attacks on webservices), removes the need for a text XML parser, reduces handshakes to initiate a typical connection, and removes redundant negotiation

  992. moparisthebest

    so it just sends a hash of the schema it wants to use?

  993. moparisthebest

    no other info about it?

  994. arc

    yes in the EXI header field for schemaId. i believe the hash URI standard allows for length too

  995. moparisthebest

    I was going to ask what stops a malicious client from uploading a 10gb schema

  996. arc

    if the hash isnt known by the server, it asks the client to transfer the whole thing, and then the server gives the client a URI to refer to that schema in the future - which might be a newer hash

  997. arc

    moparisthebest: the server should cut it off at some point obviously. schema should never be anywhere near that big, especially EXI encoded.

  998. arc

    i mean you could make the same claim for what stops a client from sending a 10g <stream:stream opening element with a gazillion attributes

  999. moparisthebest

    that is true

  1000. moparisthebest

    I wonder what current servers do with that hehe

  1001. moparisthebest

    or clients

  1002. arc

    with EXI? the few experimental ones use XEP 0322

  1003. arc

    i am not aware of EXI being used in production anywhere tho

  1004. arc

    the only complete implementation of EXI I'm aware of is written in Java

  1005. arc

    my libexi will be #2.

  1006. moparisthebest

    oh I meant I wonder what current servers or clients do with 10 gigabyte <stream:stream xml

  1007. arc

    oh, that's a good question

  1008. moparisthebest

    evil me wants to try it out

  1009. arc

    I'm willing to bet at least one will catch on fire

  1010. moparisthebest

    not at a production server of course other than mine :)

  1011. moparisthebest

    I'm guessing some are protected by a naive "no xml will contain > 10m so that's my buffer size"

  1012. moparisthebest

    or similar, but yea, testing time

  1013. arc

    well id bet actually that expat or libxml2 will dutifully attempt to parse it regardless.

  1014. SamWhited

    What is realistically the biggest packet size a server should expect? Not more than a couple of kilobytes surely?

  1015. arc

    SamWhited: with HTTP over XMPP it could be more. isnt there a way for a MTU to be set?

  1016. Kev

    Given the minimum maximum stanza size is 10k, no, a bit more than that.

  1017. Kev

    Depending what you mean by 'packet'.

  1018. arc

    i assume stanza

  1019. SamWhited

    yah, I don't know what I meant by packet… "start stream tag or any second level element" I suppose

  1020. arc

    amount of data in the XML parser which is not yet returned to the client?

  1021. arc

    er, application

  1022. arc

    moparisthebest: this is a good secure case to note

  1023. arc

    another issue servers might want to look out for is flooding it with new schemas. an LRU cache should be used to keep the number of schema from being pushed out of control by an attacker

  1024. moparisthebest

    it might or might not matter, but it could be a bit racy

  1025. moparisthebest

    like if 10000 iot devices all connect at the same time, request the same hash, server doesn't have it

  1026. arc

    yea disk size. but you can flood that with logs too

  1027. moparisthebest

    I guess they all simultaneously upload it?

  1028. arc

    that sounds like a crazy race condition

  1029. arc

    actually no, that'd almost never happen because each one has to be provisioned right?

  1030. moparisthebest

    it seems like it'd happen when you reboot the server or something though

  1031. arc

    i mean almost never happen that two try to send in the same schema at once. and one would hope the server can handle that well

  1032. arc

    oh, true. or upgrade it such that it wants to wipe the cache

  1033. moparisthebest

    maybe something like that

  1034. moparisthebest

    maybe you block the others while a few are uploading or something?

  1035. moparisthebest

    servers might be able to do something smartly

  1036. arc

    if a server policy is to, eg, use a SHA512 for added security because the operator considers SHA256 weak, even if it "has" the schema on disk it would need clients to transmit it in order to give it the hash that it wants

  1037. arc

    the schema shouldnt be large. thats why EXI encoding too.

  1038. moparisthebest

    I kind of assumed once a schema is uploaded the server would store it along with *all* the hashes

  1039. arc

    it could do that too.

  1040. moparisthebest

    anyway I'm off here for the day :) have a good one

  1041. arc

    so if a newer client asks for a sha512: right off the bat the server can respond "correctly"

  1042. arc

    all the server MUST do is return the schemaId it would like the client to refer to this schema with in the future. it SHOULD return with a hash URL, and it SHOULD record and handle any hash URL by any method the server considers secure

  1043. arc

    so that clients connecting to the server for the first time using the same schema as another client of the same model, can do so without having to send the schema first.

  1044. moparisthebest

    any reason it just wouldn't always use the hash?

  1045. arc


  1046. moparisthebest

    that seems like the only way you could be safe knowing you were both talking about the same thing

  1047. arc

    allow the server to support future hash mechanisms without clients needing to understand them

  1048. arc

    a client sends a sha256: URI. the server responds to uploading it with a sha512: uri. client records and uses what the server gave it. the sha256: URI the client started with a guess. if sha512: were to become a new standard every client could use it.

  1049. arc

    otherwise a client connecting to a server for the first time would just start with the default schema and send the schema in order to get the identifier. which could become a bit much.

  1050. arc

    in 2017 i think we all consider sha256 strong. 2020 who knows

  1051. arc

    this is just me spitballing though.

  1052. moparisthebest

    so maybe a server MUST respond with a hash, it MUST respond with the hash in the same algorithm the client sent unless it doesn't understand that algorithm, in which case it MUST respond with the hash in the 'strongest' algorithm the server supports as decided by the server

  1053. arc

    that has some odd implications too. the hash itself is added weight for every connection. if 256 is considered enough, it should use 256.

  1054. arc

    802.15.4 devices have an effective MTU of around 100 bytes, and over 6lowpan packet fragmentation can cause real connectivity issues. its best to keep the EXI-encoded stanza payload under 100 bytes

  1055. arc

    the exi header with a sha256 uri consumes almost 100 bytes by itself, iirc

  1056. arc

    if its just <open> though its fine

  1057. arc

    i imagine #futurehash is more likely to be used over 802.11ah or similar newer, low-power protocol though which isnt necessarily subjected to the same constraints

  1058. arc

    in some cities right now, every bus is driving around with a 802.15.4 transceiver in a weather-proof plastic shell and a tiny solar cell glued to the top of the bus, rechargable battery, recording and sending realtime air quality data through a makeshift mesh network using, IIRC, some MQTT-based protocol

  1059. arc

    since they use 2.4ghz the buses are regularly delinked from the mesh network due to excessive frame collisions and inability to return pings, so restarting a stream on reconnect while under pressure is a real thing

  1060. arc

    fragmentation multiplies the problem in those cases.