XSF Discussion - 2017-03-02

Damn, you've got me. I type my gpg password rather often. I can look up the other things for mutt tomorrow if you are interested

I'd rather know how to not quit mutt by accident all the time

Unbind the Q key

Whos brilliant idea was it to put quit and 'go back' on the same key anyways?

Zash: it's a sensible idea in general. Except when you want to "leave" a limit filter

I have my password saved in a GPG'ed file; mutt unlocks GPG on start to get the password, which also keeps the GPG agent unlocked for 15 minutes or whatever, which works pretty well.

Zash: for incoming mail, you can set pop_pass and imap_pass in imap, or even bind a key to a macro like "cimaps://user@domain:password@server/INBOX\n"

https://tools.ietf.org/html/rfc6778 and https://tools.ietf.org/html/rfc7017

That's so meta.

https://trac.tools.ietf.org/group/tools/trac/wiki/Imap

Zash: you might want to tell the XSF why you are pasting all the URLs in here.

Ge0rG: I'm sleep-pasting URLs I think

Zash: time to get coffee, then.

I've had my first coffee of the day at 0430 local time.

Anyways, the IETF seems to have gone through the process of figuring out better ways to access mailing list archives, so I'm trying to nudge people towards looking at the work they did.

Zash: I'm not sure how IMAP is going to help in that regard. It sounds to me like a mix of NNTP nostalgia and nerd cred.

Zash: I'd like to have a feature where you can search the ML by affected XEPs. So a kind of tagging.

And people write the craziest things into the Subject:, so you can't just /~s XEP-0123

if we could add XEP-xxxx tags post-factum, it would be great.

Makes archives predating your subscription accessible.

Zash: last time I needed that (and it was to correctly reply-to to a mail), I just downloaded the .mbox. I think that the number of people who care about that, outside the IETF, is small.

Zash: and the set of people who fail to import an .mbox into their MUA, but manage to connect to an anon IMAP is probably very small.

The underlying point is to look at what a similar organization did about pretty much the same problem.

Okay, I can buy into that

They did end up with a pretty nice search thingy.

Zash: I hope you don't mean "connect with imap, use your MUA search" approach

Ge0rG: https://mailarchive.ietf.org/arch/

Zash: it looks like a web MUA to me. I searched for "xmpp" and wasn't impressed with the results too much

OTOH, it looks like a MUA.

Oh Android. If you register your app as an Intent handler, older versions use "{handler_title}" as the display text, and newer versions use "Open with {handler_title}". I'm pretty sure I'm not the only one to find "Open with Add contact" a strange wording.

ah, that ietf-mailarchive-thing is nice

seen it a couple of times

I’m starting the writeup of the XEP-115 (Entity Capabilities) replacement. I have a few questions: 1. I would like to acknowledge waqas work and the work of the authors of XEP-115. How do I do that appropriately? The XEP-Template doesn’t have an acknowledgements section, but seeing that XEP-115 (and others) have one, I assume that’s an appropriate way to do it. Correct? 2. In the examples I will need a namespace. Where will I source it from? Should I use a namespace under my own control and the editor will choose a different one when the XEP is accepted as experimental?

Is this a replacement of 115, or an update to 115?

jonasw: there is no formal way for acknowledgements. Most authors just dedicate an entire section to it

Kev: replacement, you can probably work your way from http://logs.xmpp.org/xsf/2017-02-28/#19:49:01 upwards to see the discussion around that.

Just re-using 115 seems appropriate to me, you're not in need of drastically changing the protocol, are you?

(I note that other things like pubsub have dependencies on 115, so if you write a whole new XEP you're looking at patching a *lot* of XEPs to update those dependencies)

That's probably true

interesting point, noone seems to have thought about that the other day

a namespace bump for 115 would be less intrusive probably

A namespace bump, if needed, or maybe a backwards-compatible update (if possible) seem reasonable to me. But keep in mind it's not coffee-o'clock yet, and I don't even drink coffee.

backwards-compatible won’t happen. the algorithm (and I’m not talking about sha1 or something) is broken and in need of fixing for eight years.

I'm not utterly convinced that means it can't happen (forwards-compatible can't happen, certainly), but I'm not convinced it can, either.

i should probably announce coffee-o-clock now.

in my opinion, xep 60 doesn’t have a dependency on 115, but on 30. it’s just worded badly.

or rather, "in my reading" than "in my opinion"

from the amount new work I’m doing for it, an update to 115 feels more appropriate than a new xep, too

Kev, Steve Kille: Would MIX be interested in an atomic CAS for PubSub. For example to race-free replace the subject/topic/... of a node. I'm considering writing a CAS add-on XEP for PubSub.

what is CAS?

(I only know Computer Algebra System, which I assume you don’t mean)

jonasw: compare-and-swap

ah!

makes sense.

(or rather, tea-o-clock)

Flow: I feel that CAS will be hard to implement server-side. when do two XML subtrees compare equal?

jonasw: by node id

err item id

CAS?

okay

ah..nvm

Flow: CAS would be useful for data storage in PEP nodes, too

jonasw: It would be useful everywhere where PubSub/PEP is used

mostly everywhere :)

but yes.

and where you want to avoid accidentially deleting existing data because of a race condition

there are usecases where you add data instead of replacing by item id :)

I wonder why 115 didn't just use Canonical XML standard for c14n of disco to later hash it https://www.w3.org/TR/2008/REC-xml-c14n11-20080502/

Tobias: I was wondering about that, too, but I think canonical XML is strict with the relative ordering of elements

also I‘m not sure how many xml libs support c14n; considering that there are *still* some in use which don’t do namespaces properly

could be, yeah

jonasw, you're aware of this thread, right? https://mail.jabber.org/pipermail/standards/2011-August/025011.html

not yet

jonasw: which usecases are that?

Flow: microblogging-ish :)

ahh right

jonasw, it discusses a lot issues with current XEP-0115, that should be solved in a new version

Tobias: thanks!

I’m looking into it

jonasw: Also https://wiki.xmpp.org/web/XEP-Remarks/XEP-0115:_Entity_Capabilities

I was also planning to ask standards@ for input when I have a first draft

Flow, what? the IANA has two registries for hash names?

Tobias: Yep

that’s a good point; the one we currently use doesn’t list sha3 for example

I discovered that when searching for a registry for ISR-SASL2

Flow, einmal mit profis :P

Tobias: Hehe, to be fair, that could happen to the XSF too :)

Flow, nah...we'll only ever have XEP-0300, which can be updated relatively easy

i think IANA stuff requires lots of time and process

If someone knows if and whom we should tell about this within the IETF/IANA, then please do so/tell me.

Link Mauve: BTW, SASL2?

does anyone know the rationale for querying a specific disco-node containing the hash in the verification procedure xep 115?

jonasw, what exactly do you mean?

example 3 here: https://xmpp.org/extensions/xep-0115.html#discover

node='http://code.google.com/p/exodus#QgayPKawpkPSDYmwT/WM94uAlu0=' instead of simply querying without node. is the idea to avoid races with changing capabilities?

hm, it mentions "backwards-compatibility"

for avoiding races it seems helpful, why was it abandoned?

(even though races wouldn’t be harmful here)

jonasw: so that you get the result of that very same hash?

yes

that approach seems sensible to me

could also help with server side caching i suppose

Flow: I don’t like the approach though, from an implementers point of view

e.g. Smack also responds to the last 10 hashes

jonasw: I do like the approach from an implementers point of view

Flow, you keep a history what sets of features the last 10 smack releases supported?

Tobias: No, disco features are dynamic, not tied to a smack release

Flow: there is no harm in a race here, because if you get a race with an unknown hash (if you know the hash, you don’t care) you simply get the updated disco#info and discard the hash.

so the last 10 features of the connection

that yoo, yeah

jonasw: true, no race here, but it helps with other things, like tobias said, server side caching, and I think it's the cleaner approach

how does it help with server-side caching?

jonasw: The server can cache the response

hm okay

and send it instead of forwarding the request to the queried client

jonasw, the server doesn't need to forward the IQ to the to-JID if it knows the from-JID just wants the disco#info for a hash

makes sense

it could reply directly

seems like using a different format for these nodes would be great though: '{ecaps2-namespace}#{hash-algo}.{hash-value}' or something along those lines to make it easily recognizable

right now a server needs to track the 'node' exported in <{caps}c/> to know whether a disco-node is a caps hash

*belongs to a caps hash

is there an element I can use to link to another section in a XEP?

except <link url='#anchor'/>

IANA has *no* registry for hash names. IANA has several protocol registries to cover parameters for hashes, some of these are strings.

dwd: that makes sense and explains the odd titles for those registries.

like "Named Information Hash Algorithm Registry"

We co-opted one for our purposes in XEP-0300, but it's originally for PKIX, so it contains OIDs as well.

Maybe we should also allow urn:oid:2.16.840.1.101.3.4.2.1 for SHA-256?

no

no no no no

dwd, although that one hasn't been updated since 2000something

oids are a mess.

jonasw, How can you say that? They're terribly convenient stable identifiers. Even if Surevine only has one OID arc (Isode has two - snazzy).

ugh, the names in xep-0300 are longer than some base64-encoded hash values themselves…

jonasw, what names?

dwd: as long as you don’t need to parse them semantically, it’s fine probably, like urns

Tobias: <var> <name>urn:xmpp:hash-function-text-names:md5</name> <desc>Support for the MD5 hashing algorithm</desc> <doc>XEP-0300</doc> </var>

yeah...that's so people don't used md5 :P

jonasw, Oh, the feature names.

jokingly

well, close. >>> len(base64.b64encode(hashlib.sha256().digest()).decode("ascii")) 44 >>> len("urn:xmpp:hash-function-text-names:sha-256") 41

jonasw, Well, that's a reason to use SHA3-512, then.

my python cannot into sha3

hm, 3.6 can’t either…

#sad

but that looks like a configuration problem; it also doesn’t have BLAKE2b512 which is available in 3.5 here

jonasw, 3.6 can do sha3 just fine

Tobias: did you mean <sad/>?

mathieui: yes, it appears to be a problem with my python3.6.0a3 probably sourced from debian/experimental

jonasw, nah..i mean the trumpish hashtag sad ;)

jonasw, so 3.6 doesn't have blake2 but 3.5 has?

Tobias: or rather xep-14 <x xmlns="jabber:x:tone">sad</x>? :>

Tobias: as I said: it’s most likely an issue with my local setup, the documentation says it is there:

https://docs.python.org/3/library/hashlib.html

Tobias, 3.6 has blake2 as well

nice

meh, short names for the functions in xep-0300 would be great

or am I just missing those?

jonasw, The long names are only used in the disco#info, right?

dwd: it apperas so

jonasw, The actual use in protocol are short names, like "md5".

dwd: but there doesn’t seem to be a registry or source to refer to on which short name to use for which function.

jonasw, table 1 has short hash function names

for some, yes.

see the sentence before the table

it is lacking sha3-{224,384} for example

even including that sentence

well yeah..didn't see much sense in those intermediate values

fair point

re-using 0300 makes a lot of sense

the standard should probably be 256bit ones, and if you need more security, might as well go to 512 bit then

hm, would making new hash functions mandatory trigger a bump on the <hash/> element…?

that sounds like a *lot* of fallout.

jonasw: why should it trigger a (namespace?) bump?

Flow: I don’t know. I’m asking.

*couch*Flow logo*couch*

Kev: out of curiousity, what software are you talking about in your mail from 09:57+01:00?

i just assumed that mail was some weird welsh humor :)

jonasw, I suspect it's mailman...

as we're all here: Does any more need to be discussed regarding https://github.com/xsf/xmpp.org/pull/269 ?

or rather: my merging of it?

dwd, a new version of mailmain you mean?

or the current mailman?

Tobias, No, I think it's just whatever we're using now. I suspect there might - might - be sarcasm at play here.

Guus: FWIW, github has a review feature, and it may make sense to have one or two eyes confirm that they took a close look on the changes, possibly leaving comments.

dwd, never seen him use that before though

Tobias, No, it's unusual in those who are cursed by not being English.

dwd, you misspelled 'blessed' there

I had to change my editors dictionary to en_US (from en_GB) to write XEPs :<

What? Why?

because XEP-0134 (or -0001?) says so.

Sounds like a candidate for a PR, then. :-)

https://xmpp.org/extensions/xep-0143.html#nt-idp1712848

jonasw: I don't disagree, but as far as I know, that feature is not used by XSF. We could, sure. I don't feel that there's a need for it here (the consequences of missing something in a PR review are very unlikely to be catastrophic for our website, and I prefer a continuous release cycle), but I accept that others think differently.

Guus: it’s really low-entrance-barrier though (if you’re a github user), and I don’t mean that it should be *mandatory*.

jonasw: I'm using it for other projects. Not knowing when to use it appears to be my problem. :) I thought your PR was fine.

have you checked I didn’t slip in a try: shutil.rmtree("/") except: pass in? :)

I am assuming that you thought so, because you PR'ed it in the first place.

I’m new in the XSF, my word shouldn’t count a thing when I add code to servers.

Oh, you could have slipped in things. I recognized your name, I glanced at the code, I ran it locally, it had the desired effect and did not delete my root partition. That combined made merging the PR an acceptible risk for me.

:-)

I’m just saying that I completely understand the point of people asking for thorough reviews. I would do the same if it was my infrastructure.

Who am I to object to thorough reviews?

I think mine was thorough enough by my standards, but I am fully aware that others have different standards.

I think there's a significant difference between 'updating text on the website', which I'm fine with people generally having access to do. And "running code on our servers", for which most people don't have rights.

Guus, i agree though that i should probably have left a note in the PR that I was planning to review it soonish

Running code that people thought was fine, but wasn't sensibly vetted caused us to not take part in GSoC last year, and huge amounts of wasted effort for me in the process, not to mention the downtime of the server so the XSF couldn't fulfil its primary purpose for a day.

FWIW, the pelicanconf.py file (the only one, as I understand it, that is executed on the server) looks perfectly safe to me and adequately simple.

It also looks clearly bounded, in as much as I can solve the halting problem in my head.

dwd, as far as I know https://github.com/xsf/xmpp.org/blob/master/buildCompleteWebsite.sh is run to build the whole website on the server

I think the more crises someone has been through with production servers, the less blazé they get about deployment :)

because pelican has very limited capabilities

Anyway, I don't object to the PR based on the description, I just don't want any code deployed on XSF servers that hasn't been reviewed by iteam.

Tobias: it can do anything python can if you put it in the pelicanconf :>

Kev: I've been a production herding developer, professionally, for 10+ years.

Guus: And how many times has pushing something without checking it caused a day's worth of downtime for you? ;)

jonasw, probably

including websites that have significant amount of views (millions, monthly)

Kev: I did check.

Kev, I think you may mean blasé, rather than blazé.

dwd: I very much do.

Although there's an argument for either.

Guus: Then I have no objection. Your original comment didn't mention that you'd reviewed the code, just run it locally.

jonasw, So, not threading, then? :-)

Well, I still have an objection in principle, because I think the server admins should get to review the code too, but I'm happy in this instance if you've reviewed the code.

Kev: I'm pretty sure I did not review it up to your standards. I'm also not worried by that.

dwd: depends on the specific python implementation and the specific task. Python can very much thread in the sense that C extensions which are called from python code from different threads may in fact run in parallel. It is just pure python code which, on CPython at least, isn’t run in parallel. :)

Kev, This is build-time code, incidentally, not runtime code. So I'd hold it to lower standards.

jonasw: "Python can totally thread, as long as you code in C instead of Python"? :)

jonasw, Yeah, I'm only too aware...

Kev: pretty much

dwd: When it's run on the server, I'm not sure the standards need to be much lower. If it's malicious, same effect, if it manages to resource-starve and bring down the server, same effect. There are some runtime cases (resource-heavy, but not resource-starving) that don't apply, but the standard's still pretty high.

actually, this is why in the organisations I use pelican, the build system and the contents are separate repositories. The build system repository has strict review requirements, content lesser so.

(although, fun fact: pelican lets you write to arbitrary files from the content files alone :-))

(well, the current master branch doesn’t anymore)

jonasw, templates probably still can though, right?

not sure about that, but I don’t consider templates content.

Anyway, my opinion isn't going to matter for long. My new games PC has just arrived, and Cath is going to kill me as soon as she gets home and sees the den.

heh :)

You have time for a games PC? *envy*

Sure. It just sits there, it doesn't need much time.

Now playing games, that would take more time...

I would like to re-ask my question now that more people are active. When writing a new XEP, in the examples and specification I will need a namespace. Where will I source it from? Should I use a namespace under my own control and the editor will choose a different one when the XEP is accepted as experimental?

which of both is what will get you killed later today?

jonasw: It's easiest for the Editors if you use an appropriate NS from the start, although technically IIRC the Editors should pick one.

okay

Stripping out your NS to replace it with an xmpp one at publication time is mostly busy-work.

And while the other Editors are much less lazy than me, stil ... :)

ack

just wanted to make sure that I don’t overstep any boundaries by suggesting a namespace from the xmpp-urn-namespace

jonasw: It's slightly tweaking the process, but it's the sensible thing to do, and what everyone else does.

Guus: The mess, and that I'm not intending getting rid of my old games PC, but running both in parallel both run the risk of death-by-spouse.

Kev: in which case, I am glad I had the chance to meet you in person at FOSDEM, before your premature death.

is there any precendent to form arbitrary (i.e. entity controlled) disco#info nodes from an urn:xmpp:-namespace? so for http://… namespaces it’s obvious to use # as a separator, is there any precedent what to use with urn:xmpp:-namespaces?

I'm afraid I'm too stupid to understand the question.

jonasw, so you want to have dynamic namespaces, not previously defined in a XEP or registry?

not namespaces, but disco#info node names

nah, I’m too stupid to formulate it clearly. see in https://xmpp.org/extensions/xep-0115.html#discover <query xmlns='http://jabber.org/protocol/disco#info' node='http://code.google.com/p/exodus#QgayPKawpkPSDYmwT/WM94uAlu0='/> the node there is composed of a URL base and a hash value.

I don’t see the point of using some client-provided string as a prefix so I would like to use the namespace of the XEP as prefix. what kind of separator makes sense between the prefix and the hash info? Is there a precedent for that?

ah yes, it appears so

xep 290 also uses #

the argument that won me over on not allowing clients to dictate their resource was that of distributed hosting routing

you mean clustering?

sure, whatever term you want to have for a @server hosted by multiple servers. and sorry i completely misread the conversation above, so that statement was kinda out of the blue

I'm still not convinced of that clustering use case. "Google does it this way" doesn't cut it for me.

Ge0rG: we're going to need it for IoT

Ge0rG: Well, I guess it'd be interesting if you could explain how you solved it in your clustered server, to persuade the other clustered server vendors that it's easy?

Kev: wait, let me fire up a bunch of dockers.

right now prosody can effectively handle 40k concurrent users on an average AWS instance last i ran the brute force test. in order to scale to the size that some of these IoT manufacturers want you need multiple servers, ideally geographically distributed

arc: what about running different per-region domains?

the last sit-down I had with an IoT manufacturer they said 10m units is what they consider base level, and any solution they consider should be able to scale to ten times that

are there any xmpp installations handling north of 1m connections? I only remember WhatsApp's we-are-awesome post in that regard.

really wonder if all those IoT devices need permanent connections

per-region domains is changing the security model. Also, it means if I live in the US, but I travel to China, I'm still connecting to my server in the US (or whatever domain I registered on). We were talking about single domains, multiple-domains is a completely different thing.

Tobias: of course they do!

Tobias: for receiving input, yes. though they're not very active.

Ge0rG: I can't give exact figures (and don't know them anyways), but I'm pretty sure we (HipChat) are.

(and we also use the server-assigned-resource-part-for-routing solution, FWIW)

FWIW Prosody's clustering will use the resource for internal routing purposes

in one case a device wanted to send a "heartbeat" with 12 bytes of data every 6 seconds (1/10th of a minute)

arc: that's a very intensive use case

Ge0rG: yes, and each device having a retail price of around $15

that's the future we face and have to plan for

I like that Arc has such a high opinion of our maths that he had to explain that 6 seconds was 1/10 of a minute :D

Kev: sorry i haven't had my tea yet lol

tea <3

I for one wonder how many seconds 2/10 of a minute is.

I will readily admit that a 100m service blitzed my brain out. I mean, sure we can toss around big numbers like its nothing, but that's actually some significant engineering challenges.

arc: It undoubtedly is.

at that rate you need dedicated S2S routers. and questions like where are the heartbeats routing to

I could also imagine that a 100m IoT deployment has different requirements than a public chat service

(and also probably different sysop challenges, where having a resource string as a debug tag is less useful)

absolutely. from the EXI side those stanzas are extremely small. as long as the 12 bytes of data are encoded in int or float attributes within their custom schema, the whole stanza could be around 16 bytes. and since the devices will be communicating with a finite number of other devices, mostly on the same LAN..

my recommendation was embed their XMPP server in their 802.15.4 to wifi gateway module, to keep a majority of the traffic local and reduce their service end traffic as a first point. which i think is what they're doing

Ge0rG, client-provided debug tags aren't guaranteed to be unique, I'm really unconvinced by your argument

I've also come to the conclusion that agreeing to compromise on that basis was a mistake… if you were using the resource part as a debug tag you were using a quick hack; if that's a thing we want, we need a real solution, we don't need to make a part of the JID more complicated just so someone can see sometihng in existing logs.

Adding stuff to the JID that isn't related to routing is changing the purpose of JIDs, and that feels like a bad idea.

MattJ: a properly implemented client can provide sufficient uniqueness.

Ge0rG, you're not a server developer, clearly :)

As a general rule of thumb I don't think we should ever have to rely on a "properly implemented" client.

Indeed

SamWhited: from the EXI side it doesn't matter. the entire JID is one string in the string table. i think having a human readable (aka designed for the UI) resource after the # makes some sense. though, that could also be done through pep

MattJ: but I know a little bit about client development

at this point I tend to agree with SamWhited. for debugging, there really should be something else, like an additional optional stream header which can be used for debugging, or a stream feature to attach a debug identifier to a stream or use <identity/> as soon as it’s available.

Ge0rG, it's a nice idea, for you, with your client. But in the real world, on a real server, we can't depend on every client being Yaxim

wouldn't this make sense to attach to PEP?

I totally get why you want a debug tag, and let's do that. But I think it's separate to the resource

Or just some form of fingerprint the server constructs (so that the client doesn't have to do anythihng), eg. maybe it queries the client for its disco#info, and then hashes that along with the JID and any other info it can get and uses that to track sessions

arc, no, because PEP is per user, not per client

MattJ: couldn't the PEP .. sorry still early .. list a resource to human readable lookup?

SamWhited: for a single session, a server can just roll a random number.

MattJ: let me rephrase your suggestion: let's create a nice perfect future debug tag sometime in the remote future, and remove the existing and working debug tag right now.

jonasw: ah, yah, I guess this is about tracking clients, not sessions. oops.

The existing and working debug tag that breaks more critical parts of the system and makes everything more complicated.

for clients use <identity/> as soon as its available and log it to associate the identity with the session nonce in the logs.

identity + bare jid probably

And requires that clients do a specific thing which they may or may not actually do.

Ge0rG, given that you're currently the only person I've seen suggesting that the resource string can and should be used this way, I don't think we're anywhere near your ideal being reality either

i.e. other clients don't use the resource this way, you do. You'll update to use the debug tag, they won't

I remember at summit people complained that identity couldn't be used for this, but I don't remember why? What jonasw suggested sounds sensible, and works today.

I see that the resource is *currently* a nice way to track a client in debug logs; but BIND 2.0 won’t be there tomorrow. There’s plenty of time for server devs to adapt. This could easily be part of the UX considerations for sysops in BIND 2.0

(s/BIND/Bind/?)

I'd be fine (and glad) to include some kind of unique client identifier in bind2

MattJ: I don't know how many sysops of public servers are active in this MUC

or even include a "debug identifier" in Bind 2.0 which is never ever exposed to anything but server logs. although I think a stream header would be nicer because it allows tracking even before authentication succeeded.

ha, MattJ beat me to it

and with bind1 clients, use their provided resource as a cookie, and then use something else for the actual resource

What is it with you and writing lots of text while I'm out on a walk?

(sorry, cookie == debug tag in my mind)

MattJ: makes sense

sounds like a very useful way forward

Zash, you should take your phone, to make sure you never miss a message!

I did, for photos of all the snow ✏

MattJ: I want to be able to easily grep my logs for certain things, and to get all traffic exchanged with a given client instance (including re-auth and 0198 resumption)

(this discussion also pins me to a chair in a waiting room where I wanted to leave 20 minutes ago, but whatever)

phone? he should have always wear Glass so this room is constantly flowing above his eyeball

MattJ: or to get all traffic exchanged with a certain client software.

Zash, how much ❄?

Ge0rG: I think you actually want structured logs

I want to submit pull requests to all other clients to change their default resource string to "yaximXYZ"

cramming all those criteria in a single string isn’t doing any good

My position on resource selection is that the rules in xmpp-core are fine and don't need changing.

I agree with SamWhited that something else ought to be used for this kind of tracking and debugging.

Ge0rG: Would it satisfy you if we returned the log tag in the handshake somehow?

Zash: the rules in xmpp-core are sufficient indeed. As long as the server doesn't override what the client sends ;)

the more i think about it, the less i think about this as an issue of debugging, but more of the use case where you want your contacts to be able to specifically reach you on your laptop vs phone vs whatever

that was brought up at the summit, i dont remember by who

The rules in xmpp-core would be fine, except that if you let clients "set" a thing, they're going to stop reading the RFC at that point and assume that's the JID they get. In my mind the rules should be "the server sets the resource part, it's opaque to clients, and the clients get no say in it"

arc: That is doable via disco#info

Ge0rG: what about the following: 1. bind 2.0 allows for a "debug tag" 2. servers are strongly encouraged (via UX considerations in the bind 2.0 xep) to include that debug tag to every log message related to that client ?

SamWhited: The client gets to make a suggestion, but the server decides. Similar to how extensions and stuff work in TLS.

Because it's for *routing* which is strictly a server concern.

Zash: Yah, I wouldn't mind that, except it seems to be a source of bugs because clients don't actually pay attention to the servers decision

Or at least, that's what it sounded like at summit.

SamWhited: most client authors AFAICT don't write to the rfc, they use it as a rough guide and really write to a server

SamWhited: there is still no consensus on whether that _routing_ info should be persistent for a given client instance or not.

arc: And that's how we get "but it works in Internet Explorer".

Ge0rG: Sure, but that's orthogonal (and probably up to the server / service)

arc: Indeed :(

SamWhited: actually it's related, because the client is the only one that knows its identity on a reconnect

there should be a way to pain to those who do that, arc

Ge0rG: Have you thought about my suggestion of including a namespaced attribute on the stream header? That's greppable in logs, which gives you the sessions log tag, which you can then grep for.

jonasw: a network testing script which tests a client or service for compliance

starting with "fun" things like sending <stream:stream version="2.0">

Are there any security issues with using the stream ID as tag in logging?

Ge0rG: Ah, yah, fair enough, I guess you can't really separate that from the clients control.

and using custom prefixes.

Zash: I want to reduce the number of IDs, not increase it.

just basically go through the RFC for every MUST and SHOULD, write a test for that case, and MUSTs show up as red, while SHOULD appears in yellow - any client failing to (eg) accept a different resource than requested by the server would show up this way

and if you provide it, and its something client authors can find, they will almost certainly use it.

Sorry, I'm in a meeting currently, and I'm heavily sleep-deprived. Can't focus on the discussion here.

arc: FWIW I don't think the client needs to know its own resource in that many cases.

sure but can you think of a case where a client not understanding its resource correctly would cause a fault that you could test for on the server side?

Strip out the 'to' attribute on everything you send, see how the client reacts.

as a client, I don’t care about the to a server sends me

yea isnt it legal to do that?

No 'to' attribute is supposed to be semantically equivalent to to=full JID

i mean i guess you could test an iq ping addressed to nobody, to the client by a random resource, to the client's requested resource, and to the client's given resource

Or the bare JID in the other direction

replying to a ping that's misaddressed should at least be a warning, tho in that case it'd often be hard to say whether it was understanding its resource correctly or not

but if it only replied to its requested resource but not its given resource..

Isn't that an error on the servers part?

Zash: test servers must send bad data. thats the point.

There's been a bunch of security issues related to not validating the 'from' on certain stanzas, like roster requests and such.

the point of a test suite isnt to test whether a client behaves correctly with typical data to a properly functioning xmpp server. the point is to test whether it behaves according to the RFC, so in many cases the client would - i assume - need to close the connection and reconnect.

yeah, but from is not to

or send an <iq type='error'> or etc

i mean i above proposed one of the first tests would be <stream:stream version='2.0'> to check that the client is actually parsing the stream version according to the RFC. it should reject the connection, right there

arc: https://modules.prosody.im/mod_conformance_restricted.html may be of interest to you

Zash: i'll look at it

but does it send intentionally bad data to test?

I have a utf8 test suite I'd *love* to see how both clients and servers respond to

Yes, sends XML things forbidden by the RFC

sending PIs is bad data i guess :-)

damn i need tobunload csi

*to unload

Zash: have you tested for UTF-8? what happens when NULL is in the middle of a stanza, say in the <message><body>? or ending a <message><body> with a chr(148) followed by </body>

arc: Have we had the conversation about IDNA versions and PRECIS and how the only reasonable thing to do is crawl down under ones desk and cry?

Zash: no but it sounds like a conversation id love to have ;-)

Heh, this is true.

arc: yay! please tell me if Unicode Robot Face (🤖 U+1F916) is a legal resource character

and Unicode, and UTF-8, and natural languages

Ge0rG: I don't know but i'd love to find out!

I'm almost certain it is; I can go check if you really want.

i discovered that GNU Screen has some deep UTF8 issues, as does Synergy

I started digging in and found lower level libraries were at fault

GNU Screen only handles 1 and 2 byte unicode

internally it was using UCS2

Like how MySQL has something called "utf8" which only supports up to 3 byte UTF-8 sequences?

heh

Yup, it's valid

i think SamWhited cheated

arc: GNU libidn and IBM ICU behave differently when given Unicode outside of Unicode 3.something or whatever was state of the art at the time. One accepts. One rejects. Much fun.

https://gist.github.com/SamWhited/cc6fd0a9c0a1559c71f828f6b6c8b729#file-validjid-go

That JID implementation is using a very well tested PRECIS implementation that's built with Unicode 9

Mr Miller *IS* in the DC area, we're setting up a time for coffee

^5

Now I wish I could have Robot Face as a sRVname SAN in a LE cert

Ge0rG: Nice things, they are unobtainable.

Zash: like Unobtainium?

Oh no, Unobtanium is much more attainable than nice things.

Bummer.

BTW, why is the Board Meeting over now?

It was the board meeting to end all board meetings

Zash: I think it only ended three of them.

lol

so today's joy on the FLOSS Foundations mailing list is the announcement of the new Open Fashion Foundation, quote, "to disrupt fashion industry with lessons learned from computing industry."

Aaaawhat who let this override browser shortcuts?!

So they're going to spend all their time adding new features to cloths and ignoring the fact that the cloths are unraveling and falling off?

SamWhited: lol

this is one I won't even be synical about. Its just a pure bundle of joy that someone out there has made FOSS licensed fashion a personal mission in their life

ahem, yes, sorry about that. I mean, "good for them" :)

can you imagine a fashion show hosted by this organization?

The latest in beard and ribs fashion?

"This piece by Manuel Debrough, available under the Apache 2.0 license from github..."

Zash: oh no, dollars to donuts I'm willing to bet a fabulous gay man is behind this.

heh, I have a bit of a guilty pleasure in that I really enjoy fashion stuff (even though I know nothing about it, which is probably obvious if you've ever seen the way I dress), so that actually sounds pretty nifty

But I do enjoy seeing the things people come up with

actually I can see them trying to QueerEye geek's tshirt and jeans

aww yeah, I'm gonna be fashionable for once

the rugby club I started 4 years ago in DC just raised over $2500 in one night hosting a drag show.

https://goo.gl/photos/XEKE5peqYG2b4gfb7

when I mentioned this on IRC, one of my friends with the Gnome foundation immediately said they needed to run a drag show, and had people volunteering. The thought of that alone is priceless.

so yea I can see a geek fashion show, especially in san francisco

they could raise thousands for charity too

I can see "designer-stained t-shirts" and "artful crumpling" becoming a thing.

Hah, indeed. I'm going to start a new line: "morning coffee spill"

"Bob wears jeans (model's own) and a t-shirt (free from some conference)"

dwd: have you ever watched queer eye?

arc, Can't say I have.

gah I hate that, I have jeans with holes worn in them by myself by working before that was in fashion, and now I don't want to wear them for fear of people thinking I'm trying to be fashionable...

https://www.youtube.com/watch?v=g5dZ4QG7dW0 most of the men they makeover are shaggy geeks. they turn them metro. in almost every case the man starts with tshirt and jeans, and they end up posh with a new haircut, product, etc - also with their house/office made over.

hipster moparisthebest was into jeans and t-shirt's before they got all popular

:(

moparisthebest, You're way older than I thought, then. I recall holes in jeans being fashionable, and that was when my mum bought me clothes.

I seriously still wear the same jeans and t-shirts I wore when I was 18 and stuff, my wife tries to throw them away all the time lol

arc, See, I don't need that. I *can* dress up. I just usually *don't*.

dwd, oh maybe it went out of style and back in, or I just didn't know about it, I'm 31 :P

https://youtu.be/g5dZ4QG7dW0?t=11m25s is where they bring this one guy to buy fashonable denim to replace his "jeans"

dwd: nor I. but its a great visual

this is more like my husband and I: https://www.youtube.com/watch?v=kbf_nFtA8YQ

moparisthebest, Yeah, I'll be 43 soon, and I suspect my mother was telling me ripped jeans should just be replaced at about the time you were born, then...

its funny, i have a tshirt and jeans policy - and have gotten a lot more traction with it than otherwise.

also the beard. the bigger the beard, the more they think you know. John "Maddog" Hall taught me that trick

that’s some unexpected backlog

so much text, so laggy connection.

Ge0rG: barely worth it if you’re not into fashion. most likely not worth it on your 30% loss link there.

the link already feels like 20%. Looks like it's improving. I even have sub-second latency.

Maybe I should fire up Gajim to see how it behaves with MSN and high-latency links.

heh

If I have the joy of reading about Open Fashion Foundation today so should all of you ;-)

is there a section in a usual XEP where I can put notes on alternative variants I considered but eventually decided against? much like PEPs have, for example here: <https://www.python.org/dev/peps/pep-0448/#variations>? otherwise I might add a Design Considerations section…

jonasw: +1 for Design Considerations

that sounds right to me

# requirements it needs to do the thing # discussion we could do something, but that has these problems we colud do something else, which seems pretty good, so the rest of the spec is about this

I think that every XEP should contain its rationale.

+1

yESSSSss

Zash: hm, PEPs do it differently: requirements, then spec, then other variants. I actually like that, because when I implement something, I don’t need to read the other variants. If I want to know why the other variants were rejected, I can skip to that section. thoughts?

No it should start with the schema! :)

ah, I wish one could rely on schemas in XEPs.

my ideal documentation would just start with already written code :)

moparisthebest: no.

in the language I'm using

and it has to magically know that beforehand

yea I'm joking sorry :)

:-)

I agree with you about that PEP order jonasw

Language Specification: What the code does is correct. EOF

right :)

:D

if you think you found a bug you are mistaken, it's actually a feature

#php

and it's apparantly worked for xep115 for 10 years right?

Is fine, don't worry

... why did I automatically read what Zash just said with a russian accent?

https://www.youtube.com/watch?v=rp8hvyjZWHs (Trust me, i’m an engineer !)

Hm. I need to youtube-dl that so I can watch it. ETA: 12:51

don’t.

some of those things are actually awesome

the backhoe rowing the boat for example

jonasw: alternatively, you could stream it to the MUC with libcaca and LMC.

Ge0rG: my client cannot into LMC

I'm sure mathieui would be glad to provide a video streaming plugin for poezio :D

Tobias: you mentioned earlier that a server could cache xep115 responses for those specific disco#info nodes.

I wonder whether that’s a great idea after all.

I was wondering whether it has any privacy implications for a client.

(on behalf of whom the server is answering)

jonasw: You may be able to guess that the server has seen a disco#info before through timing

Zash: well, yes, but lets assume that a server has seen that disco isn’t revealing anything, for example because all servers use the capsdb.

I wonder whether it would be okay for a server to reply on behalf of a client if the client is not actually online. While that would prevent any unintended presence leaks if the server answers for a resource which would by itself not have answered to that specific asker, it has the downside that stuff may be confused if a server answers a request for a resource which isn’t even online.

jonasw, as long as you have not an extremely user specific client feature set, that shouldn't be a an issue

I don't think it's a problem because it's generally up to the server to enforce permissions / decide who can query what anyways, not the client.

So your server SHOULD be taking precautions to prevent presence from leaking anyhow

(or whatever is being queried)

what are the criteria for an xsd to appear here? <https://xmpp.org/schemas/>

NoooOooOOOooo! [download] 87.6% of 3.35MiB at 45.18KiB/s ETA 00:09ERROR: unable to download video data: [Errno 104] Connection reset by peer

Ge0rG: youtube-dl can resume :)

Ge0rG, it supports resum...

:)

I should know. Are you using my wifi by any chance? :)

MattJ: You have wifi?!

Too many complaints from "smart"phone users in the house to resist any longer

MattJ: free WiFi on a crowded train, moving at 200km/h ✏

kind of amazing that works at all

jonasw: https://youtu.be/rp8hvyjZWHs?t=2m37s has got to be the best hack I've seen in a long time

:D

arc, the rowing backhoe? yea that impressed me the most

yea..

there is no arguing with that one, boat motor breaks, have a backhoe on board, it's ingenious

i thought my use of a toilet fill valve in a bucket for plant watering was good

I don't usually have a backhoe on board

this is a whole new level

Zash, So what do you do if your motor breaks?

probably something boring like an oar

I guess I would have to convert it into a putt putt boat

I would also have to get a boat and a motor...

what if you had a car onboard and could get it up on jacks

change out the wheels for paddles like an old river boat?

arc, If he doesn't even have a boat he's got worse problems.

arc: Like this (sort of)? https://www.youtube.com/watch?v=dyBl9vf8Td0

thats true. Zash how will you hack up a boat to start with?

But why would I have a boat? Not really a water person.

I'd rather have cabin in the woods and some potatoes. Backhoe would come in handy then.

Oh, I *really* doubt that you want to have cabin in the woods

https://www.youtube.com/watch?v=NsIilFNNmkY

true, there are nicer things than a portal to a demi-god-demon

Can't be worse than the mosquitoes

i'll take mosquitos over the horrific monsters they send to kill you

and what rises if EVERYONE fails

I think I'd prefer the things I could kill with guns

I think the scene of the japanese school children circling around and dispelling the demon is the best

https://www.youtube.com/watch?v=IIE8Fq4Zm1E

"The spirit of the demon will now live in the happy frog!" ... "How hard is it to kill a group of 9 year olds?"

this has been an odd day in the xsf, went from talking about fashion, to boats rowed by backhoes, to demons in cabins in the woods

blame me.

with some xmpp sprinkled in :)

yea there's XMPP involved, that's all that matters. That means we can charge lunch to the corporate card right?

after all the work I did I realized this morning that the hash function isnt likely all that useful for embedded systems, and in 95%+ of the cases won't even get included in the binary

embedded xmpp is unlikely to include text xml.

It ain't no fun with the lags.

the hash function is used pretty much, if not entirely exclusively for hashing text strings in order to find a cooresponding match on the string table

anyone else have a problem that you dig too deep into a problem that you lose sight of the big picture?

oh yes… frequently.

When I dig too deep into a problem I always encounter sub problems to which there is no documented solution on the Internets, but often many people having the same issue.

Don't get me started, today has been one of those

i hate that.

I still got some glibc in my eye from yesterday.

or you dig deep enough that you realize its a problem caused by the language you're using that can't be fixed, just.. worked around

e.g. the moment when I realised (after putting log statements all over the place) that the testing tool I was using was broken, and connecting to the wrong server

(in production)

Why isn't getrandom() in glibc until like the latest bleeding edge version nobody has?

and the rabbit hole just goes deeper

and now I'm just looking for some utility that will read lines from stdin and send them somewhere as UDP packets

and trying to pretend I don't need to write my own

netcat

netcat failed on the "line" part

my first "in office" job had two charming things; 1) a ban on coffee in the office (only green tea, because of management philosophy hogwash), and 2) "Eat Me" cookies in a sealed container in the break room for when you get trapped too deep in a rabbit hole

it took me far too long to realize the reference

hah

a also found that for every schema i could think of, bitpacked EXI is better, faster, and smaller binary than compressed EXI

i didnt expect that.

that's just a type of compression though isn't it?

What's compressed EXI?

like it'd probably be equally susceptible to CRIME / BREACH type attacks?

I guess you could call bitpacking a form ofcompression..

Zash: so there's 4 modes for EXI; bitpacked, simple byte-aligned, pre-compression, and compression.

byte-aligned is essentially the same as bitpacked but always padded to byte alignment, obviously

Can you explain them in terms of ASN.1 encoding schemes? :)

compression is pre-compression plus DEFLATE

(that was a fun rabbit hole too)

so which ones are secure under encryption? only pre-compression?

pre-compression is byte-aligned, but with similar types of data grouped together on the stream. so eg all int values are together, all string values together, etc

i wouldn't propose to know the answer to that moparisthebest

MattJ: socat READLINE: UDP:?

arc, probably should have someone figure it out before starting to use/promote it though?

but the idea with pre-compression is that some form of compression will be applied on, eg, the TLS layer

jonasw, I saw that, but READLINE seems to actually involve the readline library, i.e. it's intended for human input, not piping from another program

MattJ: and STDIN doesn’t do the trick? :/

*STDIO

arc, I think most if not all TLS libs removed support for TLS level compression because it's woefully insecure

moparisthebest: i can *barely* hold enough of the EXI specification in my head to work on it. i don't have room for encryption on top of it.

New personal record. Sigh: 64 bytes from 141.44.1.1: icmp_seq=9 ttl=53 time=377539 ms

jonasw, only if they split on lines (which I see no indication of)

meh

arc, and you shouldn't have to consider it at all as long as you don't do anything that makes it insecure, compression being one of those things

MattJ: I'd write a small loop with scapy.py

I found a utility, it just needs the correct command-line arguments

but i would assume if you consider compression insecure, eg DEFLATE, Brotli, etc, then you would prefer bitpacked over all options

lua -e'u=require"socket".udp() for line in io.lines() do u:sendto(line, os.getenv"HOST",os.getenv"PORT") end'

python3 -c 'import socket; s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM); while True: s.write(s.stdin.readline().rstrip("\n"))'?

heh

Lua wins ;) ✏

at this point my primary concerns are the size of the embedded image. cutting text-domain XML out reduces the binary size of the library in about half. removing compression library is a pretty big win too.

moparisthebest: CRIME and BEAST are based on the fact that the packet size changes depending on previously sent content, I doubt that this is the case with bit-packed, from the sound of the name :)

but I haven’t looked into it, at all

wolfssl is pretty small

soo… now I have that xep-ecaps2.xml here, let’s check out xep-0001.xml on what I need to do next.

arc, well compression is insecure because if an attacker can add the string "ar" to the payload and the size doesn't increase, then add the string "arc" and it still doesn't change, and build up from there, it can figure out what's under the encryption

so if bit packing works in a similar way, it's equally insecure

jonasw: Print it on paper, fold a paper airplane and aim for SamWhited :)

right jonasw I don't know either, just saying it's probably something that should be determined

moparisthebest: hmm. no i don't think so. so the only way you could reverse engineer it would be exploiting the string table.

like it'd be another useless thing to work on if it was proven as insecure as compression arc , idk

moparisthebest: it also probably does not matter much for IoT-thing <-> gatewaything.

yea it's pretty obvious security doesn't matter when it comes to IoT haha

like Ge0rG quoted yesterday: "The S in IoT is for security"

It would actually be pretty awesome if XEPs were submitted that way…

moparisthebest: ok, so string values are stored in the string table. this refers to whole strings only, but eg a JID you're communicating with would be added to the string table and referenced by id.

Please change the font to OCR-A or something first so I can scan it back in though.

SamWhited: because you would not have to do any work, as paper planes don’t travel several thousand km?

jonasw: Says you; that just means you're not building a big enough paper airplane!

SamWhited: we could also try XMPP over RFC 1149

heh, indeed

My favorite part is that there are Errata for that one.

there was an actual implementation

ok arc so the full payload size would increase with the strings you added, say "arc" would increase it 3 bytes, *unless* that FULL string was already in there, then it wouldn't decrease at all? if I understood you correctly

that at least would not let you incrementally guess strings like 'a' then 'ar' then 'arc' etc etc

moparisthebest: so if you can send a string value containing a 3rd party JID that you want to know if that agent is already communicating with, AND you know the schema being used, then you can determine whether that agent has communicated with that JID already.

like you I don't know enough to say without a doubt that makes BREACH or CRIME not a problem, but it seems better to me...

moparisthebest: yes. I do not recall a method for partial or combined strings

assuming you can observe the network traffic between those entities, which may only be within the local wifi

RFC1149 would be faster than my current link.

i'm still loading the spec back into my head. but i remember that as a fault.

one of my criticisms of EXI actually is the lack of a "list" type

I'd feel better if someone like xnyhps said they'd reviewed it and it looked good to them :)

eeew, I just decided I should actually print EXI and read it… but it goes on forever.

this comes up in some XML schemas such as SVG, where paths are made of collections of floats, ints, and characters separated by spaces

Hm, I should look at what a printer costs

SamWhited: yea its not light reading. I recommend https://www.w3.org/TR/exi-primer/ to start with

arc: Thanks

is there an email-adress where XEPs are supposed to go? the http://xmpp.org/xmpp-protocols/xmpp-extensions/submitting-a-xep/ page linked in XEP-1 404s

that gives a very nice overview without sucking you into the details

jonasw: You can submit a PR on GitHub

Zash: nothing, just "google" for one and ask the owner kindly to send you the printouts :)

jonasw: you can make a PR of the XEP in inbox/

SamWhited: which puts my xep in the inbox/ dir?

right

jonasw: Yup

you're younger than me, you might be able to handle it better, but ive had to segmentize the details so i dont get overwhelmed. its a lot to hold in your head at once

oh I doubt that; if you can't hold the entire spec in your head I doubt I have any chance

that's flattering but I doubt its true. age wears down your memory

jonasw: See the other XEPs in there for naming, I *think* you don't want it to start with xep- for reasons that I can't remember… something, something tooling.

I'm turning 38.

yeah, figured that much

speaking of the inbox, some of those things are *ancient*, does or should it ever be cleaned out?

am I the only one *always* falling for the delay github has with showing the "you have pushed to branch X n minutes ago, do you want to pull request?", hitting F5, seeing it appear before the page has reloaded, click compare & pull request and then the page reloads and you’re back to square one?

I think the editor readme says it never gets cleaned out. We don't want to break old pages.

Oh yah, I do that all the time

break pages? do they get rendered?

or you just mean links to the xml ?

moparisthebest: they get rendered on the site, just like actual XEPs

I didn't know that

SamWhited: https://github.com/xsf/xeps/pull/440 consider yourself paperplaned also: https://www.youtube.com/watch?v=Co452wJ-3Lg (Long Distance Calling - Black Paper Planes) (Music)

https://xmpp.org/extensions/inbox/

awesome

moparisthebest: Also, ¿Porque no los dos?

(I couldn't find the adorable little girl gif to send, so you just get text)

given the current status of IoT I think I might actually focus for a few weeks on *just* the schema compiler and get a XEP out for it. the one thing im missing for the XEP is a definition for the schema of the schema

> the schema of the schema

I'm so sorry…

that meta

arc: schemas like in XML Schemas for XEPs?

how are you going to deal with the mostly incorrect or inaccurate schemas out here in XEPs?

well, probably not mostly.

but they’re not normative, I’ve been told once.

yea, in order for a client to transfer to the server the schema that it wants to use, which the server doesnt already have, it needs to be able to dump the EXI-encoded schema to the server. and that needs to be defined since every client and server needs to be able to understand it

so the EXI schema for the EXI schema needs to be defined in the XEP

that’s meta.

its why I havent touched the XEP yet.

but it needs to happen, and sooner the better

sounds like he needs something harder to me

i havent written a line of code in a month. i'm up for it.

maybe 160+ proof

there are too many movies showing that coke doesn’t end well.

oh

nevermind.

160+ proof tea?

oh I have a copeous amount of cannabis

there's a "Balmer limit" to cannabis too, though.

heh

er "Ballmer Peak" https://xkcd.com/323/

:D

jonasw, That your protoxep? ecaps2?

dwd yes

though its more a cliff. more is better, to a point, and then rapid degeneration. its around the point that you start feeling like time is on a bungee chord

jonasw, I think you win the prize for using every obscure separator character in the ASCII subset.

dwd: thanks :D

they were barely enough, I was worried I’d also need EOT

jonasw, Can those appear in XML?

dwd: no.

XML forbids control characters except htab, newline and carriage return

(those between 0x00 and 0x20 at least)

Hm. Thereis an IoT thread going on with me in Cc. I wonder who deemed me so important and why.

jonasw, Perfect. Nicely done.

dwd: thanks! :)

Ge0rG: you are the chosen one for IoT. you must lead the way, because everyone knows nobody else knows it

arc, IoT is different and special from everything else.

arc: this must be a SCAM.

I'm humored by these IoT "Meetups" full of VCs who think IoT means a standalone device that communicates solely with their service, like a modern wifi-connected thermometer that you can control with your phone through their online service

jonasw: "Cabability"

in that ideology things like protocol standards don't matter. they mostly use a HTTP ReST API between the device an their service

arc, The sad thing is that most of these devices are going that way.

Zash: that’s only because you cannot use entities in <dt>! thanks, fixed locally, waiting for more of these stupid typos before I push another commit.

dwd: only because of the novelty of it. we need to catch up to steer course

arc, And worse, those that aren't suffer - my iKettle, for instance, is controlled locally, but people want to integrate - and they have to integrate via cloud services now.

dwd: Like the e-reader thing requiring an account with some online service to display text?

why does your .. what i assume is a water kettle.. need remote access?

that's my other IoT rant that I won't get into. not everything needs a chip in it. bloody Target selling basketballs with a chip in it to count bounces and report them to your phone via bluetooth

arc, So I can set it to boil from my desk, and - more importantly - so I get a notification on my smartwatch when it does.

my basketball does not need bluetooth.

arc, I understand. You're wanting it to use zigbee instead?

dwd: lol

+1

dwd: you're doing well roleplaying an IoT VC!

arc, I'm just like a VC, except without the money.

oh so you're homeless? ;-)

dwd: My water boiler has this amazing wireless notification protocol called "loud click and the sound of boiling water slowly fading away"

a basketball with a bounce counting chip?

mine makes a sort of loud whistling noise when the water is ready

Zash, Well. I can actually hear the kettle from my desk, in fairness.

I'd think you were joking if I didn't know better

moparisthebest: I didn't read much of the backlog, but the DEFLATE option for EXI very likely is vulnerable, without, probably.

Weren't there baseballs with accelerometers in them to measure how hard they got hit?

it sounds vaguely like air being forced through a small round opening

maybe I will move to a cabin in the woods like Zash :)

Zash: uh, I once had an oven which had the protocol of "if you don’t take care the water boils over the pots edge and flows down the sides into the oven tripping the RCA and thus cutting of your power"

we have a bluetooth enabled pressure cooker. it has a bluetooth range of maybe 8 feet, 10 if you're lucky. the app you need to communicate with it has basically a clone of the physical interface on the machine

xnyhps, yea any compression like deflate/brotli/etc would be, the question was whether the 'bitpacking' optimization without compression would be

or, without what we normally call compression

I suck at wording

moparisthebest: Call it PER

xnyhps: DEFLATE only or newer methods like Brotli too

moparisthebest, EXI in bitpacking mode doesn't have back-references, which is the basic issue.

but there is the string table, which I think would argue could have issues, and that's in all modes.

your own JID, for example, will be on the string table. so if someone could send you a jid as an attribute value, i believe it could under specific conditions, confirm if that is your JID or not.

jonasw: 'the i;octet' intentional or typo?

or if your device is communicating with a server, and they know which IP you're communicating with but not the specific hostname..

Zash, ACAP Comparator. Not a typo.

Zash: that’s how it’s called. not my idea :/

jonasw: ^5 on the XEP, this looks awesome

Lance: thanks

but given what moparisthebest described earlier I think that's a lot less of a security risk, since you couldn't pull out substrings to progressively reverse engineer, and the specific conditions are more difficult to otherwise achieve

jonasw, dwd: Well it could also have been an artifact of the conversion to epub I did

right I think you couldn't progressivly build up by guessing 1 character at a time that way arc

Zash, RFC 4790, now, extracted from ACAP. My mistake, I'm behind the times.

If you're not compressing the password anyway, the thread model becomes rather vague.

which again sounds better/more secure to me, but probably not as secure as not being able to guess at all? I'm sure someone could come up with an attack

Finding out someone's JID requires quite a lot of access for not that much information.

yea no the string table refers to whole qnames and string values

Or who someone is talking to, etc.

yea its not exposing, say, integer values coming from a sensor

Surely you'd need to address things to them? So at best, you're able to try to guess if someone who you already know by IP address, who is also in a chatroom with you, is the Jid you think they are?

dwd: Yeah, and you probably have much easier ways to do that.

well it only confirms that they're a JID in your string table. it wouldnt expose, necessarily, if they were that JID vs had talked to that JID

EXI doesn't "understand" XMPP beyond the schema you provide it.

i think it might be possible under certain conditions for an IoT vendor to craft an insecure schema tho

for example sensor data should be fixed length

TBH, I don't think that the use of deflate in XMPP is a general problem anyway. In extremely high-risk cases, perhaps, and if you're dumb enough to use PLAIN and TLS compression.

in that way EXI is more secure than text xml in that integer should be a fixed length, where a string representing an integer is not

Zash: do you have a diff of xep 369 from 0.8 to 0.8.1 from your fancy difftool at hand?

for security any XEP for sensor data, it should be actually put in the security section that float and integer values should be zero-padded to their maximum value to decrease risk of data leakage

zero-padded to their maximum value? how does zero-padding to maximum value work?

if all the stanzas from a device are the same except being X length, X+1, X+2, X+3, etc based on the scale of a specific integer value, you can determine whether that value is 0-9, 10-99, 100-999, etc

Don't EXI basically work like if you were to generate optimal C structs for all the things, then send that down the wire?

so if the maximum value is, say, 255, it should send as '001' '002' etc

arc: wait, leading zeros are encoded?

jonasw: for text xml

if it's a string it has to be

what i was saying is this is a weakness in text XML that EXI doesn't have

but even if not most things send integers in a set number of bytes

sure, but eg, a light sensor could flip between 0 and 100, and that would make it obvious what the state was

ah I thought you were talking about EXI already, of which I assumed that it encodes it as binary integer

people do not generally encode 0 as <light value="000"/>

yea it encodes as a binary integer

is it a variable-width encoding?

i would have to look that up again, i havent touched that part in awhile

i know you can constrain the range of most values

that would have the same issue then, and it cannot be worked around with leading zeros

well i don't believe its variable width per value, i think its only variable width by schema. if the schema says the integer value of a given attribute is 0 to 127, it'll do the right thing.

i havent touched that since november tho, id have to read up on it again

no worries

but im like 98% certain that an integer, float, etc value is fixed width from stanza to stanza

the question is if an integer can be 0 to 65535, it obviously encodes 60000 as 2 bytes, but does it encode 120 as 1 byte or 2 ?

that'd be a type of compression too

i believe that if an integer is a short it will always be a short.

could leak something, idk

you're right it could. but i dont think it does that.

that's how everything I can remember seeing works yea

and when we draft EXI 2.0 that is something that should be definitely put on the table as a concern

in general it seems like most things pre-2013 kind of took security as an after thought and might need to be revisted today

so far the only thing I would like to add to EXI is being able to encode a delineator-separated sequence like is used in SVG

if we had that, the SVG world would be all over it

being able to encode paths more efficiently would be a major breakthrough.

jonasw: You happen to know which revisions that correspond to?

Zash: nevermind, I diffed it locally

my initial interest in EXI came from getting tired of hearing about why X chat system doesn't use XMPP, but a binary protocol, for efficiency on mobile / etc

and the same is true for SVG vs proprietary vector formats

I’m going to bring up the <feature xmlns="…" /> stuff on standards@ again.

my complaint about SVG is that most things just arbitrarily execute javascript from them

not a great security feature

I wish I'd get some more insight from The Elders on carbonated body-less normal messages...

moparisthebest: the same is true for XHTML-IM

yep arc

script content is not allowed in XHTML-IM…

but like on my discourse instance I enabled common image format uploads, for example png, jpg, gif, and svg

(reminds me, I wanted to polish up my XSLT which strips off anything not allowed as per xep 71)

then, luckily it was a friend, uploaded an svg with some XSS javascript to steal cookies and showed me :)

are there any xslt/xhtml wizards here?

I'd assume this is where the xslt wizards live :) not me though

jonasw: stuff like <a href="javascript:alert(1)"> can still exist even without allowing <script> elements

Lance: haven’t thought of hrefs, good point

but that is usually easily filtered depending on the webview used

Lance, Dependsing on CSP.

a blacklist would be a never ending hole

moparisthebest: that’s why I’m using the whitelist from the XEP.

moparisthebest, No, I mean Content Security Policy stuff would prevent inline javascript from working.

I'm not positive you can do that kind of thing with xslt

yea dwd, not sure how you get/set that with something like xhtml-im

surely if there was a handy .noJavascript() method they would have called it

XSLT could do it. You shouldn't do this with XSLT.

no matter how hard you try it will always leave a hole

arc: what exactly?

jonasw: filtering XML/HTML

hm

how else are you going to do it?

also, I think that this should be pretty sound: https://github.com/horazont/aioxmpp/blob/devel/data/xhtml-im-sanitise.xsl (leaving aside the @href issue)

I'm in the camp for saying XHTML-IM shouldn't be supported

I wasn't. now I am.

I agree

arc: I also do not like XHTML-IM.

but then again, there are people who want rich text in their IM clients.

BBcode

you can have rich text without html

moparisthebest: is there a XEP for that?

not that I know of :)

https://plus.google.com/+ArcRiley/posts/BXpPxYRcRim

someone was advocating markdown somewhat recently

(actually, a body type="text/markdown" or type="text/rst" would be great; just make sure your markdown/rst doesn’t pass through HTML…)

right :) or it starts all over

Wasn't Markdown is defined as a HTML superset?

yes, Zash

i dont think thats still a complete solution.

Nice things, you can't have them

the <a href="javascript:"> links will leak through

well as Zash said bbcode it is then

plus the issues with multiple flavors of markdown, etc

I'm sure there are plenty of libraries already ready to use

in php...

gah, bbcode is annoying too.

There can be only one! (And it is pandoc)

as the saying goes annoying or insecure pick one

I probably just made that saying up

Lance: btw one thing i love is the stream framing from websockets? the added overhead for jabber:client namespaces is completely eliminated in EXI

yes!

if back then when that was being vexed over, if someone had said "in 5 years that won't be an issue anyway because EXI" it would have made the decision much easier

jonasw: I do think that xep115 has hash agility, and signalling the caps using a second hash algo wouldn't require a ns bump

re: markdown only one markdown I know has a defined spec, http://commonmark.org/

good lord, i cant even use libxml2 anymore. its just painful.

Flow: there was some mailing list post where people discussed otherwise, in the thread Tobias linked I think

schema-based xml coding makes so much more sense

so I think if you mandated commonmark with the exception of no support for http://spec.commonmark.org/0.27/#html-blocks it might be easier, would need more thought

nothing prevents clients from using a second hash mech, as long as they still send the mandatory to implement one

Flow: You mean sending multiple <c> elements?

Zash: yep

Flow: Doesn't fix the algorithm for producing the hash tho

Zash: Right

But I don't aggree with the statement that the change of the hash function of xep115 requires a namespace bump in ecaps2

jonasw: Any particular reason for going with a new xep instead of updating xep115?

Flow: I asked here, and people suggested that a clean new xep is the better way to go.

jonasw: i see

IIRC, it was so we could flag 115 as obsoleted by the new one

jonasw: Well, I think I suggested that a new XEP was the wrong way to go, and updating 115 was preferable :)

as an encouragement to devs to upgrade

Flow: to be clear, I’m happy to drop -xxxx and merge the changes into 115 if council prefers that.

also to prevent people from doing some compat with the old stuff badly.

but considering that it were council people who suggested to go with a new xep, I followed that suggestion.

pfff, council people are not always right ;)

not even close :)

Flow: they’re, from my understanding, those who decide whether a patch to XEP-115 will be accepted though.

I think it led to the wrong outcome in this case, but I can't fault the logic of taking advice from Council in general.

Sure, asking for feedback is always a good idea.

It seems like a good idea to me to go with a new XEP in this case just to encourage people not to try and have backwards compatibility with the old one (which rather defeats the purpose of having a new one), but I don't feel strongly about it and could be convinced either way.

in any case, I’m off for tonight. may read the backlog if highlighted

defeats the purpose in this case, I mean, since it's a security issue. Backwards compatibility is sometimes a good idea.

SamWhited: 115 is a core dependency of a *lot* of XEPs. I don't think replacing it is warranted in this case.

yah, that is tricky, not sure what to do about that. Either way tough we'd have to solve that problem and I suspect the two will have to coexist for a while.

Kev: The question is: Is xep115 is dependency or xep115 *and* the current namespace of xep115?

Well, at least for the dependency, it's straightforward, as the dependency is just on the latest version of 115.

Whether it should be or not is another matter, of course.

This is a fundamental question as we will find ourselves in the situation more and more in the future. For example with the XEPs depending on xep300

Yeah, aside from PEP, most of the "dependency" for these XEPs is just the fact that it optimizes the true dependency on disco#info

Do we need a BCP kind of thing?

Do we want to update all consumers of xep300 if it receives an incompatible update?

Or do we want to sepcify a dependency as xep number *and* "namespace", and update the consumers one after another?

Lance: Well said. I hate that some XEPs give you the impression that xep115 is an alternative to xep30

Zash: BCP?

Flow: IETF thing, like a pointer to the latest RFC on some specific topic.

Best Current Practices

ahh berst current practice

Flow: RFCs never change, but a BCP may be changed to point to a new RFC

isn't the the opposite what XEP do?

i.e., they do change, so we need a pointer to a fixed revision of a xep

(which we have in our attic btw)

Final XEPs are probably the closest to how RFCs work

true

ahh, enough DNSSEC fun for today. I follow jonasw to the realm of sweet dreams where everthing is like it should be

its too bad SRV records don't allow additional information

Flow: and dreem of jumping and colliding SHA1eep?

arc: what kind of additional info?

i havent touch DNS resolution in awhile, can you send a single request for multiple SRV records?

Lance: for example, the server capability, protocol version, etc

arc: Multiple how?

with the same name sure

arc: whether or not to start with EXI, hrm?

Lance: yes, or TLS, or etc

I suppose that'd be what TXT records are for arc

yes I know there's a XEP for TLS

or encode TLS or not TLS in the name like I did haha

that would easily explode though if you try to encode more

moparisthebest: yes, but doesnt that require multiple lookups? or can the two alternative names be requested at once?

now we have _xmpp-client, and _xmpps-client, we don't want _xmppse-client and _xmppe-client for exi for example too, probably

_xmpp{s,}-{client-server}{,-exi}._tcp

yea. so.. part of EXI is the first byte of an EXI stream is never a valid text unicode string by any enconding

yea arc that's 2 seperate lookups

one way is SRV records. the other way is to just punch EXI at the server, and it either responds with EXI or not

arc, uh what about ALPN I think that neatly solves your problem?

ALPN?

tls extension, tells it the protocol(s) you'd like to speak

Application Layer Protocol Negotiation ?

http2 uses it

oh, yes that could work

ive seen this before, just forgot about it

xep-0368 uses it too, but optionally

yea i saw this mentioned somewhere about http2 awhile ago. so, what does the payload look like

A text string in a TLS extension

a byte array.

because text strings are imPRECISe

ok so we could define a meaning for that which is extensible to other things

yea Ge0rG is more correct it's a precisely defined sequence of bytes

the key is it must be possible to use EXI without support for text XML

so basically an EXI xep could depend on xmpps-* records from xep-0368, and send it's own custom ALPN protocol sequence

or optionally, both xmpp-client and xmpp-exi-client or whatever

and server would say I can speak X

at which point you'd proceed or try next SRV record

i *hope* that server support would be well deployed before its an issue

oh interesting. it doesnt look like Contiki OS supports ALPN

arc: also, once the EXI XEP is decent, I'd be happy to help with making a proper xmpp-exi websocket binary subprotocol

Lance: absolutely. but lets get a javascript library for it first ;-)

from the times its been brought up i think the right path is to kill 0322 and start fresh. the one up there is utter nonsense from an implementers point of view

50% of the document is re-implementing EXI header format in a less compact form

and it doesn't even really get into how to handle a "pure" EXI stream (not starting with text XML)

the mechanism I think is best is this: 1) Client sends EXI header with <open> framing. in the header, the schemaId field contains a hash identifier for the schema it wants to use, generally in sha256: URI format, but this allows future hash values to be used

2) if server doesn't already have that schema, it responds with EXI header for a "default" stream using the schema-schema, and gives an error that the requested schema must be provided

3) if client receives such an error, it will restart its EXI stream with the same schema and transfer that schema 4) server responds with the hash as it understands it wishes the client to use in the future (generally, sha256: URI) 5) stream restarts (or continues after step 1, if server responded with the EXI header for the same schema) normally

the error-restart method should only be needed after a server is wiped, upgraded, or the first time a client of a specific version connects to it. sha256 is suggested to minimize this (large servers will already have the schema on file) but can be boosted in the future

it otherwise uses the same framing as websocket.

vs XEP 0322 it removes the issues with asking the server to download schemas from a HTTP resource (eg, using XMPP servers to multiply ddos attacks on webservices), removes the need for a text XML parser, reduces handshakes to initiate a typical connection, and removes redundant negotiation

so it just sends a hash of the schema it wants to use?

no other info about it?

yes in the EXI header field for schemaId. i believe the hash URI standard allows for length too

I was going to ask what stops a malicious client from uploading a 10gb schema

if the hash isnt known by the server, it asks the client to transfer the whole thing, and then the server gives the client a URI to refer to that schema in the future - which might be a newer hash

moparisthebest: the server should cut it off at some point obviously. schema should never be anywhere near that big, especially EXI encoded.

i mean you could make the same claim for what stops a client from sending a 10g <stream:stream opening element with a gazillion attributes

that is true

I wonder what current servers do with that hehe

or clients

with EXI? the few experimental ones use XEP 0322

i am not aware of EXI being used in production anywhere tho

the only complete implementation of EXI I'm aware of is written in Java

my libexi will be #2.

oh I meant I wonder what current servers or clients do with 10 gigabyte <stream:stream xml

oh, that's a good question

evil me wants to try it out

I'm willing to bet at least one will catch on fire

not at a production server of course other than mine :)

I'm guessing some are protected by a naive "no xml will contain > 10m so that's my buffer size"

or similar, but yea, testing time

well id bet actually that expat or libxml2 will dutifully attempt to parse it regardless.

What is realistically the biggest packet size a server should expect? Not more than a couple of kilobytes surely?

SamWhited: with HTTP over XMPP it could be more. isnt there a way for a MTU to be set?

Given the minimum maximum stanza size is 10k, no, a bit more than that.

Depending what you mean by 'packet'.

i assume stanza

yah, I don't know what I meant by packet… "start stream tag or any second level element" I suppose

amount of data in the XML parser which is not yet returned to the client?

er, application

moparisthebest: this is a good secure case to note

another issue servers might want to look out for is flooding it with new schemas. an LRU cache should be used to keep the number of schema from being pushed out of control by an attacker

it might or might not matter, but it could be a bit racy

like if 10000 iot devices all connect at the same time, request the same hash, server doesn't have it

yea disk size. but you can flood that with logs too

I guess they all simultaneously upload it?

that sounds like a crazy race condition

actually no, that'd almost never happen because each one has to be provisioned right?

it seems like it'd happen when you reboot the server or something though

i mean almost never happen that two try to send in the same schema at once. and one would hope the server can handle that well

oh, true. or upgrade it such that it wants to wipe the cache

maybe something like that

maybe you block the others while a few are uploading or something?

servers might be able to do something smartly

if a server policy is to, eg, use a SHA512 for added security because the operator considers SHA256 weak, even if it "has" the schema on disk it would need clients to transmit it in order to give it the hash that it wants

the schema shouldnt be large. thats why EXI encoding too.

I kind of assumed once a schema is uploaded the server would store it along with *all* the hashes

it could do that too.

anyway I'm off here for the day :) have a good one

so if a newer client asks for a sha512: right off the bat the server can respond "correctly"

all the server MUST do is return the schemaId it would like the client to refer to this schema with in the future. it SHOULD return with a hash URL, and it SHOULD record and handle any hash URL by any method the server considers secure

so that clients connecting to the server for the first time using the same schema as another client of the same model, can do so without having to send the schema first.

any reason it just wouldn't always use the hash?

#futurehash

that seems like the only way you could be safe knowing you were both talking about the same thing

allow the server to support future hash mechanisms without clients needing to understand them

a client sends a sha256: URI. the server responds to uploading it with a sha512: uri. client records and uses what the server gave it. the sha256: URI the client started with a guess. if sha512: were to become a new standard every client could use it.

otherwise a client connecting to a server for the first time would just start with the default schema and send the schema in order to get the identifier. which could become a bit much.

in 2017 i think we all consider sha256 strong. 2020 who knows

this is just me spitballing though.

so maybe a server MUST respond with a hash, it MUST respond with the hash in the same algorithm the client sent unless it doesn't understand that algorithm, in which case it MUST respond with the hash in the 'strongest' algorithm the server supports as decided by the server

that has some odd implications too. the hash itself is added weight for every connection. if 256 is considered enough, it should use 256.

802.15.4 devices have an effective MTU of around 100 bytes, and over 6lowpan packet fragmentation can cause real connectivity issues. its best to keep the EXI-encoded stanza payload under 100 bytes

the exi header with a sha256 uri consumes almost 100 bytes by itself, iirc

if its just <open> though its fine

i imagine #futurehash is more likely to be used over 802.11ah or similar newer, low-power protocol though which isnt necessarily subjected to the same constraints

in some cities right now, every bus is driving around with a 802.15.4 transceiver in a weather-proof plastic shell and a tiny solar cell glued to the top of the bus, rechargable battery, recording and sending realtime air quality data through a makeshift mesh network using, IIRC, some MQTT-based protocol

since they use 2.4ghz the buses are regularly delinked from the mesh network due to excessive frame collisions and inability to return pings, so restarting a stream on reconnect while under pressure is a real thing

fragmentation multiplies the problem in those cases.