XSF Discussion - 2022-12-03

Does XMPP use namespaces on the stream other than 'http://etherx.jabber.org/streams' and the namespace that defines the stanzas ('jabber:client', 'jabber:server' etc)? I can think of only one, for dialback. Are there others?

Guus, define on the stream? All Nonzas have their own namespace

I guess that in specialized scenarios, other (private/proprietary?) namespaces could possibly be added? I've never seen that - at least not with such namespaces being defined on the stream element.

defined as an attribute on the <stream> element.

ahh, well in theory you could pre-define a ton of namespaces there

You're talking about namespace prefix declarations on the root stream?

and use this as optimization when sending an element of such a namespace

flow/zash; yes, that's what I mean.

It's one of those "XML allows it, XMPP does not explicitly forbid it, but implementations usually do not do it" things

for example like namespaced attributes

That's what I was wondering. I've never _seen_ it - but does it happen in the wild. Dialback is one, I think, but is there more?

I can not come up with anther example from the top of my head

Ok, thanks.

Guus, may I ask what the outcome of this question influences?

Openfire currently discards any namespace/prefix definitions on the stream tag. This is causing some issues while refactoring our s2s code (notably around dialback). From there, I got curious.

It's nice to be able to treat every stanza/nonza as it's own document

For a lot of reasons, not the least of which is security

That's why Openfire discards the root element (after some validation). Downside is that those namespace definitions are lost.

I was kind of surprised that that hasn't caused any issues that caused people (likely: third party devs) to reach out.

I imagine most XMPP devs have been avoiding that because of implementations with custom XML parsers that are picky about the stream header, or doesn't keep ns prefixes.

When I first started with xmpp my stock xml serializer would always put all namespace as prefixes into the stream opening element. But I quickly learned not to do that after comparing it to what I saw in the gajim xml console

Could save some bytes I expect, but probably not very many per stanza

In some cases like XEP-0198 you could save a lot on relatively small and highly frequent elements.

Zash, a stanza smaller than an ethernet frame will not reduce the size of transmission in any way.

But I plan on finishing a XEP for advertising as a server that you support this feature, which could indeed help kickstart clients to do it.

moparisthebest, you can still treat every top level stream element as its own document and still support namespace prefix definitions in the enclosing <stream/> ✏

> Link Mauve> Zash, a stanza smaller than an ethernet frame will not reduce the size of transmission in any way. Huh? if you send 90 bytes payload, the ethernet frame will be 90 bytes + tcp header + ip header + ethernet header? What am I missing?

flow, whether the result is 90+some bytes or 1500 bytes the same amount of data will be transmitted.

flow: only if you keep around some state, then you gotta remember when to clear it

If it is 1501 bytes two frames will be transmitted, so you will have sent twice as much as 1500 bytes.

moparisthebest, easy: on </stream>.

moparisthebest, sure, but that's like not rocket science?

(Or connection loss.)

Link Mauve, are you talking about MTU?

Is keeping a map of prefix→namespace that much? Think of it as HPACK maybe? ;)

Guus, yes.

I always assumed that this was the _maximum_ allowed size in one data packet, but not a fixed size.

flow: idk, it's the cause of gzip compression not being secure for instance

Guus, hmm, that’s not what I got taught at uni, but now I’m starting to doubt.

as in, I assume that you can sent _less_ data. Sending less data with more packets incurs overhead of packet metadata, which is why you'd want to use up as much of the MTU as you can. I did not know if _not_ doing that causes all of the 'remaining' space to be used up by explicitly sent data.

moparisthebest, I think we are talking about different things here

Plus you have to *not* keep anything around for WebSocket

Link Mauve: I very much am _not_ an expert here, so there is a good chance that I am very wrong.

Me neither, which makes me doubt even more. :)

And I don’t even know how to start to measure that, perhaps saturate a link with just-MTU size and then saturate it with as small a packet as possible, and compare the throughput?

tcpdump?

besides the name, it actually captures ethernet traffic, so it shows everything down to the ethernet frame

That’s one layer too high, we want physical ethernet frames.

Oh?

so does wireshark

I'm simply saying keeping state around is always more dangerous than not keeping state around

Link Mauve: no, you definitely can send packets and Ethernet frames smaller than your MTU...those won't get magically filled up with padding bytes ;)

Keeping state around is what makes XMPP efficient IMO

moparisthebest, I am not sure if I would agree

Creating a whole new XML parser for each message seems meh.

Zash, why?

flow, don't have to say things the other end already knows (because it keeps state)

TCP will try to fill it's packets up to the MTU, but only if there is enough data waiting to be sent in its buffers...otherwise it will send smaller packets...

Zash, In my case, I keep the state by synthesizing a wrapping <stream/> around the top-level element before handing it to the parser ✏

Monal creates one XML parser and can use any namespace prefix defined on the stream header anywhere in the stream...

we do something like that for WebSocket

or did we?

maybe it was BOSH

flow: why don't you use a single streaming parser and feed in data as it arrives?

because denial of service attacks due the missing isolation

so basically a top-level stream element is send from one entity

What missing isolation?

if you parse multiple stream elements with the same parser, then, due the missing isolation, one entity can affect another entity

that's the high level view of it

Isn't parsing a stream header for every incoming stanza much more overhead --> more dos potential?

but maybe things will become clearer if I post the DoS in Smack that the Jitsi folks discovered

The same for all users?

Don't do that

WebSocket with each stanza being explicitly framed is a much better way to do it, we should start doing the same over TLS/QUIC

thilo.molitor, the stream header is trivial to parse and from an trusted entity

Ie prefixing each stanza with a length

> The same for all users? > Don't do that No, one parser for every TCP stream of course

hasn't Jitsi published that yet, Flow?

(I agree, it makes a strong case for not carrying over state).

No, one parser of every top-level stream element ;)

> if you parse multiple stream elements with the same parser, then, due the missing isolation, one entity can affect another entity > that's the high level view of it But as s2s client, if you don't trust your server all bets are off anyways, no?

*c2s

but really, it depends on the parser you use how much DoS'y it gets

in Smack's case, it was ironically StAX's DoS protection that makes the connection DoS'able

moparisthebest: the grip Oracle thing specifically relies on the attacker injecting significant content into the compressed stream. Only affects xmpp in the case of error stanzas mostly

> No, one parser of every top-level stream element ;) I don't really get it...you use one parser for all TCP connections?

singpolyma: we could just compress every stanza independent of others, wouldn't that eliminate the oracle risks more or less?

No, I create a new parser for every top-level stream element

> No, I create a new parser for every top-level stream element Is that a server or client?

thilo.molitor: yes, of course

For a client I won't assume the server tries to dos me...

thilo.molitor, just be sure to never connect to moparisthebest's servers then ;)

singpolyma: then someone just has to write a new xep for that ;)

> thilo.molitor, just be sure to never connect to moparisthebest's servers then ;) :D

thilo.molitor: or just the code. The existing xep and/or tls compression cover it

thilo.molitor, client in my case, but I believe a server should do the same

esentially servers are more exposed to unvalidated input as clients are

It's just an implementation consideration to flush after every error stanza or on error stanza or whatever model you want to choose

I definitely remember langsec folks saying the XML way is better than lenght-prefixed protocols for security reasons. So I will remain skeptical to framing, and especially websockets.

but in XMPP, servers tend to just route through invalid / dangerous data to clients if they do not need the data for routing purposes, so clients also get a lot of questionable input

flow, define "invalid / dangerous data" ?

flow: but from a client perspective a server can easily do wild things like always close the connection after authentication or whatever...what I want to say is: if, from a client perspective, you don't trust your server, all bets are of anyways....

singpolyma: I'd do it after every stanza/nonza

> flow, define "invalid / dangerous data" ? Yes, please

Zash: to be clear I'm saying length prefix is best, I just mentioned WebSocket cause that's basically what it does

I can't think of a problem with a length prefix, but many vulns have happened because we don't have it

https://discourse.igniterealtime.org/t/denial-of-service-vulnerability-in-smack-4-4-if-xmpptcpconnection-is-used-with-stax/92314

Zash, invalid could mean multiple things, for example schema says attribute value is int but it actually contains "foo"

Aren't there vulns because of them too?

moparisthebest, question 1: how many bits would you reserve for the length prefix? question 2: how do you detect and handle the case where you got out of sync with the length prefix, e.g. because a faulty remote implementation swallowed a byte somewhere? question 3 (somewhat related to 2): which endianess? question 4: how to prevent people from passing it right into malloc() without thinking?

dangerous could be something that triggers the dos protection of your XML parser ;)

(pro tip: use an XML parser which doesn't need DoS protections because it doesn't support entities or whatever)

well in JAXP's case, the limits where cummulative…

and not just entity exansion, but also things like max attributes

the commulative nature makes the thing worse if you parsed the whole XMPP stream as a single document ✏

if you parse an XMPP stream into a document, that's your first mistake

I know, that's why I don't do it ;)

or maybe "into a document" has a differnt meaning for me as for you

Smack's code before me used to threat the whole XMPP stream as a single document using a single parser

if you end up with something DOM-like which contains the stream header

creating separate parser things for every stanza is another way to screw things up

well it's pull parsing, so it's not dom-like

that's got nothing to do with one another

you can have a pull parser which creates a DOM

you can have a SAX parser which is fed into a DOM generator

the only sensible way, which won't lead into valleys of pain and sorrow, is using a streaming parser and to parse the entire stream in a single parser state

fwiw, I did not run (yet) into any issues with the splitting approach

namespace prefixes?

but ymmv

well that is what we discussed initially

sorry, late to the party I am

that would be one issue though :)

if you want to support them, then you have to support them, but its not rocket science ✏

even with just the default namespace, it implies having to use some weird parser for the stream header (one which exposes prefixes and so on to the application layer, which is yet-another-way to make ones life harder with XML)

not sure where it implies to use some weird parser

(namespace prefixes shouldn't matter and you should just not even receive them from the parser; though being able to set them is sensible for legacy compatibility)

flow, how would you otherwise learn the default namespace, given that the stream header is decidedly not in that namespace?

that requires a weird parser.

alternatively you can not care about that, of course, and ignore unconforming implementations and just hope it's jabber:client/jabber:server matching the connection type.

it's been probably 3-4 years ago when I wrote the code, let me see what crimes I did ther

jonas’, not sure if I understand your question regarding the "default nnamespace"

at some point I get the stream open tag, and there will be a xmlns in it?

exactly

no sensible parser should give you the contents of that attribute

to protect you from hurting yourself

that attribute being 'xmlns'?

yes

ok, I am not sure why a parser shouldn't reveal the contents of the attribute, but in my case, the stream open is in fact not processed by a parser, but by jxmpp's XML string splitter, it is special cased one could say

So in case anyone was wondering why we don't use some XML features, this is why. :)

hmm?

jonas’: 1. 24 bit would be enough but maybe 32 to not be odd... 2. This isn't a problem is it? If you read length and it's not valid XML, close stream 3. Network endianess 4. Well it's trivial, and you can reuse buffers, unlike now where people dynamically grow allocations (and it plays better with GC langs)

moparisthebest, I'm afraid what will happen if I tell you this, but you could probably get away with chunked content encoding from HTTP (hex encoded length prefixes) since text between stanzas will likely be ignored

That was in particular build allowing for XML features like namespace declartions in <stream/> to reduce the bytes on the wire ✏

Zash: hmm probably true... Though I'm not sure it's very helpful if it's optional, I was thinking it'd be advertised on the connection level, ie you'd know before connecting

A length prefix on regular stanzas seems redundant, but it would be neat to have a way to switch to "this is binary now" with a length prefix to allow arbitrary binary content in almost-xml without base64

moparisthebest, well if you do things like that, stream feature and negotiate it

or EXI it

> well in JAXP's case, the limits where cummulative… Isn't that dos protection having upper limits for the entire stream moot anyways? I mean at least in the xmpp context you'd process a stanza and then forget about it...and I think many streaming XML protocols do the same...

So isn't turning off these global limits a better dos protection than trying to split the XML string into its stanzas/nonzas on the string level with something not a true XML parser just to concatenate this string chunk with another containing the carved out stream header and then feed that whole thing into a true XML parser?

flow: Seems overly complicated and error prone to me...

If your parser could only do global limits rather than per stanza limits, you maybe should use another parser (maybe something written in rust?)

thilo.molitor, I am not sure if I would describe it as overlay complicated myself. I belive you need some first parsing step that enforces stanza limits, and then the <stream/> handling is trivial

at least that was my experience when I implemented it

and I also found that this low-level, featureless XML parser that does the XML splitting to be fairly easy to implement

especially if you only have to take care of valid-XMPP XML

thilo.molitor, I am not sure if there are may further XML streaming protcols besides XMPP fwiw

I am not sure about SOAP, but I believe SOAP messages are freestanding

There have been streaming atom over http things in the past, but most of those are streaming json now

globally turning off the limits in JAXPs case is problematic if JAXP is used elsewhere in the same application/(JVM instance) where they are required. in some cases, you can't simply turn them off globally

and sadly JAXP has no way to configure per-parser limits ✏

and wrt RIIR: I'd rathyer stay in the JVM ecosystem and this makes the whole thing portable

I think using a general purpose XML library rather than an XMPP specific one is also always a mistake

but really, the main argument is that it's sensible split the incoming stream in individual top-level stanzas for isoliation purposes, then the limits of the actual XML parser do not really matter, as they are probably higher than the usualy maximum stanza limit (which I believe is currently in the range from 64 KiB to 1 MiB)

moparisthebest, that's a bold statement, why no re-use existing libs as building block?

i'd assume that implementing your own XML parser is far more error prone, especially if done in an unsafe language

A long and continued history of security vulnerabilities has proven it to be a mistake

He didn't say build your own.. But use one that doesn't have the 90% you'll never need

so because others are unable to implement a XML parser securely you have to do it yourself because you will not make any mistakes?

general XML parsing is very very complicated and full of demons, parsing the small subset of XML that XMPP uses not so much

I think most XML parser are able to be configured to use the sensible XML defaults that XMPP requires, some of them are even configured out of the box that way

it's really not that much, many of the issues are due to entity references and external entities, disable them and you eliminate many error causes ✏

moparisthebest: why would you need something xmpp specific?

but those are not really much about parsing, more about XML transofrmation

I'm not aware of any "subset" that xmpp uses. Unless you mean we don't use doctypes

singpolyma: because it has about 999x less code than a general XML parser and therefore 999x less bugs ✏

singpolyma, if I understand moparisthebest correctly, then he assumes that a parser that only implements the XML stuff required for XMPP is superior

compared to using a "standard" XML parser

flow: but that's ~all XML stuff afaik

I am not sure if I can follow moparisthebest argument

Yes, smaller and less buggy, which means more secure

singpolyma, entity declarations, comments, stuff like that

if you use a niche XMPP XML parser, then this code probably gets less review than the real XML parser

Zash: declarations are doctype yeah. Does xmpp explicitly disallow comments???

I believe so

and most standard XML parser can be configured in a safe XML way, which is "accidentially" just what XMPP requires from XML

If only I could link to sections in the RFC

"less review" isn't a very good argument

singpolyma, yes we disallow comments in the stream

singpolyma, https://xmpp.org/rfcs/rfc6120.html#xml-restrictions

but I am not aware of any security issue wrt to XML comments (happy to be proven wrong)

That's insane and dumb but ok

flow: maybe they can be but a massive source of CVEs is when they accidentally aren't

This isn't conjecture I'm just looking at history

I mean, processing instructions are mostly just comments

Unless you define some that you understand

the real XML mine field are entity references and processing instructions ✏

And yeah, doctypes are a whole different thing we obviously don't need ✏

flow: comments are a security issue when different things interpret them differently, ie your parser thinks something is commented out and mine doesn't etc, also a very common vuln in parsers of all types

moparisthebest, but i've never heard of XML comments being ambiguous?

you are right in general of course, that this is the typical problem of (human-readable) comments in documents for machines, but I never heard that this was an issue in XML (again, happy to be proven wrong)

What is the use case for comments in a network protocol?

there is none, but it's probably also the XML feature with the smallest attack surface

still, I lean towards that it is sensible to forbid them in XMPP

Zash: same as anywhere else. For humans to betters understand when reading

I'm not aware of comments allowed .. outside of email, which seems like an argument to forbid them.

eeehhh, maybe, but in my year working with XMPP I never thought "hey, it would be great if this XMPP stream had comments"

and even then, you can immitate comments by using custom extension elements

and, most importantly, provide <error/> with a good textual description that reflects the internal state of the implementation emitting the error, aiding debugging

(of course without leaking any secrets ;))

*in my years working with XMPP

flow: sure. But also pretending we use a "subset" of XML when no one is hand rolling an XML parser seems just asking to be violated. I expect most implementations allow comments because there's no reason not to

Relevant https://modules.prosody.im/mod_conformance_restricted.html

singpolyma, I believe you are right about this assumption, but like to point out that jxmpp's XML splitter bails on comments (for that exact reason) ✏

flow: https://bugs.chromium.org/p/project-zero/issues/detail?id=2254 is a good example of even raw XML being "ambiguous" and yet another reason for length prefixed stanzas

https://hg.prosody.im/trunk/file/0.12.1/util/xmppstream.lua#l228

Gloox still hasn't been fixed by the way

Fwiw I agree stanzas should be split from the stream before sending to an XML parser https://github.com/moparisthebest/xmpp-proxy/blob/master/src/stanzafilter.rs length prefixes sure would make it simpler though

Was expat fixed?

I believe I asked at the time and ejabberd's fastxml and/or expat was fixed

Both of the bugs seem to come from hand rolling utf8, which I guess is the sort of thing you have to do in C

moparisthebest, well it is ambigous to broken implemetantions, but then all bets are off. I was thinking about someting like https://trojansource.codes/

moparisthebest, fwiw, I can totally get behind the idea of length prefixes for top-level elements in XMPP 2.0

Then everyone would *have* to hand roll parsers :P

no

well yes, the first level parser

that extracts the length prefix and an XML string

but then you can use an off-the-shelf XML parser for parsing the XML string

flow: ha yea that was the kind of thing I was thinking about, nice website

flow: hopefully you wouldn't actually allocate a string for the XML data!!!

singpolyma, hmm?

Yea but you already have to roll your own first level parser, it's just hard vs reading a length

I think I would, and do

The way you described it made it sound like you would read <length> bytes off the stream into a string and then feed them into a parser

Which is pretty awful

it is *harder* then reading a length and the filling a buffer until the length is reached

singpolyma, why is it pretty awful?

Because you need to store the while stanza in ram

right, so?

stanzas have to have an upper limit

we are not sure where the limit should be, but I think it's between 64 KiB and 1 MiB

Only because people implement things in this way :P

Ideally you'd have a pool of buffers to read stanzas into, and never resize them

sure 1 MiB is probably not something your embedded microcontroller can handle

that's why I personally would prefer a smaller maximum stanza size, probably around 128 KiB

If you have a parser already there's no reason to buffer the entire content. In any format or protocol. This is why we have streaming parsers

max_stanza_size_bytes = 262_144

And then we don't need no length prefixes either

Sorry that's what it is forever

the entire federated network agrees

I thought I was lower than that?

singpolyma, I don't think that this is true

Thinking of the 10k number?

Oh no, I'm forgetting base64 overhead

you need to buffer the entire top-level stream element for processing after parsing, in one representation or the other

Yay, 3x memory usage right there!

Zash, 2x I'd say

but, yeah, you could tripple buffer

1x for the buffer, 1x for the collapsed string, 1x for the parsed representation.

flow: depends on your application. A server for instance does not since it can just copy the bytes to the target stream once it know which that is so needs only some small state derived from the parser events not the whole thing

Zash, in my approach, the collapsed string is the buffer

I'm told some past server did that singpolyma

singpolyma, that's dangerous for the server to do

Sounds fun for running clients out of memory

because what if the sender suddenly becomes very slow

yet you already started to feed the send data into the receiving sink?

Ooh yea and the slowloris, good point flow

XMPP servers have to store-and-forward

moparisthebest: that's the client's problem :)

just like ethernet switches ;)

They don't have to, but probably should

Probably should avoid I think you mean

You could totally ruin s2s links as a client that way

I'll send half a stanza then just never finish lol

What does the server do to recover?

essentially XMPP is totally borked when it comes to queuing discipline

how should a server behave if a client suddently starts to receive data veeerrry slowly?

flow: how does the client receive speed affect the server at all? Just keep putting bytes in the socket

(that is basically the reverse of the "Slow HTTP Attack")

singpolyma, if a clients does not drain its outgoing buffer on the server side, then the buffer either grows indefinitely OR ✏

flow, `<sm:a/>` times out, it gets kicked

or it has to put backpressure on some other entities in the network (which is bad)

or it has to drop stanzas (which is also bad)

all three options in that scenario are bad and we have no solution

I... Don't think that's how it works? Server side writes the packets to network they don't need to buffer there?

singpolyma: I'm talking about a client A sends a stanza destined for a remote server, or even another client on the same server, if the server starts streaming the stanza before client A finishes, what does the server do if client A never finishes?

singpolyma, of course there is a buffer and/or the write() will block

open 3000 s2s connections!

moparisthebest: right. That's an interesting case. And of course one could argue that mam requires storing it anyway, so maybe that's the right initial sink.

fwiw, there is a fourth option which is, I believe, employed by most serious XMPP servers: drop clients that are to slow

probably not ideal, but obviously has served us well so far

still, I'd love to see a better solution, but this probalby requires a fundamental different approach from XMPP

But if they only sent half a stanza and you already started sending it down other streams then you have to close *all* the streams

In normal Instant Messaging, being slow enough to run into these kinds of problems is rare, so if it does happen it's likely the connection is borked and closing it is the thing to do

basically the solution is that clients to not get data pushed, but that they pull, and that most state resides on the server (which is similar to djb's mail 2000 approach)

Zash, the problem is that it is often not the client being slow, but the others being to fast

flow, what's that? send notifications and then clients (and servers) poll?

something like that: the server sending just a tiny bit of notificaton that causes the client to initiate a poll

really nothing that we will probably see soon (or ever) in XMPP

Sounds like a great way to build a self-inflicted DDoS machine

how?

It changes the rate of things to depend on the receivers instead of the sender, so if you host on underpowered hardware (as some do because XMPP is very efficient).

yep, that's the pill you have to take. as I wrote, most state is on the server then

which would then, as yout pointed out, required stronger server machines

Affects mostly broadcast things like chat rooms tho

but in my vision, every family and/or household hosts its own service (or uses some sort of provider that operates if for them at home or at a cloud)

and then the load on the server should be not really significant, as the number of users is low, and by that the state is not that big

fwiw, that is also why I ♥ IMAP

Adding a bunch of roundtrips just to receive a small chat message seems unwarranted.

Tho it is, in effect, what you get on iOS and other restricted mobile settings.

jonas’, to pick up a previous topic: could you elaborate why a user learing about the value of the default namespace would lead to potentially hurt the user?

flow: they might try to use it for something instead of letting the XML lib handle it

Hey folks, would be great to one or the other reviewer! https://github.com/xsf/xmpp.org/pull/1209 We still don't know all contributors

emus: Regarding your request on changing the Monal feed on Planet Jabber, I have a question. What is the difference between monal.im and monal-im.org? I see the latter is newer, but it is kinda confusing.

ralphm: Monal changed their owners and made an entire new website CC thilo.molitor

.im is outdated, see last blogpost ✏

ok

Woo DNSSEC