XSF Discussion - 2021-05-18

qrpnxz: yes

I am severly confused by s2s comm. After a lot of reading, I think I'm close to understanding. RFC-6120 states that "foregoing considerations imply" that while completing STARTTLS and SASL negotiation two servers would use one TCP connection. Good. Therefore, bidi (XEP-0288) can be agreed upon during this period. The question is then how to do two connections. After negotiating the first connection, am I supposed to end the receiving server's stream, dial the initiator (with the same `id`?? seems illegal), and negotiate that connection, after which the initiator ends their stream on the second connection, and then we can talk to each other?

Because that kind of seems insane.

or i guess that the two streams are supposed to have different IDs and the second stream can reply to <iq/> from the first on the other connection without any problem just by having the id of that request?

ok so it looks like after SASL streams must be restarted. I suppose what you do is that the initiator starts a new stream, but the receiver instead goes and creates the second TCP conn?

When you said bidi I thought of https://xmpp.org/extensions/xep-0288.html where there's no second connection.

In standard xmpp without that, the other server creates their connection back whenever they want, usually when there's a reply or something to deliver. it's completely independent of any prior connection.

Unless you use Dialback, then there's a bunch of connections in every direction and everything is confusing and backwards.

Ok i think it should be as i now understand it then. Good.

Thx

Dialback being https://xmpp.org/extensions/xep-0220.html

dialback looks like something i never wanna implement

We should consider obsoleting dialback again.

Why? DNSSEC adoption keeps going up

And does anyone validate TLS certificates when doing dialback? That seems to work fine

it's the other way around mate, who'd wanna do dialback after validating TLS

I mean when you connect to bob.com's server, you validate it has a valid cert for bob.com before sending anything

Right, with TLS

Yes, dialback is still needed for validation the other direction, to prove to bob.com who you are

That's also with TLS

With sasl external it is, but dialback also works fine in the absence of that I guess

TLS and SASL support is required, and SASL EXTERNAL is also required for servers. So there's just about zero reason to do dialback afaict

There's some servers that run with self-signed certs, some with expired certs...

cron to the rescue

I would strongly urge anyone implementing s2s today to just not implement dialback

> There's some servers that run with self-signed certs, some with expired certs... Expired certs are probably just caused by mistakes, but why would you use self signed certs nowadays?

In case the domain is irrelevant.

and you don't need hierarchy

What do you mean by irrelevant?

not applicable or pertinent

If you want to do s2s with others I think it's relevant.

yes for s2s it's very relevant

thought you were asking in general

for things using tls

It was about s2s afaiu.

Dialback’s pretty convenient for testing, mind.

for testing what

For quickly running up a bunch of servers without having to generate a root, trust it in all the configs, issue certs etc.

For testing the server.

sounds like you can also just whitelist these

I found it easier to get certs for my testing servers.

plus, at some point you may want to test the code involving certs anyway :)

You *can* do it by generating a CA, committing configs with that trusted and issued certs in etc., but...

flow: Are you genuinely going to suggest that we don’t test our EXTERNAL paths?

hehe

anyhow, all that talk about dialback being so complex actually makes me want to implement it

of course, I first would need to implement an XMPP server

not sure if I am up to that challenge, I like my life as client library dev very much

But tests involving TLS are desperately slow, so being able to test with dialback is quite desirable (or you can use null cyphers in TLS or something, but we compile with those disabled).

i want to implement a server and client rn. So far I've not been discouraged. XMPP is rather straightforward for the most part.

rn?

right now

So our S2S tests don’t do TLS except those that test TLS.

(Well, our in-process tests. Our integration test suite uses TLS)

qrpnxz, implementing client *and* server is a huge task, fwiw

i'm starting with server

I was about to suggest to start with the client side of things ✏

but go for it, you can only learn from it

so XMPP: Core AND XMPP: IM AND i have to learn GUI? no way lol

how about just XMPP: Core server and done

sure, that's the nice thing about XMPP, you can implement a very basic server

Client's not even required really. You can just make the server if you don't plan on having other users LOL ✏

hmm, i don't see a way to make TLS optional without having a dummy feature...

In what sense?

You just don’t mark it as required.

Or if you mean optional to have configured, rather than optional to negotiate, you don’t advertise when it’s not configured.

Basic messaging between clients in Prosody was working in just a few days. It's everything after that that's the difficult part :)

> You just don’t mark it as required. Kev: Core says that if it's by itself then it's mandatory by default

flow, I've implemented dialback from scratch once, and fixed it at least twice. Don't recommend... it's simple enough to start with, then gets horrendously complex as you add in TLS, BiDi, etc.

I also think I discovered a bug in the XEP the other day

But I haven't reported it because I can't even be sure it's a bug in the XEP, or my understanding of it

dwd, basically anything that touches DNS makes me sad

qrpnxz, Yes, you have to implement, and negotiate, TLS to be standards compliant. But you can deliberately choose not to.

qrpnxz, Actually DNS is the easy bit. Unless you're talking DNSSEC, which then becomes complicated again.

MattJ, i'm writing something about a problem in XEP-0288 as well, but first i'm making sure it's actually a problem and/or proposing solution ;)

dwd, i see, ew

qrpnxz, Especially when deriving candidate subject names.

qrpnxz, And most interested in fixes for XEP-0288, of course.

> implement, and negotiate, TLS to be standards compliant serveral places in the standard hint at situation in which you get to, for example SASL, and tls is not negotiated yet. And indeed if you you have both tls feature and something else, tls is optional.

but i don't see a way to only have tls feature, and have it be optional, yet you have to show tls feature, so... a pickle

well, i do see a way: a dummy feature. But kind of ugly

If you don't have any other features how is it optional?

The stream wouldn't be able to proceed

qrpnxz, You just don't advertise TLS, and do advertise SASL.

i just specifically said that if it's the only one std says that it's mandatory by default so idk where you are getting i said that

not advertising tls is not the same as optional tls, also you cannot NOT advertise tls

you must advertize tls if you support it, which you must support because of the std as well

You're saying you want a scenario where you advertise only TLS, but you want it to be optional

qrpnxz, You really can. You're no longer standards compliant, because we think you should do TLS. But if you have a compelling reason not to be standards compliant, then go for it.

mattj, it's not necessary that i want such a scenario, but that it's kind of ugly the way you have to do it (dummy feature)

qrpnxz, I don't get where you're coming from with this "dummy feature".

It’s not a dummy feature.

if i have tls feature, and another feature that is not tls, and tls does not have <require/>, then tls is optional

It’s an alternative mechanism.

hence if i wanted to make tls optional, and i didn't have a legitimate feature to advertize alongside tls, then i'd have to create a dummy feature that doesn't exist to put along with tls, and then not mask tls as <require/>

qrpnxz, If a server doesn't advertise TLS at all - and structurally, it need not - then a compliant client might refuse to continue. But a client that *also* doesn't mandate TLS (which is most of them with non-default configuration) will happily continue without. If you have an alternate stream encryption (or similar), then a client which understands that might negotiate it in preference to TLS.

A server that doesn't advertize tls is not compliant, to start. A client that doesn't mandate tls simply doesn't use the feature *if it's voluntary-to-negotiate*. ✏

qrpnxz, Have you noted the last bit of https://datatracker.ietf.org/doc/html/rfc6120#section-5.4.1

qrpnxz, So if the server doesn't consider TLS to be mandatory but does support it, just don't include <required/> in the stream feature.

right, that adds to the brainfuck. That means that there should never be the case that TLS is the only feature and it doesn't have <require/>

qrpnxz, I don't understand why this is confusing. And yes, you're right (though <required/> would be redudant in that case).

it's redundant, but it's required. They should have just writ it so that without required it's optional to be consistent

qrpnxz, Well, sure, and it is. But if the only thing the stream is offering is to negotiate TLS, and you choose not to, then you can't do anything else.

it's not optional in that case.

qrpnxz, What difference does it make?

MattJ> But I haven't reported it because I can't even be sure it's a bug in the XEP, or my understanding of it

standards@ it maybe?

qrpnxz, I mean, sure, if TLS is the only thing offered, it follows that it *is* mandatory to negotiate, and therefore the server MUST include <required/>. But in practice, what cases fail to interoperate if it's not included?

required only really matters so you know why you can’t continue, rather than aiding interop.

none, yet the server is not compliant ;) And if you want to make it optional you have to add a dummy. Not a helpful special case.

I don't understand where you got 'dummy' from?

do you know what a dummy is

Yes.

Because no-one who wants to do tls is going to decide they’re not going to do it because it’s not marked as required. And similarly if it’s required you’re not going to be able to continue without it. It’s just useful for debug when things don’t work.

Maybe the missing piece is that if you are only offered TLS, the only way to not do TLS would be to do Dialback, which is advertised differently because hysterical raisins.

perhaps you missed my message where i explain i copy here: ``` hence if i wanted to make tls optional, and i didn't have a legitimate feature to advertize alongside tls, then i'd have to create a dummy feature that doesn't exist to put along with tls, and then not mark tls as <require/> ```

qrpnxz, Why do you think you need this dummy feature?

you need it to mark tls as optional if tls is the only legitimate feature

qrpnxz, Why do you think that?

because that's what the standard says. If it's on it's own, it's required. Period. So if it's gonna be optional, there needs to be another feature.

qrpnxz, Ah, I see. So you want a client to go straight to SASL?

But how would you SASL without TLS?

qrpnxz, If so... advertise SASL.

And that's where in practice you'd do Dialback.

Not personally, but i'm just saying that's what you'd have to do. You can't advertize SASL before TLS, and you have to advertise TLS

qrpnxz, And why do you think you can't advertise SASL alongside TLS?

Why can't I advertize SASL before TLS?

standard says so

ref or it didn't say so ;)

i will look :)

we should take notes how different people interpret the standard text and see if we can clarify this in a future revision of the text

qrpnxz, So you want to not do SASL *or* Dialback *or* TLS? What is it that you *do* want to do?

you are right you can do them at the same time, though if you didn't do SASL either then you'd need a dummy again ✏

qrpnxz, And shouldn't you advertise *that*?

I don't want to do anything like I said. Just commenting on the standard.

And then?

one min

and nothing

I'm lost.

how can i help

qrpnxz, So you're saying that a thing you don't want to do can't be done? And this annoys you because?

no, it can be done. Just you have to use a dummy maybe.

the annoying part is that there's an unnecessary special case of tls being required if it's the only thing.

whereas it could just be required if you put <required/>, which you must do if you consider it required anyway

I think you maybe reading to much into "TLS required" in the standards text

it basically says that an implementation can only claim standards compliance if it implements TLS

not that you have to jump through loops in the protocol, not offering TLS is perfectly fine from a protocol perspective

it says a lot of things

qrpnxz, OK. So you want the client to negotiate something which is unadvertised?

no

qrpnxz, So what do you want the client to do?

> not offering TLS is perfectly fine from a protocol perspective no, std requires you offer tls

i don't want the client to do anything dwd

qrpnxz, Then what on earth is the problem?

i've already said to you multiple times idk what you want from me

qrpnxz, Give me an example of the thing you don't want to do but think is impossible.

i already told you multiple times there is no impossible thing

qrpnxz, So why do you thinka dummy feature is needed?

maybe slow down a bit and see if we can steer this into a productive and helpful direction

i will tell you one more time: because if it's by itself and it doesn't say required it is still required, if you want to make it not required and there's no other legitimate feature, you must have a dummy feature

qrpnxz, OK. And what would the client do at this point?

if it were optional? idk, w/e the client wanna do, or negotiate a mandatory feature.

that would be SASL or dialback at this point, right?

sure, if that were a feature

so why not, instead of a dummy feature, announce SASL and/or dialback?

qrpnxz, OK, so there'd be another, non-dummy, feature for it to negotiate then. Where does the dummy feature come in?

yes we already established this is also an option. That's why i said *"if there's no other legitimate feature"* ✏

ok, we are going circles :)

If there is another feature, then TLS feature comes back to the world of mortals and behaves normally. Only if it would otherwise be the only feature, then you need a dummy to make it optional. ✏

Why would it be the only feature?

Because if you want the client to do "something else", then the server needs to advertise the feature is available at that point.

it's almost always the only feature. I don't know all the XEPs though.

the dummy is not for when you want to do something else, it's for when you want to do nothing.

If you want to do nothing the stream can't proceed, that's the point

it can if it's optional

It can't, there's nothing else to do

there is, if they are optional it's stanza time

MattJ, Maybe it's the equivalent of just holding hands.

It's not stanza time until it's authenticated, which requires SASL

(let's put dialback aside...)

sasl is not required afaict

You can't send stanzas without authentication

but if it is then you put sasl next to tls, which like i said, is ofc not a situation in which you need the dummy

qrpnxz, RFC 6120 says SASL is both required and mandatory-to-negotiate.

required to support, and mandatory-to-negotiate when advertized

It is a bit awkward that there's no explicit "you're done, feel free to send stanzas now"

Also awkward that an empty `<stream:features/>` means exactly that...

Zash, Well, for C2S you could advertise <bind/>.

if all features are optional and there's nothing you want to negotiate, you just go ahead. I think it's for the best. Saves a message

dwd, thought we were mostly discussing s2s

but yeah

But there *is* XEP-0361 for the oddball cases where you need that.

nice

it's def expensive to start xmpp conn with all those stream restarts and feature lists

for contrained environments. Desktops no problem ofc

that negotiation ping-pong is also old-fashioned when it comes to protocol design. these days you want to just send all the stuff you want to do in a single step, e.g. bind2

but I guess the negotation ping-pong is less of an issue in s2s

Depends on the environment. But yes, generally.

On a related note, I may have found a problem in XEP-0288. XMPP: Core states: ``` A <features/> element that contains both mandatory-to-negotiate and voluntary-to-negotiate features indicates that the negotiation is not complete but that the initiating entity MAY complete the voluntary- to-negotiate feature(s) before it attempts to negotiate the mandatory-to-negotiate feature(s). ``` However, XEP-0288 states: ``` If a server supports bidirectional server-to-server streams, it should inform the connecting entity when returning stream features during the stream negotiation process (both before and after TLS negotiation). [...] If the initiating entity chooses to use TLS, STARTTLS negotiation MUST be completed before enabling bidirectionality. ``` Now consider I serve the following: ``` <stream:features> <starttls xmlns='urn:ietf:params:xml:ns:xmpp-tls'> <required/> </starttls> <bidi xmlns='urn:xmpp:features:bidi'/> </stream:features> ``` Core says that I MAY negotiate bidi first, but 0288 says that I MUST do STARTTLS first. 0288 should allow you to enable bidi before TLS to defuse this situation. I can't think of why that should be disallowed.

Central to a lot of how XEPs work is that any behaviour can be overridden by negotiation. ✏

So if core says something and 288 says “But when doing 288 you should do this instead”, that is fine.

E.g. core says “There’s just <message/> <presence/> and <iq/>” and 198 says “but if you’re doing 198 there’s <a/> and <r/> too”, and that’s fine.

What Kev says, but also bidi is predicated on mutual authentication, so it'd be a bit weird to negotiate that before you decide if the other side is who it says it is.

I see yeah cool np

But maybe weird is OK.

i don't see how bidi is predicated on authentication, it just means says "dont' bother with another conn"

in fact, it's perfectly ok to do bidi but not TLS in the XEP

Sending stanzas in general is predicated on mutual authentication. Even XEP-0361 uses out of band authentication.

And if you do TLS, pre-TLS state is tainted.

So you couldn’t negotiate bidi before doing TLS and continue using it afterwards.

Kev, That's a good point.

i don't see such predicate in 0361, it merely recommends it

361 definitely only works bilaterally :)

oh i think ik what you are talking about kev, gonna look up the quote in the std

here: ``` The initiating entity MUST discard any information transmitted in layers above TCP that it obtained from the receiving entity in an insecure manner before TLS took effect (e.g., the receiving entity's 'from' address or the stream ID and stream features received from the receiving entity). ```

bidi would be such an information

therefore if you do bidi first, no more tls

You'd get another set of stream features after TLS and the stream restart

and contraversly, if you do tls, it must be first

right

*contrapositively lol

However, you may have a point - it's useful for the receiving entity to know if bidi is intended as early as possible, ebcause otherwise it'll start to open connections. (Or might do; Metre certainly will).

I think in Prosody bidi is not even advertised until after TLS and certificates have been verified and such

0288 doesn't require it do advertise before tls, only recommends it

dwd: But if you’re making such decisions, they have to be reversible, right?

Because the bidi before TLS is tainted.

Kev, Yes, but you could at least hold off on opening return sessions.

Although I’m not sure what decision you’d make based on an *outbound* session having bidi advertised.

but you really gotta do it before sasl, because after sasl it's assumed you aren't gonna send anything else on that conn

As on an outgoing session it’s the other end that can elide the second stream because of bidi, not you.

What's all this about opening return sessions early?

That sounds like a Dialback thing

uhh, not sure

Zash, It's a non-bidi-thing.

Hm, without dialback, something to verify that bidirectional connectivity is possible would be good...

that's what XEP-0288 does

One-way s2s is a bit annoying when it happens

Kev, Yes. So, you're a server. You get an inbound session claiming to be from example.org. You're advertising bidi. Until the inbound session has negotiated bidi, then you can save some time by initiating an outbound session to example,org. This is safe (in terms of auth, etc) even if the inbound session cannot authenticate.

XEP-0288 isn't all that widely deployed

Kev, But, if the inbound session then negotiates bidi, it was a waste of effort.

no way i personally waste time on outbound connection before auth, and bidi has to happen before auth, so problem solved for me lol

dwd: Yes, but I don’t know what advertising bidi does here. Because it’s you that’s advertising it on the inbound session, so you don’t know if the other end is going to use it later to determine whether to open the return session or not.

Kev, Right, because we don't negotiate until later. If, as qrpnxz says, we negotiate early, then this changes.

Kev, Even if we'd *later* have to renegotiate...

qrpnxz, It's very rarely a waste of time, and often makes things significantly faster to get to stanza exchange.

if you advertise bidi with tls, you can do bidi and no tls (unless TLS is required, then you must do TLS first). If you advertise bidi with SASL, then you can enable bidi, and after SASL you will have a bidi connection. Or if you don't enable it and then do SASL, then it's one way

dwd, yeah it's a tradeoff of waste of resources on non-auth and time save for successful auth. I choose not letting people waste my time without auth every time tho

.onion domains are a widely deployed reason to still support dialback I think

Prosody's mod_onions just accepts any *.onion certificate

onion does touch dns though, and i cant think of you you wouldmt trust that conn

*doesn't

jesus i rly messed up that msg

h8 typing on phone

ok on computer now. Onion addr is based on pub key crypto iirc, so if you connected to it, I think you are fine, but not sure tbh will have to look into it.

rly need to go to bed rn tho

qrpnxz, right, but then how do you validate the connection coming *from* something claiming to be a .onion ?

MattJ, wait for incoming SASL auth too? or how do you validate incoming .onion streams ?

It accepts any certificate, in that it doesn't reject them. But it doesn't use certs for authentication either. Still need to do Dialback.

ah ok, good

I suppose the dreaded "raw IP" requires dialback too

You can get certs for IP addresses actually...

Prosody can't federate with bare IPs tho, so glhf

(depending on version and whether it's the future yet)