jdev - 2021-01-21

Could someone tell me about a library that implements XEPs? I would like to see how such libraries/their API is designed

jubalh: https://github.com/igniterealtime/Smack/tree/master/smack-extensions/src/main/java/org/jivesoftware/smackx

jubalh, https://docs.zombofant.net/aioxmpp/devel/api/index.html maybe?

thanks flow & jonas’

jonas’: who uses aioxmpp again?

jubalh, some people do, I don’t know who they are

which is nice, in a way

I sometimes get bug reports ;)

Don't you use it for s.j.n?

I see, I thought maybe there was something popular. I'm not up to date on which client uses what

Martin, yeah well, I thought it was obvious that I use it :)

jubalh, no client has been ported to aioxmpp

ok

it seems to be not unpopular in IoT applications, judging by the questions/bug reports I get

thats interesting

Internet of Python Things?

Internet of Schlangs

jubalh: gloox

Doesn't IoT have mosquitos?

jubalh: https://pkg.go.dev/mellium.im/xmpp as well

thanks

After discussing it yesterday I went and implemented the WebSocket subprotocol. It was suprisingly easy and required very minimal changes despite the lack of a root element. Now if I can get it working with HTTP2 I can avoid BOSH altogether, so thanks to whomever suggeseted the HTTP2/Websocket RFC.

but will it work with HTTP3 :)

No idea, I'm sure it will eventually if it doesn't work to start.

I keep expecting new versions of HTTP to just have bidirectional streams built in (I thought server push in HTTP2 was going to be this, but for various reasons it's not and can't really be used that way)

my impression is at the HTTP level, 2 and 3 are pretty close to the same, it's just underneath and above that has completely changed

Didn't people conclude recently that HTTP/2 push was a failure?

Zash: probably. It's not reliable to say the least. Although I'd say it was an issue with how it was advertised, not the actual feature. If you just use it for what it was designed for (preloading images and things on the page) it's pretty great and when it's unreliable because a proxy drops it or whatever you just go back to the same thing you had before, no worse. It was a nice optimization when it could be used.

like http2 is (http2 over TLS over TCP) and http3 is (http3 over QUIC (combined TLS and TCP guarantees but over UDP))

Can I be upset that they seem to have invented invented TCP + IPSec?

No, because it's not the same thing at all. My limited understanding of QUIC is that it does a better job than TCP of congestion control (which is what all the tons of complicated HTTP stuff multiplexing various streams was trying to work around)

I'd be curious what differences are in the HTTP layer if it's mostly the same just HTTP over QUIC. Probably getting rid of all the multiplexing stuff since there's less tail time now, but otherwise?

There are multiple TCP congention control strategies.

But maybe it was more SCTP, with multiple independent streams over the same connection.

Zash: and they're all terrible. No matter what you still have the TCP warmup period, as far as I know. QUIC is supposed to be a lot more efficient (though I admit, I've just seen the numbers from Google, I don't really know how it works)

I dunno, it's about time we replaced TCP IMO, so I'm hopeful that this is really better.

It also combines the TLS/TCP handshakes, which is nice. Less stuff to do before you can actually start sending bytes, which has been a significant problem at some companies I've worked at

in addition to combining TLS+TCP handshakes, it also supports network roaming (bye-bye stream management, we don't need you anymore), and (I guess not useful for XMPP) fixes head-of-line blocking

The head-of-line blocking thing will be a big one, I've run into that a lot with HTTP/2 streams

not having to redo connections on network changes alone is a big enough reason for XMPP to use it imho

Indeed

I see a lot of discussion about websockets, but no actual upgrade mechanism being proposed

Although apparently there's something called WebTransport that I don't fully understand that offers bidirectional streams with arbitrary protocols on top? Maybe this will just replace websockets

(and it supports QUIC)

Now as usual I fall back to the question of what packages this should all be in. I could have the current negotiator just have a websocket option and the dial package (which does SRV lookups and creates TCP connections) support websockets, or I could have a new websocket package that has a custom dialer, a custom negotiator, and a bunch of helper methods to eg. dial and negotiate an XMPP session all in one Go that mirrors the functions in the main XMPP package that do that except for TCP.

Organizing code is hard.

Actually, I wonder if I can make this completely transparent from the client side: the dialer would do all the lookups and dial TCP if it can or websocket if that's all that's available (or if a PreferWebsocket option is set or something) and then the session negotiation would just check what type of connection it is and use that protocol. That seems bad somehow, but I'm not sure why.

Probably best to just always let the user be explicit about how they're connecting.

Sam Whited, in a theoretical near-future where a XEP that supports lookup with SRV2 exists, fully transparent makes sense

as that lets the server admin set preferences across websocket, TLS, quic etc

right now SRV and websocket have no relationship so having your library decide priority seems bad?

That's a fair point, the question is really "shoudl the client dev or the server admin decide how we connect"

Sam: FWIW, in Swiften we have a Connector that you tell what you want to connect to, and when it completes it gives you back a connection. At the moment that's just doing 'normal' SRV lookups and picking a service to which you connect. But there's no reason that shouldn't give you an appropriate type of connection based on available transports.

right now the client/library *must* decide, in the future that becomes a question I guess

(Connection here being an abstract interface that it gives you a concrete instance of)

Kev: thats more or less what I do too. The difference here is that right now the library doesn't care if the connection you hand it is a TCP socket, a Unix domain socket, an in-memory pipe, or even just a string buffer with some pre-configured responses as long as it can read/write to it. If I add websocket it has to know what type the connection is and do a different handshake

The 'Connection' for us abstracts away the details like that.

So (if we did wss, which we don't yet), the wss handshake would be done inside the Connection.

Ah, I see, my "connection" is just something with Read/Write methods

There is a separate concept, the Negotiator, that does the handshake and hands you a session (so right now I have a default negotiator that does XMPP feature stuff, a websocket one that's the same but it uses <open/> instead of <stream> and a component one)

Prosody has a thing where you combine a network resolver strategy with a thing that executes it and returns a connection.

This all makes me think I need to at least provide a way to be explicit and provide the low-level building blocks (so a websocket.Dial function or something), but at a later date I can add websocket support directly to the higher level thing that resolves SRV records and the like. Best of both worlds.

Sam: Ah, right, you mean the application level wss exchange, not the wss transport exchange. Yes, you're right, my bad.

Kev: oh yah, sorry, I wasn't very clear there looking back

yea a "how to connect" flag from the application/client seems appropriate, so today you'd have "SRV" or "wss" or "any" and next year you might add "srv2", maybe?

Describing how M-Link handles this would probably take too long, but it has two layers of this abstraction. One for the connection type, and one for the session type. So you'd have a WSS connection, and a Websocket Client Session or such.

Oh hey, I already implemented XEP-0156 then forgot about it and never used it. That's convenient.

"bye-bye stream management, we don't need you anymore" stream management could still be used for e.g. TCP to websocket handovers

or between QUIC and those yea

I don't think I've ever noticed this before, but the Web Host Metadata format in XEP-0156 doesn't actually say if the URLs are for c2s or s2s connections. The TXT format does though. I'm not sure what to do about that.

Just try them all if it's from the host metadata file, I guess.

I’ll see what my stuff does, sec

treats them all as c2s

note that s2s TXT records are not specified by '156 ✏

-> https://xmpp.org/extensions/xep-0156.html#registrar-altconn-values

Thanks:msay begin

oops, sorry

it also talks about clients all the time

> The attribute name SHOULD begin with the string "_xmpp-client-" or "_xmpp-server-" It doesn't register it, but the document is the normative text, not the registry I assume

in the same sentence it says though that they should be registered

my takeaway is that currently there’s no spec which uses the `_xmpp-server-` prefix

Sure, and then it doesn't do that. That just says that they should be registered, not that the registry is normative

it also doesn’t define it elsewhere

(what `_xmpp-server-$thing` is supposed to mean)

I mean, it specifically says that prefix exists, that sounds like defining it

And I can't imagine what else "server" would mean other than "s2s" if we assume "client" means "c2s"

not covered by the spec though

I don't understand why not? Nothing else says "this is only for c2s" as far as I see

the linked RFCs do, though

for the registered protocols that is, anyway

What RFCs?

XEP-0205 says it’s c2s, RFC 7395 also talks about client-to-server

(I’ll just ignore XEP-0025 (!= XEP-0205) for it being obsolete) ✏

so even if you made up the _xmpp-server-xbosh thing, it’s unclear how that’d even work given that '206 talks about c2s only

Oh, you mean there's no s2s for websocket at all? I mean, I don't see how that would be all that useful, but I also don't see why not. Just grepping around in 7395 I don't see anything that forbids it either

(similarly for websockets)

afk

oh, nevermind, I see where it says that in the introduction

Fair enough, if I can ignore s2s that sounds good to me. Maybe the -server thing was just leftover from when normal TCP stuff could also be in here.

or just as a provision for future things

So. How many alternative connection methods are there now?

Zash: Last time I counted it was "too many"

but we can just add 1 more using SRV2 to replace them all! https://xkcd.com/927/

Is SRV2 actually a thing anyone is working on?

more than that, it's a requirement for HTTP3

so all browsers etc have already adopted it

I can't say what they actually decided on for a name in here or risk being burned at the stake

> The specific name for this RR type is an open topic for discussion. "SVCB" and "HTTPSSVC" are meant as placeholders as they are easy to replace. Other names might include "B", "SRV2", "SVCHTTPS", "HTTPS", and "ALTSVC".

I thought "HTTPS" was already decided.

moparisthebest: what's already adopted by HTTP3 browsers?

This I-D?

I believe so, maybe an older version, or the one it replaced

Maybe I misunderstood but I got the impression that it requires DNS software to adopt additional processing rules.

oh that would be bad, I didn't get that impression, but it's been awhile...

There are words in that direction here: https://datatracker.ietf.org/doc/html/draft-ietf-dnsop-svcb-https#section-4

I was just reading that

I wrote a parser for this and was hit by how overly complex the RR type is compared to just about everything else in DNS.

unless I missed something those are all SHOULD include related records in Additional section, which, doesn't impose any requirements if a DNS server doesn't understand the format, just makes full resolution harder on the client

and, most but not all DNS servers already do this with SRV etc right?

check this for example: `dig srv _xmpps-client._tcp.moparisthebest.com @ns1.moparisthebest.com`

SRV queries including A+AAAA? Yeah

it returns all associated A and AAAA and even my TLSA in there

Oh!

so that's just like "that nice thing you do for SRV now? it'd also be nice if you did it for SRV2!"

Bugs me a bit after 20 years of refusing to implement SRV

yea, suddenly YOU KNOW WHAT WOULD BE A GOOD IDEA :P

I'd call NIH if not for the additional stuff included, tags and such

Extensibility ...

shoulda put XML in those dns records...

You can, SVCB is extensible!

Many SRV records have parameters with registries that can be added to, but this lets you add entire new formats inside SVCB/HTTPS, which seems a bit too much

but that's needed for "encrypt your SNI with this key" and "use http3 here" or "use http2 here" or later "encrypt your alpn with this key" etc

it's what'll let us replace all the existing bosh/wss/tls/starttls stuff with 1 lookup in the future

DANE

As you said, returning related records to a SRV query is entirely possible

Yah I don't really understand the difference between this, SRV records, and URL records (or whatever they were which I already thought were just more flexible SRV records)

how would you share a key to encrypt the tls handshake with those other ones? different per host mind you

Make it part of the URI with URL records I guess (although I barely remember how they work)

but okay, sounds like this is merging more stuff than I was thinking

then URL, https:// could mean HTTP 1.1, 2, or 3

it's a big ball of wax I'll give you that

Sam Whited, mostly off-topic, but have you ever looked at ActivityPub?

Zash: vaguely, once, a long time ago. I don't remember anything about it really

Why?

Because it's made out of JSON-LD, which is the JSON encoding of RDF 😛

Do they actually define what you have to send in one place with actual docs?

I don't really care what the encoding is all that much as long as it's not spread over 5 sites and/or only gives you a blob of XML in lieu of docs.

I didn't even know that was a thing.

That's why I'm so pissed at our usage of DOAP. Any time I ask for docs on what I can actually write someone gives me a blob of XML schema that I have no idea what it's telling me to do.

A whole pile of xmlns-prefixes isn't any prettier in JSON, IMO.

I don't care if it's JSON or XML especially.

HAHA it seems to vary

https://www.w3.org/ns/activitystreams# works as docs at least

But from what I gather, everyone just copies what Mastodon does anyways.

I don't know about this particular thing, and I'm sure people will always just copy the original implementation up to a certain point, but if that's the recommended way to do it (as it was with DOAP) that means your spec is too complicated and broken and you should do something else IMO.

I firmly believe it's always better to define behavior in a spec than having a reference implementation, and I fail to see why we're defining a spec if you can't actually implement it from that and people just say "just copy/paste, change values, and hope it's all correct".

Oh wow, it seems that my blog is still syndicated at planet.jabber.org despite the fact that I rarely publish anything tech-related anymore. Sorry about that. ;-) At least my latest post is about RFC 8838...

Nothing wrong with having your blog there, or having non-techical content on planet.

Well, perhaps this will inspire me to post more often about technical topics.