jdev - 2021-01-21

  1. jubalh

    Could someone tell me about a library that implements XEPs? I would like to see how such libraries/their API is designed

  2. flow

    jubalh: https://github.com/igniterealtime/Smack/tree/master/smack-extensions/src/main/java/org/jivesoftware/smackx

  3. jonas’

    jubalh, https://docs.zombofant.net/aioxmpp/devel/api/index.html maybe?

  4. jubalh

    thanks flow & jonas’

  5. jubalh

    jonas’: who uses aioxmpp again?

  6. jonas’

    jubalh, some people do, I don’t know who they are

  7. jonas’

    which is nice, in a way

  8. jonas’

    I sometimes get bug reports ;)

  9. Martin

    Don't you use it for s.j.n?

  10. jubalh

    I see, I thought maybe there was something popular. I'm not up to date on which client uses what

  11. jonas’

    Martin, yeah well, I thought it was obvious that I use it :)

  12. jonas’

    jubalh, no client has been ported to aioxmpp

  13. jubalh


  14. jonas’

    it seems to be not unpopular in IoT applications, judging by the questions/bug reports I get

  15. jubalh

    thats interesting

  16. Ge0rG

    Internet of Python Things?

  17. jonas’

    Internet of Schlangs

  18. DebXWoody

    jubalh: gloox

  19. Martin

    Doesn't IoT have mosquitos?

  20. Sam Whited

    jubalh: https://pkg.go.dev/mellium.im/xmpp as well

  21. jubalh


  22. Sam Whited

    After discussing it yesterday I went and implemented the WebSocket subprotocol. It was suprisingly easy and required very minimal changes despite the lack of a root element. Now if I can get it working with HTTP2 I can avoid BOSH altogether, so thanks to whomever suggeseted the HTTP2/Websocket RFC.

  23. moparisthebest

    but will it work with HTTP3 :)

  24. Sam Whited

    No idea, I'm sure it will eventually if it doesn't work to start.

  25. Sam Whited

    I keep expecting new versions of HTTP to just have bidirectional streams built in (I thought server push in HTTP2 was going to be this, but for various reasons it's not and can't really be used that way)

  26. moparisthebest

    my impression is at the HTTP level, 2 and 3 are pretty close to the same, it's just underneath and above that has completely changed

  27. Zash

    Didn't people conclude recently that HTTP/2 push was a failure?

  28. Sam Whited

    Zash: probably. It's not reliable to say the least. Although I'd say it was an issue with how it was advertised, not the actual feature. If you just use it for what it was designed for (preloading images and things on the page) it's pretty great and when it's unreliable because a proxy drops it or whatever you just go back to the same thing you had before, no worse. It was a nice optimization when it could be used.

  29. moparisthebest

    like http2 is (http2 over TLS over TCP) and http3 is (http3 over QUIC (combined TLS and TCP guarantees but over UDP))

  30. Zash

    Can I be upset that they seem to have invented invented TCP + IPSec?

  31. Sam Whited

    No, because it's not the same thing at all. My limited understanding of QUIC is that it does a better job than TCP of congestion control (which is what all the tons of complicated HTTP stuff multiplexing various streams was trying to work around)

  32. Sam Whited

    I'd be curious what differences are in the HTTP layer if it's mostly the same just HTTP over QUIC. Probably getting rid of all the multiplexing stuff since there's less tail time now, but otherwise?

  33. Zash

    There are multiple TCP congention control strategies.

  34. Zash

    But maybe it was more SCTP, with multiple independent streams over the same connection.

  35. Sam Whited

    Zash: and they're all terrible. No matter what you still have the TCP warmup period, as far as I know. QUIC is supposed to be a lot more efficient (though I admit, I've just seen the numbers from Google, I don't really know how it works)

  36. Sam Whited

    I dunno, it's about time we replaced TCP IMO, so I'm hopeful that this is really better.

  37. Sam Whited

    It also combines the TLS/TCP handshakes, which is nice. Less stuff to do before you can actually start sending bytes, which has been a significant problem at some companies I've worked at

  38. Sam Whited is looking at the differences now, there's a lot more here than I realized

  39. moparisthebest

    in addition to combining TLS+TCP handshakes, it also supports network roaming (bye-bye stream management, we don't need you anymore), and (I guess not useful for XMPP) fixes head-of-line blocking

  40. Sam Whited

    The head-of-line blocking thing will be a big one, I've run into that a lot with HTTP/2 streams

  41. moparisthebest

    not having to redo connections on network changes alone is a big enough reason for XMPP to use it imho

  42. Sam Whited


  43. Sam Whited

    I see a lot of discussion about websockets, but no actual upgrade mechanism being proposed

  44. Sam Whited

    Although apparently there's something called WebTransport that I don't fully understand that offers bidirectional streams with arbitrary protocols on top? Maybe this will just replace websockets

  45. Sam Whited

    (and it supports QUIC)

  46. Sam Whited

    Now as usual I fall back to the question of what packages this should all be in. I could have the current negotiator just have a websocket option and the dial package (which does SRV lookups and creates TCP connections) support websockets, or I could have a new websocket package that has a custom dialer, a custom negotiator, and a bunch of helper methods to eg. dial and negotiate an XMPP session all in one Go that mirrors the functions in the main XMPP package that do that except for TCP.

  47. Sam Whited

    Organizing code is hard.

  48. Sam Whited

    Actually, I wonder if I can make this completely transparent from the client side: the dialer would do all the lookups and dial TCP if it can or websocket if that's all that's available (or if a PreferWebsocket option is set or something) and then the session negotiation would just check what type of connection it is and use that protocol. That seems bad somehow, but I'm not sure why.

  49. Sam Whited

    Probably best to just always let the user be explicit about how they're connecting.

  50. moparisthebest

    Sam Whited, in a theoretical near-future where a XEP that supports lookup with SRV2 exists, fully transparent makes sense

  51. moparisthebest

    as that lets the server admin set preferences across websocket, TLS, quic etc

  52. moparisthebest

    right now SRV and websocket have no relationship so having your library decide priority seems bad?

  53. Sam Whited

    That's a fair point, the question is really "shoudl the client dev or the server admin decide how we connect"

  54. Kev

    Sam: FWIW, in Swiften we have a Connector that you tell what you want to connect to, and when it completes it gives you back a connection. At the moment that's just doing 'normal' SRV lookups and picking a service to which you connect. But there's no reason that shouldn't give you an appropriate type of connection based on available transports.

  55. moparisthebest

    right now the client/library *must* decide, in the future that becomes a question I guess

  56. Kev

    (Connection here being an abstract interface that it gives you a concrete instance of)

  57. Sam Whited

    Kev: thats more or less what I do too. The difference here is that right now the library doesn't care if the connection you hand it is a TCP socket, a Unix domain socket, an in-memory pipe, or even just a string buffer with some pre-configured responses as long as it can read/write to it. If I add websocket it has to know what type the connection is and do a different handshake

  58. Kev

    The 'Connection' for us abstracts away the details like that.

  59. Kev

    So (if we did wss, which we don't yet), the wss handshake would be done inside the Connection.

  60. Sam Whited

    Ah, I see, my "connection" is just something with Read/Write methods

  61. Sam Whited

    There is a separate concept, the Negotiator, that does the handshake and hands you a session (so right now I have a default negotiator that does XMPP feature stuff, a websocket one that's the same but it uses <open/> instead of <stream> and a component one)

  62. Zash

    Prosody has a thing where you combine a network resolver strategy with a thing that executes it and returns a connection.

  63. Sam Whited

    This all makes me think I need to at least provide a way to be explicit and provide the low-level building blocks (so a websocket.Dial function or something), but at a later date I can add websocket support directly to the higher level thing that resolves SRV records and the like. Best of both worlds.

  64. Kev

    Sam: Ah, right, you mean the application level wss exchange, not the wss transport exchange. Yes, you're right, my bad.

  65. Sam Whited

    Kev: oh yah, sorry, I wasn't very clear there looking back

  66. moparisthebest

    yea a "how to connect" flag from the application/client seems appropriate, so today you'd have "SRV" or "wss" or "any" and next year you might add "srv2", maybe?

  67. Kev

    Describing how M-Link handles this would probably take too long, but it has two layers of this abstraction. One for the connection type, and one for the session type. So you'd have a WSS connection, and a Websocket Client Session or such.

  68. Sam Whited

    Oh hey, I already implemented XEP-0156 then forgot about it and never used it. That's convenient.

  69. flow

    "bye-bye stream management, we don't need you anymore" stream management could still be used for e.g. TCP to websocket handovers

  70. moparisthebest

    or between QUIC and those yea

  71. Sam Whited

    I don't think I've ever noticed this before, but the Web Host Metadata format in XEP-0156 doesn't actually say if the URLs are for c2s or s2s connections. The TXT format does though. I'm not sure what to do about that.

  72. Sam Whited

    Just try them all if it's from the host metadata file, I guess.

  73. jonas’

    I’ll see what my stuff does, sec

  74. jonas’

    treats them all as c2s

  75. jonas’

    note that s2s is not specified by '156

  76. jonas’

    note that s2s TXT records are not specified by '156

  77. jonas’

    -> https://xmpp.org/extensions/xep-0156.html#registrar-altconn-values

  78. Sam Whited

    Thanks:msay begin

  79. Sam Whited

    oops, sorry

  80. jonas’

    it also talks about clients all the time

  81. Sam Whited

    > The attribute name SHOULD begin with the string "_xmpp-client-" or "_xmpp-server-" It doesn't register it, but the document is the normative text, not the registry I assume

  82. jonas’

    in the same sentence it says though that they should be registered

  83. jonas’

    my takeaway is that currently there’s no spec which uses the `_xmpp-server-` prefix

  84. Sam Whited

    Sure, and then it doesn't do that. That just says that they should be registered, not that the registry is normative

  85. jonas’

    it also doesn’t define it elsewhere

  86. jonas’

    (what `_xmpp-server-$thing` is supposed to mean)

  87. Sam Whited

    I mean, it specifically says that prefix exists, that sounds like defining it

  88. Sam Whited

    And I can't imagine what else "server" would mean other than "s2s" if we assume "client" means "c2s"

  89. jonas’

    not covered by the spec though

  90. Sam Whited

    I don't understand why not? Nothing else says "this is only for c2s" as far as I see

  91. jonas’

    the linked RFCs do, though

  92. jonas’

    for the registered protocols that is, anyway

  93. Sam Whited

    What RFCs?

  94. jonas’

    XEP-0205 says it’s c2s, RFC 7395 also talks about client-to-server

  95. jonas’

    (I’ll just ignore '25 for it being obsolete)

  96. jonas’

    (I’ll just ignore XEP-0025 (!= XEP-0205) for it being obsolete)

  97. jonas’

    so even if you made up the _xmpp-server-xbosh thing, it’s unclear how that’d even work given that '206 talks about c2s only

  98. Sam Whited

    Oh, you mean there's no s2s for websocket at all? I mean, I don't see how that would be all that useful, but I also don't see why not. Just grepping around in 7395 I don't see anything that forbids it either

  99. jonas’

    (similarly for websockets)

  100. jonas’


  101. Sam Whited

    oh, nevermind, I see where it says that in the introduction

  102. Sam Whited

    Fair enough, if I can ignore s2s that sounds good to me. Maybe the -server thing was just leftover from when normal TCP stuff could also be in here.

  103. jonas’

    or just as a provision for future things

  104. Zash

    So. How many alternative connection methods are there now?

  105. Sam Whited

    Zash: Last time I counted it was "too many"

  106. moparisthebest

    but we can just add 1 more using SRV2 to replace them all! https://xkcd.com/927/

  107. Sam Whited

    Is SRV2 actually a thing anyone is working on?

  108. moparisthebest

    more than that, it's a requirement for HTTP3

  109. moparisthebest

    so all browsers etc have already adopted it

  110. moparisthebest

    I can't say what they actually decided on for a name in here or risk being burned at the stake

  111. moparisthebest drops this and runs https://tools.ietf.org/html/draft-ietf-dnsop-svcb-httpssvc-03

  112. Zash cries at the mention of SRV2

  113. moparisthebest

    > The specific name for this RR type is an open topic for discussion. "SVCB" and "HTTPSSVC" are meant as placeholders as they are easy to replace. Other names might include "B", "SRV2", "SVCHTTPS", "HTTPS", and "ALTSVC".

  114. Zash

    I thought "HTTPS" was already decided.

  115. Sam Whited

    moparisthebest: what's already adopted by HTTP3 browsers?

  116. Sam Whited

    This I-D?

  117. moparisthebest

    I believe so, maybe an older version, or the one it replaced

  118. Zash

    Maybe I misunderstood but I got the impression that it requires DNS software to adopt additional processing rules.

  119. moparisthebest

    oh that would be bad, I didn't get that impression, but it's been awhile...

  120. Zash

    There are words in that direction here: https://datatracker.ietf.org/doc/html/draft-ietf-dnsop-svcb-https#section-4

  121. moparisthebest

    I was just reading that

  122. Zash

    I wrote a parser for this and was hit by how overly complex the RR type is compared to just about everything else in DNS.

  123. moparisthebest

    unless I missed something those are all SHOULD include related records in Additional section, which, doesn't impose any requirements if a DNS server doesn't understand the format, just makes full resolution harder on the client

  124. moparisthebest

    and, most but not all DNS servers already do this with SRV etc right?

  125. moparisthebest

    check this for example: `dig srv _xmpps-client._tcp.moparisthebest.com @ns1.moparisthebest.com`

  126. Zash

    SRV queries including A+AAAA? Yeah

  127. moparisthebest

    it returns all associated A and AAAA and even my TLSA in there

  128. Zash


  129. moparisthebest

    so that's just like "that nice thing you do for SRV now? it'd also be nice if you did it for SRV2!"

  130. Zash

    Bugs me a bit after 20 years of refusing to implement SRV

  131. moparisthebest


  132. moparisthebest

    I'd call NIH if not for the additional stuff included, tags and such

  133. Zash

    Extensibility ...

  134. moparisthebest

    shoulda put XML in those dns records...

  135. Zash

    You can, SVCB is extensible!

  136. Zash

    Many SRV records have parameters with registries that can be added to, but this lets you add entire new formats inside SVCB/HTTPS, which seems a bit too much

  137. moparisthebest

    but that's needed for "encrypt your SNI with this key" and "use http3 here" or "use http2 here" or later "encrypt your alpn with this key" etc

  138. moparisthebest

    it's what'll let us replace all the existing bosh/wss/tls/starttls stuff with 1 lookup in the future

  139. Zash


  140. Zash

    As you said, returning related records to a SRV query is entirely possible

  141. Sam Whited

    Yah I don't really understand the difference between this, SRV records, and URL records (or whatever they were which I already thought were just more flexible SRV records)

  142. moparisthebest

    how would you share a key to encrypt the tls handshake with those other ones? different per host mind you

  143. Sam Whited

    Make it part of the URI with URL records I guess (although I barely remember how they work)

  144. Sam Whited

    but okay, sounds like this is merging more stuff than I was thinking

  145. moparisthebest

    then URL, https:// could mean HTTP 1.1, 2, or 3

  146. moparisthebest

    it's a big ball of wax I'll give you that

  147. Zash

    Sam Whited, mostly off-topic, but have you ever looked at ActivityPub?

  148. Sam Whited

    Zash: vaguely, once, a long time ago. I don't remember anything about it really

  149. Sam Whited


  150. Zash

    Because it's made out of JSON-LD, which is the JSON encoding of RDF 😛

  151. Sam Whited

    Do they actually define what you have to send in one place with actual docs?

  152. Sam Whited

    I don't really care what the encoding is all that much as long as it's not spread over 5 sites and/or only gives you a blob of XML in lieu of docs.

  153. Zash

    I didn't even know that was a thing.

  154. Sam Whited

    That's why I'm so pissed at our usage of DOAP. Any time I ask for docs on what I can actually write someone gives me a blob of XML schema that I have no idea what it's telling me to do.

  155. Zash

    A whole pile of xmlns-prefixes isn't any prettier in JSON, IMO.

  156. Sam Whited

    I don't care if it's JSON or XML especially.

  157. Zash

    HAHA it seems to vary

  158. Zash

    https://www.w3.org/ns/activitystreams# works as docs at least

  159. Zash

    But from what I gather, everyone just copies what Mastodon does anyways.

  160. Sam Whited

    I don't know about this particular thing, and I'm sure people will always just copy the original implementation up to a certain point, but if that's the recommended way to do it (as it was with DOAP) that means your spec is too complicated and broken and you should do something else IMO.

  161. Sam Whited

    I firmly believe it's always better to define behavior in a spec than having a reference implementation, and I fail to see why we're defining a spec if you can't actually implement it from that and people just say "just copy/paste, change values, and hope it's all correct".

  162. stpeter

    Oh wow, it seems that my blog is still syndicated at planet.jabber.org despite the fact that I rarely publish anything tech-related anymore. Sorry about that. ;-) At least my latest post is about RFC 8838...

  163. Zash

    Nothing wrong with having your blog there, or having non-techical content on planet.

  164. stpeter

    Well, perhaps this will inspire me to post more often about technical topics.