jdev - 2024-05-16

moparisthebest: Can you now please explain to me, why text parsing is more efficient than working with binary data?

It often is not

ikr

Victorious: there's a few things, one is the "text is wasteful in terms of bandwidth" when in actuality it takes the same bandwidth to send 1 byte or 1000 bytes over the internet where MTU is always 1500

>when in actuality it takes the same bandwidth to send 1 byte or 1000 bytes over the internet where MTU is always 1500 you can always send a few messages together

message in more broad meaning, that includes service-related stuff

Sure, in both cases. now come up with a binary encoding as eXtensible as XML but that's smaller than XMPP with compression, I'll wait

EXI is an existing attempt at this that virtually no one has bothered adopting because it doesn't matter in the real world

>now come up with a binary encoding as eXtensible as XML why some chat protocol needs that level of extensibility?

moparisthebest: if there are no more arguments, I'll leave

Victorious: seems like you just came here to pick a fight

singpolyma: I was invited here to continue one discussion, nothing more.

Victorious: XMPP is not a "chat protocol"

Binary isn't a magic bullet, but sure if the only thing you care about is efficiency of size or decode speed then a bespoke packed binary will always be best at those things

moparisthebest: in other chat we were talking about chat protocols, xmpp was just mentioned.

Regardless, even in chat only uses, it's obvious it needs that eXtensiblity by... All the extensions? And everything not-xmpp attempting to poorly recreate it, see: matrix, IRCv3 etc etc

Luckily matrix and IRC are also text so not relevant 😉

singpolyma: yeah, thats my point. But binary protocols are harder to design :(

To clarify, XMPP is not only for chat

Victorious: I would never suggest bespoke packed binary for a standardized protocol

singpolyma: what extensions? I'm not a xmpp user

XMPP has a binary only protocol named EXI if that's what you want, but in the real world, not much use

sorry, that message was for moparisthebest

Well, EXI isn't xmpp related, but xmpp implementations with very specific needs could use it

https://xmpp.org/extensions/

EXI is https://xmpp.org/extensions/xep-0322.html

sigh, now I need to waste my time on yet-another-protocol and its extensions just for a quick discussion

moparisthebest: if we required everything to go via the XSF we could have packed binary that we extended sanely. But it's very hard to do ad-hoc extension, which I would argue is important at internet scale

>https://xmpp.org/extensions/ no wonder xmpp isn't widespread

there is like six gorillion extensions

>>https://xmpp.org/extensions/ > > no wonder xmpp isn't widespread Hmm. It is widespread actually? Not sure what you mean ↺

(some of which can be realized client-side)

No one cares about that extensions list just ignore moparisthebest 😛

>Hmm. It is widespread actually? Not sure what you mean maybe, as I've said - I'm not a xmpp person.

In the end the reason almost every internet protocol is text is to make debugging easier, honestly

as for extensions: properly designed protocol may escape the need for them

yeah, its just easier to understand but for machines its harder to process

And yes, most internet protocols also support ad hox extensibility because we can never assume to have thought of every use case someone might have

> yeah, its just easier to understand > > but for machines its harder to process Not enough harder to matter in 99% of cases though. Which is why its so popular ↺

binary protocols have to be incompatibly redone whenever the schema changes a little. a design like xmpp's avoids that

>And yes, most internet protocols also support ad hox extensibility because we can never assume to have thought of every use case someone might have Agree. But in modern age it isn't hard to ignore unsupported features (like telegram client does) and it's not hard to break compatibility

graceful degradation is a desirable feature

wgreenhouse: well, you can have extensible binary just fine. But not ad hoc extensible, at least not well ✏

>binary protocols have to be incompatibly redone whenever the schema changes a little that depends on the protocol design if its designed for maximum efficiency, it will be hard to maintain compatibility without bit/byte sacrifices

I honestly don't know if there are any standardized, widespread, binary internet protocols

In the 80s even we were using text protocols already

QUIC

> QUIC Sorry, I meant layer7. Obviously tcp ip itself is binary ↺

quic is also replacement for text-based http

there is xmpp over quic

xmpp is still based on xml, so it doesnt matter what transport protocol is being used

> quic is also replacement for text-based http Oh you mean http3? Pretty sure it still allows arbitrary headers... ↺

>Oh you mean http3? Pretty sure it still allows arbitrary headers... yeah, and binary-based chat protocols also carry text messages

Sure, but with http the verb and headers are the protocol... So if they're both still text I don't know how it's not a text protocol? I guess you no longer have to split on colon, perhaps a small win there at scale

Xmpp is a bit higher level than http I think. And instant messaging is changing a lot over the years (threads, message reactions are some examples tht come to mind, but this happens all the time). So that explains why a very flexible protocol is needed, and with xml namespaces this is easy to do. With a binary protocol you need a registrar to allocate identifier numbers, and then people don't fill in the proper paperwork and pick whatever number they think is unused

If in your case you think you don't have this specific problem, sure, you can try a binary protocol. You will surely occasionally need to replace it with an incompatible version, but that can also be acceptable in some usages

>With a binary protocol you need a registrar to allocate identifier numbers pulkomandy: There is no need for a registrar, just use a header or version of protocol. There is no registrar for binary file formats!

There is, for example for iff chunks or png chunks

good point, but there is no internet police that will imprison a person that makes a new binary protocol

But again this really depends on your needs. In xmpp it is really about having several independant extensions, and each client or server can pick what it needs and ignore the rest. This works well for xmpp, maybe not so well in other places

No, but if you want several apps to talk with each other, you have to set some rules. And if the rules are "I made a new version of the protocol and all old apps are incompatible with it", it seems not great?

I agree with you, but that sometimes happens. Clients/servers need upgrades, e.g. for security reasons. It shouldn't be a problem.

> as for extensions: properly designed protocol may escape the need for them Lol no, exactly the opposite ↺

I mean, sometimes the XMPP community acts as though ad hoc extensability is bad, wanting everything to be a xep and only to use the urn:xmpp namespace. But I personally think it's important and am glad we have it

>Lol no, exactly the opposite can you explain? the way I see it, perfect chat protocol should just pass messages, it must care only about some very basic things, rest is up to a client and server software

Victorious: if everything iskup to the client anhid server that's exactly when you need ad hoc extensibility

So the client and server can safely make stuff up that don't clash with each other

singpolyma: I guess a protocol version should be enough.

tbh I think I need to start drafting something binary - then I'll have more to share with you

If you have a protocol version only then the client and server can't make stuff up and everything must be in the protocol

It's not just about passing the data around. The client also needs to know how to present the data to the user. And in xmpp, the server also does more than passing data around. You can query history for a chat room, upload files, do audio/video chats, store stuffon the server and query it (for example, your contact list)

So it's not just about the protocol. Sure, it's easy to define something like mqtt, at the protocol level this is binary and doesn't need extensions. But then, it's just a wire protocol, and you can't do much more with it. And so you will have the extensibility in the layers above it

And more layers means more complexity and overhead, usually

... copy paste from xsf@ ... Hi. A question about avatar conversion (xep-398). What's going to happen if I publish my vcard without photo? Will it reset avatar in pubsub? XEP says "A service MAY support conversion from vCard to PEP in order to display avatars in clients that only support XEP-0084.". I assume it means if two clients are connected (one publishing vcard and second one supporting just xep-84, then the second one MAY receive pubsub update). and that's all. From other side the xep insist all the control to be done via pubsub. So it's no clear should I always add an avatar to the publishing vcard or not.

Victorious: note XMPP isn't a "protocol used only for chat" or "between a client and a server" , it's a federated message passing protocol meaning it's between clients and servers and also between servers and other servers, if you try to invent your own, you'll reinvent XMPP, it's inevitable and repeats itself every so often That's why most of the world just uses XMPP...