-
jjj333_p (any pronouns)
how insane would it be for flexibility's sake to roll my own xmpp library, just me, the golang tcp stack, and an xml parsing library
-
singpolyma
For any reason other than learning I would strongly suggest to not since several already exist. But it wouldn't be insane
-
jjj333_p (any pronouns)
mellium is what ive been looking at so far, but i mainly want the ability to read and send arbitrary xml data in stanzas. this may be possible, im just chewing on ideas
-
singpolyma
yeah mellium is what I have used before. I know there are one or two others
-
jjj333_p (any pronouns)
do you know if it allows for that?
-
jjj333_p (any pronouns)
and yeah theres go-xmpp (more bot oriented, more simple less flexible) and go-sendxmpp which is a tool not a lib
-
singpolyma
You can add arbitrary XML payloads for sure
š 1 -
moparisthebest
> how insane would it be for flexibility's sake to roll my own xmpp library, just me, the golang tcp stack, and an xml parsing library Literally everyone does it, it's easy ↺
š 1 -
jjj333_p (any pronouns)
alr
-
Zash
Only to give up halfway trough implementing an xml parser from scratch and go on to loudly point out how bad XMPP is?
-
doge
"Parsing" xml even when you already have an xml parser is trickier than it seems, by "parsing" I mean building the insane incomplete DOMs which no general purpose xml library supports. You have to keep track of namespaces, namespace prefixes. For example, You can have expat readily convert ns prefixes to namespaces for you, but then you will not be able to see the actual prefix names, which are actually needed for namespaced attributes...
-
jonasā
what?
-
jonasā
doge, can you elaborate?
-
jonasā
I don't see how you need to track namespace prefixes on the application layer to process namespaced attributes (source: I have written a parser which does not expose prefixes to the application and fully supports namespaced attributes)
-
Zash
Prosody doesn't and will either drop namespace prefixes completely or turn them into ns1:foo="" etc. XMPP hasn't collapsed, therefore I conclude nothing really depends on namsepace prefixes.
-
jonasā
*nod*
-
doge
That is what I did in my parser too, but I'm just saying that it's an example of absolutely pointless complexity.
-
doge
I am not too familiar with XML, I never worked with it outside of xmpp, but reading this https://stackoverflow.com/questions/3931817/xpath-expression-from-xml-with-namespace-prefix/8181480#8181480 I got the impression that xpath is sensitive to these prefixes. Luckily, as far as I know, XMPP libraries do not use XPath anyway because they just come up with their own ad hoc partial dom implementations
-
jonasā
the xpath _syntax_ is indeed not great for that
-
jonasā
the semantics are not sensitive to prefixes. there are other xpath-ish things which use a different syntax allowing you to specify the namespace URI instead of the prefix.
-
doge
Alright, I just remembered that initially I wanted to use an existing dom library but didn't because of the mess prefixes and namespaces created. Turns out to select ns'd node, you have to also register your local prefixes...✎ -
doge
ah right, I just remembered that initially I wanted to use an existing dom library but didn't because of the mess prefixes and namespaces created. Turns out to select ns'd node, you have to also register your local prefixes... ✏
-
Zash
I find I haven't really needed to care much about namespace prefixes, so I don't see the complexity in that. Good demonstration of why you should just use a library tho, or you end up talking like this, often loudly.
-
doge
ah right, I just remembered that initially I wanted to use an existing dom library but didn't partially because of the mess prefixes and namespaces created. Turns out to select ns'd node, you have to also register your local prefixes... ✏
-
doge
Well, if you are building your own parse, you will need to encounter this. There was actually, I think, only one instance where I saw an attribute prefix in the wild, and it made my initial parser implementation crash.
-
doge
> or you end up talking like this, often loudly.
-
Zash
Web people don't write their own JSON or HTTP parsers, or they'd complain as much about those :)
-
doge
https://draugr.de/upload/00f530bb49b4e1685114d28f8bb6fb44b2fcbc2f/r5dyhkxKI6qSGSkUuf6fhle87sKijlOgHIdpUMyl/zb2rhfKs4RBkthZMMj75v2DJw6knXYFq5B3rCGyqCwevNjScd.png
-
doge
> Web people don't write their own JSON or HTTP parsers, or they'd complain as much about those :) Actually you wouldn't need to write an XML parser either if XMPP made use of XML in a remotely sane way. ↺
-
Zash
Now, let me tell you about all the quirks we've had with our HTTP and JSON parsers in Prosody! ;)
-
Zash
> Actually you wouldn't need to write an XML parser either if XMPP made use of XML in a remotely sane way. I think it's perfectly sane. ↺
-
jonasā
you need a SAX parser, sure
-
Zash
or better yet, an XMPP library that handles all that for you so you can build your app instead of fiddle with low level protocol fiddlery
-
jonasā
or that, yes.
-
kalkin
> or better yet, an XMPP library that handles all that for you so you can build your app instead of fiddle with low level protocol fiddlery I think I have a dejavu. Just a week ago I told doge he doesn't want to write his own library. Moparisthebest said he shouldn't listen to me, this is fine š
-
doge
I had already made a parser by then lol
-
doge
Btw it's not like the existing libraries are terribly well documented...
-
Zash
kalkin, did moparisthebest semi-jokingly point out that basically every popular client maintains their own library? I don't think that's a good argument to start from scratch.
-
doge
Like for example if we take not even a full xmpp library but an omemo library. The only one I could find that could be said to be documented at all would be an async python one which has its own complications. Actually, I think that one has quite thorough documentation, but I mean What few C libraries I coukd find have basically no documentation at all.
-
doge
> kalkin, did moparisthebest semi-jokingly point out that basically every popular client maintains their own library? I don't think that's a good argument to start from scratch. Maybe that is actually a good argument because there's gotta be a reason why they do it from scratch. Repeating after others who have already succeeded is often a good strategy. ↺
-
kalkin
doge that's an interesting interpretation
-
kalkin
Any Dev I talked to, said it sucks that they don't use some 3rd party li✎ -
kalkin
Any Dev I talked to, said it sucks that they don't use some 3rd party lib ✏
-
kalkin
> kalkin, did moparisthebest semi-jokingly point out that basically every popular client maintains their own library? I don't think that's a good argument to start from scratch. I think this was implied. š
-
Zash
I would imagine it's more often of necessity, for lack of a suitable library. Or they ended up forking or maintaining an existing one
-
doge
For one, libraries tend to be language specific. It's not as bad if it's a C library, It tends to be easy to port it and to ffi it from a higher level language. And even then, it usually won't be idiomatic for your language. worse yet you have to tie it into your event loop or spawn another thread, and both have complications of their own. That's not to mention that I don't think there are well-maintained c libraries. There is python, but you now have to carry around and embed a python interpreter. You have to code python to glue. At that point you might as well switch your entire project to python, and maybe you don't want to do it that much.
-
doge
In the case of dino where they opted to use vala, I don't think reusing any library other than C would've been feasible, and C libraries don't really exist afaik
-
doge
And also let's not forget people just like tinkee and they tend to like to tinker in their own language and in their own code. Since nobody is really paying for a new client, the development process has to be fun first and foremost.
-
jonasā
"C libraries don't really exist" libstrophe, libcouplet, and purple.
-
jonasā
to name three off the top of my head, I assume there exists at least one more (just because)
-
doge
> "C libraries don't really exist" libstrophe, libcouplet, and purple. Last time I checked pidgin barely even supported group chats as in no bookmarks, didn't support omemo, didn't support mam.. As for the other two, I don't know much about them, but how well documented are they exactly? Will it be easier to dive into their code than to just go from scratch? ↺
-
doge
> to name three off the top of my head, I assume there exists at least one more (just because) It's not quantity that matters. ↺
-
jonasā
one of these (not libpurple) is used by profanity (a TUI client)
-
jonasā
I used one of these (not sure which, it's been almost a decade ago) and it was easy enough.
-
kalkin
> "C libraries don't really exist" libstrophe, libcouplet, and purple. Nice
-
kalkin
Common Lisp Xmpp libraries don't really exists. jonasā: your entry š
-
kalkin
I actually searched for them and found onl clxmpp which is old...
-
jonasā
kalkin, nice try, but I don't know the CL ecosystem almost at all.
-
Kev
cl-ngxmpp? :)
ā¤ļø 1 -
kalkin
Nice but sadly still missing a lot
-
rburns
is it good practice that a bot should ping the server periodically to maintain it's connection? or should that not be necessary?
-
jonasā
rburns, it is good practice to use XEP-0198 <r/> elements to ascertain that a connection is still live. you shouldn't ping unless the stream has been silent for some time.
-
rburns
ok, thanks. looking at the xep. is it right that the <r/> elements should be sent after sending a message.
-
jonasā
it is one way to do it
-
jonasā
you can send them whenever though
-
jonasā
and when I said "ping", I meant "send a <r/> for only liveness-checking-purposes".
-
jonasā
(there are other reasons to send a "<r/>". there are also other ways to "ping", but <r/> is by far the most efficient)
-
jonasā
rburns, but, your XMPP library should handle all that, does it not?
-
rburns
yeah, looking at the debug log from the xmpp library. (xmpp-client on nodejs) it initially sends the <enable /> element and I can see it respond to <r /> elements with <a/>. nonetheless. the connection is lost consistently after a short time. when the connection is lost. there's no close/disconnect events fired. it silently disappears. I can see the disconnect in the server logs
-
jonasā
okay, that sounds like a library bug
-
jonasā
if it does '198, it should either handle reconnection transparently, or it should at least let you know about stuff when it breaks.
-
rburns
ok, thanks for your help. will dig around some more.
-
lovetox
Are there any known conditions why a server would not put messages to self into MAM ?
-
Stefan
jonasā: profanity is not using libpurple.
-
Stefan
Sorry, did not read correct.
-
Stefan
Anyway, Chatty is using libpurple and profanity (and xmppc) are using libstrophe. I think libstrophe is fine. Documentation is ok and it's working.
-
jonasā
Stefan, glad to see I know my stuff :)
-
jonasā
(in the sense that i have a rough overview of what happens)
-
singpolyma
>> "C libraries don't really exist" libstrophe, libcouplet, and purple. > Last time I checked pidgin barely even supported group chats as in no bookmarks, didn't support omemo, didn't support mam.. As for the other two, I don't know much about them, but how well documented are they exactly? Will it be easier to dive into their code than to just go from scratch? Your library doesn't need to support high level things like bookmarks and mam to be useful. Just doing DNS, connection, TLS, sasl stuff is very time saving by itself. If you want a high level library I agree there are few of those (why I'm making one) but for low level there are lots of fine choices ↺
-
jonasā
tls, sasl, and I'd add stream management and reconnection logic to it
-
singpolyma
> yeah, looking at the debug log from the xmpp library. (xmpp-client on nodejs) it initially sends the <enable /> element and I can see it respond to <r /> elements with <a/>. nonetheless. the connection is lost consistently after a short time. when the connection is lost. there's no close/disconnect events fired. it silently disappears. I can see the disconnect in the server logs rburns: is this for xmpp.js ? We are adding proper handling of this in the main dev branch just in the past weeks. Should be in next release ↺
-
jonasā
you really don't want to bolt that onto a library as an afterthought, it doesn't end well.
-
singpolyma
I mostly agree yes
-
doge
> Just doing DNS, connection, TLS, sasl stuff is very time saving by itself I don't think implementing those yourself is more hassle than having to deal with ffi
-
singpolyma
What stack are you in?
-
rburns
singpolyma: yes, after more digging I found your PR recently merged which looked related. Look forward to the next release
-
doge
tcl
-
singpolyma
I thought calling C from TCL was something it is good at?
-
doge
Writing a c extension is indeed rather easy. But with xmpp you deal with a dom. You deal with a recursive structure, you have to tie that in with whatever high language object system you're using and its garbage collection. With Tcl it could be even easier to do that than perhaps with python, but wouldn't be tcl-esque at all or convenient to use. Connecting it well enough for it to be pleasant to use would be at least as much work as just re-implementing the DOM at tcl level imho
-
doge
And then you also have to integrate it with the event loop. Which is easier but still another obstacle. So these obstacles combined I think make it easier to just reimplement it in tickle.
-
moparisthebest
Huh? What's recursive ?
-
doge
dom nodes
-
moparisthebest
I mean a dom is a tree, but so are like your favorite json nodes š¤·āāļø
-
doge
Yes, but for json you already have a language specific library which also isn't insane with namespaces prefixes etc etc. Most XML libraries seem so huge and bloated and have so many features that honestly it may be for the better to just not use them in a networked application....
-
singpolyma
Yeah you only care about prefixes at parse time. You don't need them in the data model
-
doge
If there was a library that handled authentication, stream management and reconnecting, and then gave you like just an XML stream that you would reparse, or better yet provide a length prefix stream of normal XML documents you could pass into an existing xml library, perhaps I would use that.
-
singpolyma
Sounds like xmpp-proxy
-
singpolyma
Just using a sax parser is easier imo but it does exist
-
doge
> Yeah you only care about prefixes at parse time. You don't need them in the data model I meant the other kind of prefixes. It turned out that to use namespaced nodes from an XML library you actually need to register your own local prefixes, and to insert those at every corner.. ↺
-
doge
Not as is a big problem, but it just highlights the level of insanity we're dealing with and the tendency to create problems out of nowhere.
-
singpolyma
Sure some libraries do thinks that way because people prefer that syntax to clark notation or whatever✎ -
singpolyma
Sure some libraries do things that way because people prefer that syntax to clark notation or whatever ✏
-
doge
I honestly prefer to just ignore the namespaces for most of the time. Or "same namespace as the parent until otherwise specified"
-
singpolyma
I've seen some libraries to that. I find it a terrifying way to operate, but if you're careful it can work
-
moparisthebest
> Yes, but for json you already have a language specific library which also isn't insane with namespaces prefixes etc etc. Most XML libraries seem so huge and bloated and have so many features that honestly it may be for the better to just not use them in a networked application.... doge: https://www.moparisthebest.com/images/jsonschema.jpg ↺
-
Zash
moparisthebest, don't forget JSON-LD and its namespace prefixes!
-
moparisthebest
> moparisthebest, don't forget JSON-LD and its namespace prefixes! Indeed... In the end to implement something sufficiently extensible for XMPP you'd need all this "complexity" in json too, which ends up being even worse imho ↺
-
jonasā
and its namespace prefixes WHICH ARE USED IN VALUES
-
jonasā
they're making the same mistakes as SOAP again.
-
jonasā
making it absolutely impossible to decouple parsing and application
-
doge
>> Yes, but for json you already have a language specific library which also isn't insane with namespaces prefixes etc etc. Most XML libraries seem so huge and bloated and have so many features that honestly it may be for the better to just not use them in a networked application.... > doge: https://www.moparisthebest.com/images/jsonschema.jpg "Look, somebody made this stupid thing on top of this sane thing. It means our stupid thing is not as stupid after all! " ↺
-
moparisthebest
doge: s/stupid/required/ but sure
-
doge
> they're making the same mistakes as SOAP again. SOAP's mistake was existing. ↺
-
jonasā
indeed.
-
singpolyma
I don't mind using soap with a good library but I'd never want to implement it
-
singpolyma
SOAP is back now anyway they just call it graphql now. What's old is always and ever new again
-
singpolyma
This has all happened before and it will all happen again
-
jonasā
what is dead may never die
-
Kev
This is the way.
-
Zash
So say we all
-
jonasā
It is known.
-
moparisthebest
It does seem like many software developers are opposed to using soap...