jdev - 2021-07-02

What are the typical transfer speeds achieved during a jingle file transfer?

512bit, whatever the link between the two clients can achieve

there is no "typical" number there

depends on the negotiated transfer method

Okay, trying again to phrase a problem that's been bugging me, maybe someone here will have insights. I'm building a MUC library and it's showing me the weaknesses in my core XMPP library. One is that I have no way to track stanza errors (other than IQs). When I send a join presence for the MUC for instance, I always expect a response so I block until receivnig it, something like muc.Join(timeout, jid) roomOrError. This works for the self-presence because the MUC handler is tracking all presence payloads that have the <x xmlns="muc#user"/> or whatever in them so it can associate the response and hand it back. It can't hand back an error though because that is not a MUC specific payload and it doesn't know about it.

So instead I thought about having a general xmpp.SendPresence(timeout, presence) error that will block until it gets a presence error. I can call that from within muc.Join to send the presence, then if the success response happens I can cancel the timeout (don't worry about that, just realize that timeouts can be canceled early) and return it, otherwise we wait for the error until the timeout is hit and then return an error if no error or success was received.

However, this feels wrong. Using xmpp.SendPresence or whatever the *expected* behavior will be that it times out and returns that it timed out 90% of the time (since we hopefully don't get presence errors often). It works for the MUC case, but for genreally using it it feels weird.

Has anyone else run into something like this?

I think the summary of what you're trying to say is that in some cases in XMPP there is no explicit success response

Which is indeed the case

I have generally found that I don’t need stateful errors for very much outside iqs.

Yet any stanza can trigger an error, that you would want to handle

Heh, yes, I suppose that's all I was saying with all of that. It makes it hard to track errors because the timeout is indeterminant (sp?).

I

Normally it's not a problem for that reason, but MUCs it can be since I want to block until the join was successful or an error is received (or the timeout is hit). It makes more sense to have the timeout only if there is an error and a success case

You can always* use ids for tracking errors to particular stanzas if you care.

I think the answer is partly as Kev says - the stanzas without an explicit success /usually/ don't need one for practical use-cases

Kev: yah, that's what we're talking about

When you say block, I assume you mean that a MUC join is happening in its own coroutine that you’re blocking, not that you’re blocking the calling site?

I've thought about this too. Not sure I'd want anything blocking, but some way to race a timer came to mind.

Yes, sorry blocking a coroutine.

(Since MUC joins you generally want to be async and to continue processing other data while you wait for them)

Blocking is the normal and expected behavior for things like this; the user can always await it if they want it in the background (more or less)

Right.

Blocking the join, not blocking the caller.

The issue is more or less that if I've registered a "muc" handler the success case gets picked up (it's a MUC specific payload), the error case doesn't (unless I want the MUC handler to handle *all* error presences)

I’m missing why it’s hard to timer this (probably my fault).

The timer isn't the issue, that part is fine

Sorry, I've been struggling to explain this; not your fault.

It's even more fun. The true completion signal is the subject message 🙂

For a MUC join at least.

What we do in Swift is we start a timer on join. If we succeed the timer’s cancelled. If it doesn’t succeed and the timer fires the user is warned that the join probably won’t complete - but you could easily hard fail instead.

hmm, good point, I was taking the completion message to be the self-presence.

A way to have composable completion events might be nice.

Zash: Ah. It’s the subject message…or the first non-history message because some servers don’t send a subject. Although that was an old ejabberd bug, and it’s probably been fixed for a decade.

Kev: that's what we do as well. I'd also like it to be able to return the presence error (if any) though.

Then you could take a message and a timer, or a message and a thing that watches for a message receipt, or a presence and a thing that watches for the subject.

Sam: But that’s easy enough, isn’t it? If you get a presence error reply to your join stanza, you have the error.

Kev: right, but this is a MUC handler that receivese MUC paylods. The presence error is a normal generic one, not a MUC specific thing. So I'd have to listen for *all* presences in the MMUC handler to do that. This isn't actually how my library works right now, and it seems like there is a better way.

Sam, make a thing which allows you to listen to presences with a specific ID

I would personally just listen to all presences, and check the id. Or have a ‘tell me if this id comes in’ handler.

Right, so like I said in the orignial post if I do that it works okay for MUCs, but then if the user uses it the API feels weird:

then it’s tied semantically to the MUC by the ID and it’s less ugly

But I would very very much not make sending presence into a thing with a reply handler, because 99% of the time that’s just pointless.

xmpp.SendPresence(timeout, presence) error or whatever. The expected case is that the timeout is triggered momst of the time. Feels weird.

Yes. That is what I would not do.

Right, what Kev said

Sam, how about xmpp.WaitPresenceError(timeout, id)?

Works great for MUC, not great for 90% of other cases :)

jonas’: yah, that's what I was saying (more or less)

and keep xmpp.SendPresence fire-and-forget naive

But why not make xmpp.SendPresence(presence, optionalHandler, optionalTimer)?

Off for lunch :)

Kev, it may have been Prosody (too) which didn't used to send a subject if the subject was empty/unset. Fixed now anyways.

That's what I've done right now (ish). There's Send for sending any general stanza in a fire-and-forget way, SendPresence for sending a presence specifically and waiting for an error (or timnig out).

And a bunch of other sort of related things that don't matter.

Anyways, maybe this is the only way with how I have this structured (or give the MUC thing access to all presences and what not, but that feels wrong) and the answer is "Okay, SendPresence is weird, don't use it most of the time"

Thanks for the brainstorming.

fwiw, it seems that aioxmpp does not track presence errors during muc join based on the ID but on the from

I thought about adding a way to track based on from so that any errors from a specific thing get forwarded to the MUC handler but that requires being able to modify the handler at runtime (eg. when we join a room, add "chat.myserver.com" to the valid from) which bothers me a bit, and also means that every separate plugin has to do its own id (or from) tracking internally to match responses with requests instead of the main XMPP handler thing just doing it for everything

But maybe that would be better than adding weird SendPresence functions that are expected to fail

(or timeout, rather)

Hmm, now the subject thing is bothering me a bit. Does it matter if they can do things before the message history is sent, or should I block until the subject is set? I can't tell that it would matter, you might just send a message before getting context back, which feels weird but isn't the end of the world. Also I'm kind of assuming a MUC-MAM future anyways, so I dunno if I care about history/subject as much.

> fwiw, it seems that aioxmpp does not track presence errors during muc join based on the ID but on the from Pretty sure that’s what we did in Swiften, without checking.

Sam, aioxmpp doesn’t prevent you from doing things before the subject was received. It only uses it as a marker for when to start trusting nickname<->jid relationships because it has up-to-date presence or somesuch (I’d have to look it up).

Makes sense; for now I'll probably consider join successful on self-presence then. We'll see if I use subject for anything later (so far I haven't even looked at that and am not tracking the subject at all)

It would be weird for the join to fail between self-presence and subject.

As far as I can tell from the spec it can't, you're actually done once you get self-presence. Subject is just something that might or might not show up later.

Zash: What if the server has to hit storage to fetch the subject, but not the members? That *might* mean a noticable delay between them.

(it's a SHOULD, so it probably will, but I don't see anything that relies on it)

But ultimately a network can fail at any point.

True. The whole thing could crash at any moment!

Sam: What relies on it is it’s what implementations use for knowing the join’s complete :)

Sam, hm? I thought sending subject was mandatory

SHOULD doesn’t mean OPTIONAL, it means “Things might break if you don’t do this, so be sure you know what you’re doing” ;)

Kev: why rely on that instead of getting self-presence?

Sam: Because presence comes before history.

I mean, if it's what everything does I can do that too, the spec just makes it sound like self-presence is the thing that means you're joined. What does it matter if I've gotten history or not yet?

And you almost always want to know what’s live messages and what’s join context.

So you need the subject to know you’re done with the history and everything after will be live changes.

Oh yah, I guess I might want to draw a line after history or something to distinguish it for the user

Also, subject is a MUST.

Not SHOULD :) ✏

oops, so it is. Weird, I swear I just read that and it said SHOULD.

SHALL == MUST?

Yes

Words.

Still, I'm joined after self-presence. When I get subject, I can draw the line and start live messages. Does it matter that I was already joined?

Anything I send I'll still get the reflection back after the history (I assume).

If you wanna have a "ready" signal, separate from "success", I suppose.

May matter for UI stuff.

What's the difference between "ready" and "success", I guess?

I just don't see how it would matter in the UI or anything

If you wanna show a spinner or something while it's throwing history at you?

In that case it seems like I'd just do the Join, show the spinner, then when I get the history back I'd clear the spinner. I don't have to block the join call on it.

I think some clients don't let you start writing until then

The writing thing makes sense, but this is a library so the clients could always still do that.

Returning on self-presence seems more flexible in that way; clients can still do either. If I return on subject they are stuck with that.

But also I have a nagging suspicion that if everyone else does subject there's probably a good reason that actually will break later that I'm not thinking of.

> // SendPresence is like Send except that it returns an error if the first token > // read from the stream is not a Presence start token and blocks until an error > // response is received or the context times out. > // Presences are generally fire-and-forget meaning that the success behavior of > // SendPresence is to time out and that methods such as Send should normally be > // used instead. > // It is thus mainly for use by extensions that define an extension-namespaced > // success response to a presence being sent and need a mechanism to track > // general error responses without handling every single presence sent through > // the session to see if it has a matching ID. > // > // SendPresence is safe for concurrent use by multiple goroutines. Good enough (I hope).

Thanks again for all the help.

Sam, what's that about returning an error if not reading a presence from the stream?

Wrong type of stream, it's confusing out of context but it copies XML tokens so you'd do SendPresence(timeout, Presence{Type: whatever}.Wrap(somepayload)) which returns an XML token stream that gets copied to the XMPP stream.

See SendIQ (https://pkg.go.dev/mellium.im/xmpp#Session.SendIQ) for an already existing example.

I should probably not re-use "stream" in the docs, that's confusing.

Sounds a bit overloaded, yes.

"first token read from the reader" or something will fix that. Done.

Or "from the input".

There are now 16 basic sending methods that match (Send|Encode)(Message|Presence|IQ)?(Element)?

I don't love that, but it does work quite well in my experience so far even if it's a lot of options.

Does anyone still use talkonaut?

What is a talkonaut?

Looks like it's similar to jmp.chat? The website doesn't work for me, so I dunno if it still exists or not.

Ah, so a telephone xmpp transport. Never used one of those.

> Talkonaut by GTalk2VoIP, Inc.

Hehe

Wow, haven't heard of Talkonaut for a while

GTalk2VoIP sounds like it's long dead also.

When was gtalk switched off?

Never, it's still going I think :)

Still responding on 5222 anyway

Oh, I thought it's dead for a long time already.

s2s is dead, c2s isn't.

Yeah, they announced it was. Then it continued to work for years. Then they eventually turned off s2s (which nobody used anyway because it didn't do TLS). Now they're shutting down Hangouts, which is likely tied up in the same infrastructure, so it possibly might actually disappear soon.

Is there even a client nowadays? Can't really follow googles pace of inventing and killing chat systems. 😁

You can use any XMPP client, but it only really works with other XMPP clients. If people are using the Google Hangouts client they appear online but don't receive messages.

All that based on old observations, I haven't poked around in the past year or so

Sounds like a long forgotten service running on a long forgotten machine in the basement. 😂

Well, that Hangouts users appear online suggests it's all the same infrastructure, and the Hangouts conferences in the browser used XMPP for signalling even after they announced the end of GTalk, so it's very likely still all production.

But now Hangouts is supposedly dead (EOL June), all that may change

MattJ, s2s also didn’t let outsiders communicate with Hangouts users, fyi.

or they may just keep it limping along for those few enterprise customers who insist they can't live without it

With an extremely annoying failure model, where our messages would simply get redirected to /dev/null.

Yeah

Just pretend it never existed and keep on working on xmpp world domination. 😃

Also Hangouts video calls from the Firefox plugin were printing MUC and Jingle debug stuff on stderr.

Is there a way I can upload files? ✏

everytime I try to do it, it just errors out with "File transfer failed"

https://imgur.com/a/sbAyQ0m

What server do you use?

sqli.io