XMPP Council - 2013-02-20

when should one use a stream-feature and what are the alternatives?

Features related to the stream itself?

the delayed delivery case Lance mentioned yesterday...i.e. having 200 users on your roster, and receiving 200 offline presences with delayed delivery, so you know when they went offline could be quite a bit of traffic

heh

That doesn't happen until you send your initial presence.

right

So it doesn't seem that it needs a stream feature to stop it.

so it could be in caps?

s/caps/disco?

Could be in caps, even, yes.

hmm....

Although you'd get the presence if they were online, so ... :)

hmm...

or could stanza filtering filter that out

didn't we have a XEP for that?

SIFT

could it filter delayed delivery out of presence stanzas?

SIFT didn't get that specific, but it is extensible

meaning, you can add new conditions

Tobias: Some servers already send unavailable presence

So that traffic already exists.

Right.

so nothing we can do about it?

Cry?

from a server to the client, SIFT (possibly with a special extension) should be able to do what you want

but what m&m says

the trick is to get servers to implement it (-:

Someone argued that it made sense to drop such unavailable presence

I bet MattJ will have it done in prosody by lunch tomorrow if you asked nicely

m&m: Wait, for what?

(-:

is MattJ up and running again?

for the "SIFT-unavailable-with-delay"

Ah

I wrote a plugin yesterday to filter out unavailable in response to probes if it had no childs

And ding ding ding.

Zash: what is the exact issue with the offline presences

1) Roll call

I'm here!

Zash: after the meeting

presente

I'm here

same here

And I'm assuming we have continued apologies from MattJ until we hear otherwise.

He said he was going to be here

anyone know how is these days?

Zash: Ah, OK. Maybe he will.

ralphm: I've not spoken to him at all.

I'd worry, but he got word through to Peter that he'd be offline, so I just hope he gets fixed soon.

Aaaaanyway.

2) Peter'd like to issue an LC on 152 (Reachability Addresses)

MattJ: Welcome back.

normally I'd be +1 on LC

Hey :)

m&m: There's a "but"

Never gladder to attend a meeting before

IETF is kicking my butt

MattJ: WB!

and the meeting hasn't even happened yet!

I and stpeter might be the only ones in this predicament

IETF doesn't want to be reachable/

?

ralphm: opposite problem, it's too reachable

m&m:So you'd like to delay the LC on 152 until after the IETF meeting?

anyway, I've no objections to LC, but I would like to extend the deadline by a couple of weeks

sure thing

Kev: or that

A one-month LC, then?

Long Call

I am up for that

LCLC

Or LC^2.

either works for me, although delaying would probably work out better for my inbox (-:

L²C

heh

Zash wins

Congratulations.

\o/

MattJ: Opinion?

I have some catching up to do

later or longer last call are either fine with me

Longer LCs are fine by me in general

as long as we remember the deadline

oops :-)

We've had to extend LCs multiple times in the past

which reminds me, I have some BOSH patches to submit

I trust Peter to remember the deadline :D

MattJ!!!

☎

that's what we have a calendar for, not?

stpeter!!

ralphm: :-)

Tobias!!!

stpeter: http://logs.xmpp.org/council/130220/: Chatroom logs for council@muc.xmpp.org (Wednesday, February 20, 2013)

MattJ: Are you OK with an LC on this one?

+1

Excellent.

3) Date of next meeting?

SBTSBC?

wfm

yes, as long as we keep extending the deadlines (-:

👍

ralphm: I'm going to assume that's the special unicode glyph pair for "works for me" :)

y'know, XEP-0152 has been sitting around for a long time, so there's no big hurry about a Last Call, although it would be nice to advance it before draft-ivov-xmpp-cusax becomes an RFC

THUMBS UP SIGN (U+1F44D)

oh, unicode

Was it? o_O

Oh, yes, that's very different.

In the font Swift uses it looked like a left sleeve and a right sleeve.

Anyway.

4) Any other business

I think Peter'd like to ask us to go look at WG stuff.

(Precis)

right :-)

Yes. When can we advance −308 to draft?

Yeah, Gunnar was chasing me about that the other day (in a mail that went straight to spam, bad dog Gmail).

heh

here....everybody roughly okay with the direction my XEP-0012 deprecation work is heading in? new experimental XEP for idle, new informational XEP for presence+delayed delivery for last presence change time.

Kev: it is not a glyph pair?

Tobias: tentatively

I'll try desperately to address LC feedback as soon as I've got a spare cycle.

/nod

I

Tobias: yes, that seems fine, but I need to look at it in detail still (next week)

gah

what stpeter says

I've got internal teams asking about the status

of 308

:-)

Tobias: It needs something for uptime, too. That one's widely deployed, and is even used. Although I'd rather something better :)

Kev, yeah...directed presence probes to hosts with delayed delivery response :)

available presence + delayed?

i'll mention it in the informational XEP

m&m: Words fail to describe how many balls I'm trying to juggle at the moment, but I will try to add that one as well. And watch it all come crashing down.

Kev: I feel your pain

Any other any other business?

Kev: need co-authors?

stpeter: I think I just need a few hours.

stpeter: Or a time machine.

Kev: which you don't have!

I'll get to it. Maybe I'll have some minutes watching films with Cath at the weekend or something.

:/

ok

for Kev's sake, I think we're done

/nod

yup

i have nothing else, at least.

Right.

Thanks all.

yay :-)

so

Deoxyribonucleic acid? :)

ralphm: Offline presence in response to probes are redundant, unless they carry some kind of extended data, like delayed delivery.

Kev: very close. Domain Name Associations (or Assertions)

draft-saintandre-xmpp-dna

Yes, I knew.

Really.

(-:

Zash: arguably they also tell you that you still have a presence subscription and that the remote host is reachable.

ralphm: Right. Otoh it's traffic you may or may not want. The person who wanted this filter works on a mobile client

so, for a mobile client, I can think of two considerations

1) battery, 2) bandwidth

For case 1) I'd argue that it is very likely that your antenna is in fully up mode when you receive it.

For case 2) I'm pretty sure it compresses quite well

And for (1) you want something like google:queue.

If the remote host isn't reachable then you will get an error. If you no longer have a subscription you would get a unsubscibe(d) reply, no?

(playing devil's advocate)

Zash: *maybe*

Zash: Yes.

Well, you needn't get an error.

You will get unsubscribed for no longer having a sub.

Not for 3921 implementations

That's true.

Oh

So, as I said, arguably they do carry information.

Which doesn't invalidate your case

but we should do google-queue/SIFT

google-queue was the thing where it queued up "less important" stanzas, like presence until something more important like a chat message or iq came in?

ralphm a 3) issue is the effect on startup time for the app, which IIRC was the issue raised by the mobile developer

Zash: Yes.

Lance: It shouldn't affect startup time for the app. It's all async.

Not other than is covered by (2) anyway.

If s2s setup for sending probes takes a long time, it might even lower the radio state before it gets replies

Zash: hence google-queue

Yes

you wouldn't get those until you'd send something or receive a normal message

I don't really believe in 3) for properly designed applications

Saying that.

This really does hurt clients.

That's why the stpeter roster test is nasty.

Kev: so is that because of badly written clients or the shear amount of data that has to be sifted through?

(genuinly curious)

ralphm: I spent a very long time profiling and tuning Swift for this - it's the roster updates.

The roster's sorted and every time you get a new presence that may potentially update the roster order.

oh, hmm

how did you resolve that? delayed updating the roster order?

So you've got the pain of doing a lookup across the entire roster for the right JID to update, in all the right groups, then the pain of calculating the new presence to render (actually insignificant usually) and then the pain of reordering the widget.

No, just fiddling with data structures.

Maps instead of vectors for finding the JID in the roster makes a hell of a difference.

I got the stpeter test down below 10 seconds for Swift - I think the next best client at the time was Psi with about a minute.

Modern hardware will obviously improve this dramatically.

so but in this case, we are talking about unavailable presence

that wouldn't affect anything in the roster

It does, somewhat.

It doesn't cause a redraw (unless you have a status as well).

But it does need you to do all of the data structure walking to find that it doesn't cause an update.

kind of. because its FIFO, if you have a large roster and lots of unavailable, the app appears to be much laggier than it really is on startup

But did you really do a redraw for every presence? One of the first things I'd try is delaying it by, maybe, a second?

I just update the model, and let Qt manage when the redraws happen.

Kev: sure, but I assume you can batch that, too

It wasn't paint calls themselves that were appearing in the profiling, at the point that I stopped work on it.

I'm sure one could squeeze yet more out of it - being around an order of magnitude faster than anyone else was enough for me at the time :)

hah

and here's the one behind the issue. hey Ge0rG

hi! I'm Georg from yaxim.org

but, still, if you also control the server side (the one the client is connecting too), don't we have something like quickstart for non-BOSH connections?

hah, that's funny. I was just thinking about how yaxim also doesn't seem to like large rosters :-)

ralphm: This is the guy

there is roster versioning, which reduces the overhead; but it only eliminates half of the traffic when connecting

Zash: awesome. We are full circle.

delaying UI/storage updates on incoming presences is only a mitigation measure... the incoming traffic still remains

Ge0rG: I was just trying out yaxim the other day. It does indeed seem to have issues with my roster. xabber seems to have less problems with it (no offence, also, it has other problems)

ralphm: What's your largest roster group, and how many do you have?

ralphm: yaxim is storing roster+presences in sqlite, with ca. 100ms of writing time for every. single. item.

That sounds like a good thing to not do, then :)

27 roster groups totalling 343 contacts. The largest one is 45

ralphm: That sounds sufficiently small that it shouldn't cause too many issues.

Kev: right

yes, I am currently preparing a huge rework of the backend; but it will take time

Same order of magnitude as my roster (although larger).

Kev: so I assume Ge0rG has some work cut out for him

and I do not really see a reason in delivering bare-jid unavailable presences (as unavialable is probably the default state anyway)

nevertheless, something like XEP-0305 might also be useful for our TCP binding

Ge0rG: http://logs.xmpp.org/council/130220/

ralphm: http://logs.xmpp.org/council/130220/: Chatroom logs for council@muc.xmpp.org (Wednesday, February 20, 2013)

Ge0rG: we did some talking about it before you joined

ralphm: I think 305 doesn't help at all here, actually.

198 would help...

provided you had a session before

Right.

Kev: I'm sorry, I meant 198 of course

Right, 198's fine as long as you have an existing session.

But for initial login you don't.

but not for initial connection, or if you have been offline for a longer time (5 or 30mins in prosody?)

Ge0rG: Configurable, defaults to something like 5min

but you can throw away the unavailables right after having parsed them

It would be nice to have some way to suspend a session so it wouldn't timeout

ralphm: No, you can't.

Kev: if he thinks they are useless you can

ralphm: Right, but you can't know they're useless until after you've checked against the roster.

ralphm: for that you need to check the previous state of the JID

Although if sqlite is the bottleneck here, you can certainly easily bin them before that.

Kev: that was my point

also, not everybody can use tls/stream compression, and data traffic is not free everywhere in the world

well, ehm

Ah, that's interesting.

Why would you not be able to use stream compression?

so what is the additional information carried by presence-unavailable from bare JIDs in 3921 implementations?

Ge0rG: Nothing, but 3921's obsoleted now.

it is actually likely they wouldn't send them

Or, well, sometimes nothing, depending whether there was a <status/> there

ralphm mentioned 3921 as part of the pro-transmitting argument

Ge0rG: I did

Ge0rG: the point is that you can't know

which I am trying to understand now

Kev: stream compression is buggy in certain client-server combinations (I blame openfire, but not enough datapoints yet)

Ah.

Ge0rG: some implementations will not send them, some will send unsubscribe (and some not) and some will send an error (or not)

Kev: https://github.com/pfleidi/yaxim/issues/85

Yes, indeed, I remember something about OpenFire's compression stuff being broken.

do blame openfire

:-)

but blaming openfire will not fix the real-world problem for my users

Ge0rG: No, indeed.

and they will blame yaxim instead

and if they get around fixing stuff, also let them fix the thing where you have to send presence to receive iq's and messages to a full resource

ralphm: if old implementations are not guaranteed to send anything, why would we demand new implementations to send a presence-unavailable?

There is no demand

We don't, do we?

We don't

It's a SHOULD

we allow it

Zash: Ah, is it?

That's domanding it, then.

thats why I am asking

Unfortunately I have to go for dinner

Kev: It is?

so my suggestion would be to change it into a SHOULD NOT :P

this will not help your case

at least not very soon

It's not true, though, that an unavailable contains no extra information. It does - it tells you that you know the state of the remote user.

ok, actually it needs some more context explanation

Until you receive your first presence from them you've no idea if they're online or offline.

Kev: but you should assume the remote user is unavailable by default...

or there should be yet another presence state "unknown"

The behaviour where the server tries to keep a consistent view of the availability of contacts at all times would help this

was it Dave who wrote about that?

[offtopic] is there an (http) API to look up the XEP name given the number?

Ge0rG: Sorta

There's an XML file

!xep 198

Kev: XEP-0198(sm): http://xmpp.org/extensions/xep-0198.html Stream Management - Standards Track/Draft - Updated: 2011-06-29

Ge0rG: Or, if you know the number, just stick it into that url

Zash: I want name and number, for a jekyll plugin

you know, if you are doing something, do it right the first time :)

Ge0rG: http://xmpp.org/extensions/xeps.xml

cool, thanks

http://web.archive.org/web/20120208104426/http://blog.dave.cridland.net/?p=144

Zash: http://web.archive.org/web/20120208104426/http://blog.dave.cridland.net/?p=144: Advice to Santa on fast presence delivery &#8211; text/plain

FWIW, when talking about 'doing it right first time', having played with both jekyll and nanoc, I prefer nanoc.

I'm not going to jump trains again ;)

Fair enough.

I did, FWIW.

(Wordpress->jekyll->nanoc)

I's been only two weeks since I set up http://yaxim.org/

Ge0rG: http://yaxim.org/: yaxim - yaxim

and it is good enough

OK.

Ah, maybe octopress makes things less bad.

bye, again, jabber.org

:-/

That one was my fault.

I did a stupid.

Kev: don't do stupids

Excellent advice.

I cannot fault it.

I am also not sure it is all right now

I cannot join a MUC at conference.ik.nu from jabber.org