XMPP Council - 2012-04-11

Meeting with nothing on the agenda in ~55 minutes.

w00t

It took me a long time to work out "Today being Monday, tomorrow being Wednesday"

MattJ: Submitted MAM yet?

Not yet

MattJ: +1 on stream:error everywhere

Thanks, good to know I'm not just missing the point of that thread

You MUST answer <iq> requests, but does that mean we shouldn't send a stream:error if there are any outstanding?

Oh, good question. Yes, I guess it does :)

In Wokkel I just set up a general handler for that not-stanza-but-still-direct-child-of-the-stream-element-that-does-not-have-a-proper-name

or you treat the stream:error as a "catastrophic failure", like a severed network … d-:

linuxwolf: I do. In Wokkel, you get a Deferred back when sending out an IQ. If the current stream disconnects, the deferred is fired with an exception to that effect.

/nod … it's what I've done just about everywhere, too

mostly because you can't know what really happens

if a server's at the point of emitting a stream:error, you have bigger problems than unanswered iq's

I think the order here is actually mandated.

Because the server must be processing the stanzas in order - so if you send <iq/><message><illegal/></message> ther server isn't allowed to process the message (and find there's an error) until it's finished with the iq.

Stream errors for server going down or whatever don't fit into this, I realise.

right

also http://xmpp.org/rfcs/rfc6121.html#roster-syntax-actions-push

I rarely see the <illegal/>, but definitely see the crash (-:

… which, in spite of the MUST, still allows for not sending a response

graceful-crash?

Kev, define "finished" (with the iq)

(-:

Kev, does handling in order require responding in order?

Of course.

Of course roster pushes in any new design would be <message/> stanzas

It's not handled until the server has replied.

Ok, so...

ralphm: everything is pubsub

linuxwolf: hah, earlier specs for pubsub used IQs for notifications

Kev, if you send a vcard request to an occupant in a MUC room, the server isn't allowed to process any further stanzas from you until that person's server responds with their vcard?

linuxwolf: this is one reason I think that's a bad idea

in any new design, we can just do <publish/>, <subscribe/>, and <notify/> as TLS (top-level-stanzas)

"just"

linuxwolf: I guess we'd just drop '<presence/>' and keep '<message/>' and '<iq/>'.

yeah, pretty much (-:

MattJ: It's not the server that's processing the iq.

So we can have a stream:error before the iq is responded to?

Yes.

Good

Of course

Just not before an iq is responded to by the server.

A roster get or whatever.

s/an iq is/an iq addressed to the server is/ ?

Right.

I think you're talking about something aside from the main issue

Naturally.

My original question was whether you could close a stream with unhandled iqs. We've establised a definite 'yes' for iqs sent to non-server JIDs

and I see no reason a server must reply to an iq, if it's invalid for example, instead of throwing a stream error

Kev: are you also saying that responses to iqs to the same entity should come back in order?

gosh, its seems that I've missed a fun discussion

and I'm thirdly not convinced (but admittedly on more shaky ground) that iqs must be responded to in order

because I don't equate "handling" with "responding"

MattJ: I agree

MattJ: agreed, that seems QUITE problematic, to me.

stpeter, pre-council meeting :P

especially if dealing with large scale

I think in-order processing is only about accepting stanzas in order and routing them in the same order as much as possible. Nothing beyond that.

stpeter: claim atmospheric issues

or claim it's Tuesday

damn proprietary Outlook invites

hah

fumso

(f*** u microsoft office!)

Righty.

Shall we have our not-a-meeting, then?

1) Roll call.

I'm here.

presente

Gift

hi

Tobias: ?

kev

ah..council meeting

youp..i'm here

Did anyone have anything for the agenda?

ok my call is done

I'll add GSoC, then

2) GSoC

The applications are in, thanks to the people who answered my call for mentors and have commented on proposals.

I submitted the slot requests to Google yesterday, based on the applications people volunteered to mentor and the comments on the proposals.

There's still time for people to comment on proposals if they want, but I'd rather they didn't juggle anything about too much, as we're pretty much done at this point (apart from the Jitsi guys deciding which proposals they want to accept - but they've chosen a number).

ok

That's about it.

nice

We've asked for more slots than is usual for us, so I'm expecting Carol not to be able to hand them all over, but hey ho.

(-:

Kev: would you be so kind as to point me to the message on the GSoC list that says exactly where we complete certain tasks? or is it all somewhere at google.com and I just need to log in someplace?

When that happens I'll look at the comments and scores people have left on the proposals and try to pick the ones we select somewhat fairly.

google-melange.com

ah ok

thanks

I think that's everything GSoCcish.

3) Date of next meeting.

SBTSBC

?

wfm

hopefully MAM will be a topic of discussion

/zing

We can but hope. Nearly 50% of the XSF's GSoC applications were to work on MAM support.

heh

Right, SBTSBC it is.

4) AOB?

nope

nada

None

nope

Sounds good then.

ok thanks guys, I'm slowly getting back into mostly-xmpp mode, so I'll start working on things of interest here again soon

Thanks all.

Kev: I'm still interested in your responses to the pre-meeting discussion

ralphm: Interested in what way?

Kev: well, for example, the question I asked

That's not entirely unambigous at this point :p

stpeter: Enjoy.

ralphm: Kev: are you also saying that responses to iqs to the same entity should come back in order?

(and the stuff following that)

Yes.

The entity is obliged to process them in order, their server is obliged to send them to your server in order, and your server is obliged to send them to you in order.

Kev: I'll rephrase

Kev: are you saying that responses to iqs should come back in the same order as the requests?

To a given bare JID? Yes.

As I said before, that seems problematic.

I don't see why.

It's what 6120 says you need to do.

If I send an IQ to an entity that then goes out to request a web resource to produce a response, and I send it another unrelated IQ that can be responded to immediately, that is really an issue.

The second iq must be pipelined until the first is processed.

I'm not sure if I agree this is what RFC 6120 says, but I do think it is stupid

and I don't see why you'd want that kind of behaviour

Interesting, I'm wrong, I think.

What I say only holds true if the entity is a server.

Really? Also, define 'server'

If it's a client, there seems to be no restriction on the processing order of data it receives.

"Thing at the S end of a C2S link, or at the end of an S2S link"

also, 'processing' is rather vague

you also mentioned MUC as an example

Did I?

Oh, no MattJ did

He mentioned waiting for the response to an vcard request

a MUC will send this request on to the actual occupant. Do all groupchat messages have to be queued until the response can be sent back?

(for the requesting occupant)

That's a fun question.

RFC 6120 says:

If the server's processing of a particular request could have an effect on its processing of subsequent data it might receive over that input stream (e.g., enforcement of communication policies), it MUST suspend processing of subsequent data until it has processed the request.

I believe that the 'if' is rather (intentionally) vague, and not as strict as you seem to make it out to be.

Right.

e.g. I'd say it does apply for subsequent publish requests to the same node

It seems that the following cases are strictly linear:

but not to different nodes, or any other unrelated iq or message

Sending anything to your bare JID Routing stanzas Receiving routed stanzas

And that for other requests it's vague.

Oh, no, I'm wrong.

To your bare JID or to the server itself.

so that's all about delivery

and I agree with *that*

So routing must always be in order, and processing must always be in order if it's your own server - but other entities can process in their own chosen way, subject to the note about subsequent data.

I still don't want my server to not respond to an iq:version if it is still figuring out what my roster is if the order of requests is 1) roster 2) version

That much is entirely clear in 6120

It must process those requests in order.

Process, or respond?

what MattJ says

this is crucial

and not clear at all to me

They're one and the same.

and if this is ambiguous, I think it should be made clear that handle/process != respond

Kev: I strongly disagree

Kev: and I don't see any text to support that

If I say "please get me my roster", you can't possibly claim to have completed processing of that request until you have given me my roster.

Sure I can!

If I set my mind to it.

Kev: also, I am not sure that RFC 6120, section 10.1, shares your interpretation of 'processing' in this case.

It does not talk about processing requests, but rather processing of stanzas.

"e.g., in-order processing of a roster get and initial presence as described in [XMPP‑IM]"

I'd say that was pretty unambiguous.

I think that's because of the paragraph I pasted. So a business rule for this particular exchange of different related stanzas.

So I agree on that example, but not on the general case.

But when taken with the complete bullet: "Stanzas sent by a client to its server or to its own bare JID for direct processing by the server (e.g., in-order processing of a roster get and initial presence as described in [XMPP‑IM])."

there is no reason to delay responding to an iq:version request if you sent out a roster request before that

It is clear that's just an example of the general case to which this rule applies.

Kev: so if the current text divides us in interpretation, what do you think about what I say in reality? I'd be more than happy to suggest we amend this.

I think this is a good rule.

I think it is a nightmare.

I think it's possible to find cases where it doesn't serve a purpose (as in your example), but I don't think it's harmful, and I think it's much simpler than trying to create a list of all requests that must be processed in order, which requests they must be processed in order with and which can pass through, etc.

ralphm: In your particular example it's really easy for the client to just request the version first, if response to that is more urgent than getting its roster :)

Kev: seriously?

Kev: so an entity should guess how much time it takes for a server to respond to unrelated queries and order them accordingly before sending them out?

No, not really, because just about all queries a client sends to a server it's going to get responses to ~instantly anyway.

Typically how many requests does a client make of its server? Roster, private, vcard, version maybe? These are all going to get processed quickly.

And for heavy processing platforms (i.e. not the clients own server), these rules don't apply.

I'm talking about XMPP Core, not necessarily about XMPP IM. So while such behaviour might be ok for IM clients, I'd certainly don't want to have such a thing anywhere that generic.

Kev: disco

linuxwolf: Disco should return instantly too.

Unless you were just adding it to the list, rather than a counterpoint, in which case...right.

this "instantly" is a big assumption, IMO

and I was adding to the list

disco#items to a MUC service, on the other hand, isn't subject to these rules.

Kev: I'm willing to bet that most server developers strongly disagree with your stance on this.

they do

Kev: also, a muc service could be part of the server you are talking to. I.e. council@xmpp.org *and* ralphm@xmpp.org.

ralphm: It could, this is true.

in any case, if you want to be reasonably sure things are fully handled in the proper order, you wait for the previous iq before sending the next

and that is something we ought to recommend somewhere anyway

linuxwolf: Well, no, you don't need to, because 6120 says it'll do it for you.

Kev: as this discussion points out, your stance is not universally accepted

linuxwolf: so that's really a business rule of particular exchanges, not of all message/iq processing, right?

processing is a vague term

ralphm: true, but at least for my experiences, it's true far more often than it's false

linuxwolf: but individually, per types of exchanges (say presence/roster v.s. iq version v.s. pubsub publish)

linuxwolf: not on all incoming messages/iqs

ralphm: true

linuxwolf: It's not vague, at least for iqs it's fairly clearly defined.

And it includes sending the response.

Kev: if people disagree with an interpretation (in general) it is not clearly defined

ralphm: it comes down to what's important to "ensure" is handled

I don't see how you can possibly interprep 6120 to not think iq processing includes sending the result.

Kev: see above.

ralphm: Which part of the text makes you think it doesn't include the return?

"in general the server will respond to the IQ stanza"

Kev: I don't think processing means it has to be completed; it has to be started in the received order, but not necessarily finished

which therefore means it doesn't necessarily have to respond in the received order

Kev: the part that says "If the server's processing of a particular request could have an effect on its processing of subsequent data it might receive over that input stream"

Kev: which for my example is false.

ralphm: Yes, that is for stanzas that don't fall under the previous list.

Kev: not in my world

"In-order processing applies (a) to any XML elements used to negotiate and manage XML streams, and (b) to all uses of XML stanzas"

That really doesn't seem ambiguous.

as long as you talk about stanzas, it is about delivery

in my opinion

And to make sure people don't make that mistake while reading it, it then has bullet 1 in the following list to say that it also includes the processing.

I still think it comes down to how one defines processing

Let's put it differently: I am going to suggest this text gets changed.

because a) servers don't work that way, b) I think it is undesirable

c) it might even be harmful

it seems MattJ and linuxwolf agree with me. I'm curious about other server devs of course. I also think my interpretation shouldn't impact clients at all.

Other than changing the defined behaviour of the server.

Kev: do you clients send out an iq and ignore or error on any other stanza from the same (local?) entity until that iq gets a response?

do you*r*

Ignore or error? No. I'm not convinced it won't change behaviour, though, if the client e.g. doesn't know its own nickname before it tries to join bookmarked rooms.

Kev: and to be sure: we don't agree that it changes the behaviour. My take is that it clarifies it.

Kev: in general, if you need the output of one request as the input for another, you require order on the client side. That doesn't impact this at all.

Kev: now this is just silliness … of course you can't act on data you don't yet know about, but that doesn't mean you can't handle an iq:version returning before that vcard or PEP item retrieve

linuxwolf: Unless the PEP item retrieve is also part of determining your nickname for joining MUCs, for example.

Or if the PEP retrieve is for your bookmarks and you get those before you know your nickname.

Kev: right … but how does the iq:version impact this? Does swift break if it got the iq:version response before it got the PEP item-retrieve, even though you (theoretically) sent them in reverse order?

Kev: again, if you need all responses, then yes, you should wait for all of them

Of course, clients could be written (subject to significant additional latency) to deal with not having guaranteed order - but at the moment 6120 says that the server will process its requests in order, and as such they don't need to.

there is no additional latency

linuxwolf: There is if you're suggesting that where ordering matters you must wait for a reply to the previous before requsting the next, which you suggested earlier.

Kev: only if you need the reponse!

Kev: for those things that matter, you'd have to no matter what!

I think we're agreed you can't join rooms until you know what rooms to join, yes?

and, you can't join room until you know what the user's preferred and/or registered nick is, yes?

but, that doesn't mean the client must way for the bookmark request to complete ...

… before requesting the PEP-registered nickname

Kev: what linuxwolf and I are saying actually improves latency.

and also wastes less server resources as a bonus

nor does it mean you must wait for one particular MUC room to respond to your register get before you send the next

for those things I MUST have the result, they get sent serially, pending said response

Except 6120 says you don't need to do that, because the server will enforce ordering.

Sure, you *can* do it, but you don't need it.

assuming again that the MUC room is local, I'd say that communicating with two rooms is talking to different (bare) JIDs, so that doesn't apply.

Kev: you keep repeating that the RFC says that, but it is your interpretation thereof

Sure, based on reading the words in it :p

The only way around it is if you start twisting 'processing' to mean 'doing some things it was asked to do, but not all of them'.

Kev: accept that other people might interpret words differently. If you hate that, provide functional tests for the next revision.

Kev: yes, a lot of have

because when you're dealing with 100k's to 1M's of users, it tend to need to

And elsewhere there's text that says, in a different context, that processing includes sending the reply.

I have to eat now

to be continued

I need to get back to my day job

OK, I found that 6121 says that IQ processing includes sending the reply stanza when the request is sent to the bare JID (8.5.2.1.2) - I imagine we can find similar text for the other cases.

it is too bad stpeter missed all of that

there are logs

oh, is that new?

sure :)

http://xmpp.org:5290/muc_log/muc.xmpp.org/council/120411/

will read later, but right now I'm deep in work on SASL and PRECIS

yeah, I found them. I thought they were disfunctional

they are also mentioned in the subject :)

ralphm: they were for a little while, yes

right, http://logs.xmpp.org/council/ is easier to remember :)

Tobias: that's how I found them. Obviously I knew we used to have them

somehow swift disallows copying the URL out of there :/

must be a security feature

Tobias: maybe you tried to do it out-of-order

heh

Yeah, I'm not sure why that doesn't work. Patches welcome.

stpeter: by the way, the calendar entry for the council meeting was an hour late

damn timezones

1500UTC we're meeting at these days.

it *seemed* wrong when I typed in 16:00

but I just copied that over from previous meetings

it was correct in our winter

fixed for next week's meeting

stpeter: really? I don't see an event next week.

ralphm: "refresh all calendars" worked in my calendaring program

hrm … it's not showing up for me, either

the XSF Calendar or the XMPP Council Calendar?

guess google doesn't pull it often or something

I'm using the latter

clearly we need calendaring over pubsub :P

if I use < http://xmpp.org/xsf/XSF.ics >, then I see it … but I was originally using < http://xmpp.org/xsf/xsf-all.ics >

hmm

the latter now emits a "fatal error"

I always use the files at http://xmpp.org/calendar/

why does everything move!

heh

and not leave any forwarding address

COOL URLS DON'T CHANGE!

damnit

/rant

< http://maddox.xmission.com/ >

actually lighttpd already had a redirect in place: "^/xsf/XSF.ics" => "http://xmpp.org/calendar/xsf-all.ics"

hrmph

ah … calendar

why do people use applications that can't follow http redirects!

COOL APPS FOLLOW SPECS!

I don't know if I can trust this calendar

it's not using TLS d-:

true

TLS everywhere!

™

ALL THE COOL THINGS USE TLS

UTF-8 Everywhere™®

:)

:)™

I now realize our lunch conversation is bleeding into this place

yepper!

I think I need another meeting

I'm sure you'll have another one soon enough

indubitably