-
Kev
After the comment on the list about the carbons/MAM race condition, I was thinking further on this last night/this morning. It’s *really* unpleasant to solve. Easy to solve in protocol, horrible to try to actually implement said protocol.
-
Ge0rG
Kev: a carbons/MAM race condition post on the ML? I haven't sent any mails for months now!
-
flow
Kev, care to elaborate on the last part?
-
flow
is there some resource where the issues and potential solutions are listed? a wiki page? I know there were countless discussions of this, but sadly, the details a no longer in my memory (but probably in background storage somehwere ;))
-
Ge0rG
pick a random ML post by me about carbons and or mam ;)
-
Kev
Basic situation is that as a client you’d like to be able to login, enable carbons before receiving any messages and know the MAM ID directly preceeding the first message you will receive. (i.e. you don’t get dupes and you don’t get holes)
-
Kev
But on the server actually achieving that is rather unpleasant when the MAM query is going to be async, and you’re going to be continuing to receive messages for the user while that async operation is in flight.
-
Kev
s/MAM query/query to the backing store for MAM/
-
flow
ahh, so the atomicity of the protocol operation is hard to implement on the server side, without getting a performance hit. but I assume it's trivial on the client end
-
Kev
It’s trivial on the client end in the sense that the client doesn’t have to implement atomicity.
-
flow
I am not sure where the async part and "continuing to receive messages" comes from, though. can't you simply make it a synchronous operation?
-
Kev
I’ve come up with a relatively straightforward solution for a single node server, but haven’t got anything that works in a cluster yet.
-
Kev
Block the whole server until the db/fs result comes in?
-
flow
clarify "whole server"
-
flow
you block the inbound elements of the client performing the operation
-
flow
maybe even
-
Kev
That isn’t what’s needed.
-
flow
isn't this operation part of bind2?
-
ralphm
Is it a problem if there's an overlap?
-
Kev
ralphm: It could be construed a problem if the overlap causes dupes.
-
ralphm
But the dupes can be detected?
-
Kev
Unless we tell clients they need to dedupe, which would be …unpleasant.
-
Kev
flow: Yes, I was pondering how to actually implement bind2 in a non-trivial server implementation :)
-
ralphm
Well, we don't have exactly once delivery semantics, so shouldn't clients already do so?
-
ralphm
Generally clients also have local message storage and those need to be compared to MAM results as well?
-
MattJ
Kev: the client can't receive messages until the resource is bound, which is precisely why this should happen during binding
-
MattJ
Oh, but the user can, right
-
Kev
MattJ: Yes, that’s the point of Bind2, but that doesn’t make it easy to avoid the race on the server.
-
Kev
Yes, precisely.
-
MattJ
So after binding, reprocess any messages since the ID you gave the client?
-
Kev
What does ‘reprocess’ mean here?
-
MattJ
Pretend they just arrived, deliver them to the new client
-
Kev
By querying the archive, or by caching them in-memory in the server?
-
MattJ
That's up to you
-
Kev
Well, not really, because if you do it with an async operation (1), you get the same race again :)
-
Kev
But although I can solve this for the trivial case, I haven’t come up with a model that works for a multi-node server yet.
-
Holger
Right, it just moves the problem from the client to the server node.
-
flow
can't you even deliver them to the new client via the mam archive? not all messages are archived✎ -
flow
can you even deliver them to the new client via the mam archive? not all messages are archived ✏
-
flow
so you have to do it by storing the messages in memory
-
flow
which probably means you need a defense mechanism in case the capacity exceeds a limit or a timeout happens
-
flow
or, erm, hmm, messages that are not archived are not relevant here…
-
flow
so you could do both, a in-memory cache and a mam query as fallback
-
flow
wait, the mam query races against new live messages, right?
-
Kev
You’re starting to get the picture of how this is less straightforward than it sounds, aren’t you? :)
-
flow
the sad thing is that I think I had that picture a while ago, but then lost it again
-
ralphm
Holger: FWIW, moving complexity to the server is a good thing
-
Holger
I do agree with that.
-
Kev
I agree with that to an extent. Implementation complexity I agree with. Pushing computational complexity onto the server adds up pretty fast.
-
MattJ
Unrelated: does anyone recall which spec contains the MIX roster annotation stuff?
-
MattJ
Aha, XEP-0405
-
MattJ
That's confusing
-
MattJ
Ok, so XEP-0405 is a candidate to replace or merge into PAM (XEP-0376)
-
arc
Did it connect?
-
arc
Oh finally. !
-
arc
Ralphm: you here?
-
Sam
o/ if you meet today please consider approving or sending feedback to the initial fiscal host rules (and bear in mind that we can always change them later, we just need something on the website to make opencollective happy): https://github.com/xsf/xmpp.org/pull/915
-
arc
I just spent 4 hours setting up an alternative xmpp host along with updating all my SSL and let's encrypt pieces.. so I certainly hope we are meeting today!
-
arc
It appears that something in the let's encrypt API changed and that threw out all my Cron tasks for monthly cert updates.
-
arc
MattJ, dwd: ?
-
MattJ
Here o/
-
MattJ
Not looking hopeful :/
-
arc
No it's not.
-
dwd
Here!
-
dwd
Sorry, had some networking issues.
-
dwd
MattJ, arc ^^
-
arc
Join the club 😆
-
MattJ
Ooh
-
arc
Well we are missing a chair
-
arc
ralphm: is apparently away this week
-
Zash
Be the chair you wish to see
-
arc
I have two sick hens and just spent 4 hours trying to get lets encrypt to work again, so not it
-
dwd
Well, this is fun. I can see inbound messages on Gajim on my laptop but not reply, but if this works I can send but not receive on my phone.
-
arc
You are received!
-
dwd
Man that's weird.
-
dwd
So, anyway. I've yet to review the fiscal policy, and in particular I don't know if it should be a XEP to give us the change process.
-
MattJ
Only if I can append a cookie recipe
-
dwd
I do need to do a pass over the CoC in order to pick up comments.
-
dwd
And yes, a cookie recipe, I'm totally up for that.
-
MattJ
There is also another document for review: https://github.com/xsf/xmpp.org/pull/933
-
dwd
Ah, I had not seen that.
-
dwd
On a quick skim, I'm mildly concerned by the notion of declaring an Experimental Procedural XEP to be, in effect, Active.
-
MattJ
:)
-
dwd
I understand why, but it has risks.
-
MattJ
The rationale is that the scope is just operators@, for the window of time until the CoC advances
-
MattJ
and it avoids a copy/paste into that document
-
MattJ
We've had more need of moderation this week, and I'd really like to get this document up
-
MattJ
I don't even know that it *needs* board approval, I'd happily host it on my personal domain and link people to it there while I'm the one moderating the MUC
-
dwd
Understood. I won't block it, and as I say I understand the desire to get something up ASAP.
-
dwd
And indeed, I don't know that it needs Board approcal, but it certainly doesn't hurt. It's a well written document, I'd be happy for it to be on our site, and governing the operators@ channel.
-
MattJ
Yes, I'd obviously rather board approval and having it on xmpp.org - otoh if we don't approve it this week and further need arises, I would rather just merge it or host it elsewhere meanwhile
-
dwd
arc, You have an objection to just merging it now?
-
dwd
arc, Sorry, that sounded leading. I mean, do you have any objection?
-
arc
I do not have any objections no
-
arc
I think at this point the more eyes that are on it the better
-
MattJ
👍
-
MattJ
Obviously it's not set in stone... if people are in favour of the document in principle we can (and likely will) amend it down the road
-
dwd
Right, we seem to have run out of steam. I'll get on with a CoC update tomorrow and hopefully move that along toward a Last Call.
-
arc
Sounds good. +1 week?
-
MattJ
wfm
-
dwd
Sounds good to me.
-
moparisthebest
since the meeting is over, relevant to earlier dwd + arc https://www.moparisthebest.com/images/xmpp-vs-matrix.jpg (I couldn't find original so I remade it from memory)
-
ben
heh that's very accurate
-
edhelas
moparisthebest :D
-
edhelas
moparisthebest time to put it on /r/xmpp
-
mdosch
😂
-
jubalh
moparisthebest: :D
-
jubalh
i will have to steal this for twitter
-
eevvoor
emus, I made a pull request for the xmpp provider list.
-
eevvoor
Due to the blabber.im takedown in three days.
-
emus
thx, I recoomend to let the emotions calm down a bit. Seem they change their minds quickly
-
eevvoor
:D
-
eevvoor
Not good for server admins to be emotional.
-
emus
well, everyone is emotional, the responsibility is to get to know all of them that you know how to deal with it accordingly