-
MattJ
nicoco: the additional fields were added in a later version of the XEP. To be compliant with this version, an implementation MUST support those fields, and MUST advertise #extended.
-
MattJ
It's not about shaming anyone or not 🙂
-
Alastair Hogge
Well said.
-
nicoco
MattJ: thanks for the explanation. so if an implementation only supports 3, it urn:xmpp:mam:1 then, I get it. (the shaming part was a joke, no worries ;))✎ -
nicoco
MattJ: thanks for the explanation. so if an implementation only supports 3, it should advertise urn:xmpp:mam:1 then, I get it. (the shaming part was a joke, no worries ;)) ✏
-
MattJ
No, mam:2 did not define the extra 3 fields
-
MattJ
There are many mam:2 implementations in the wild that don't support the latest revision of the XEP
-
pep.
Is there a list of o payloads that shouldn't be broadcasted by a MUC? (to prevent impersonating it)
-
jonas’
no
-
pep.
All the <{muc*}x/> stuff? And things with stamps? (which are?)
-
jonas’
anything in the MUC namespace obviously.
-
jonas’
and delay and stanza-id, yes
-
pep.
occupant-id
-
jonas’
only if you advertise the occupant-id feature
-
jonas’
(likewise for stanza-id, btw)
-
jonas’
and delay and stanza-id must only be stripped if they attempt to impersonate your domain
-
jonas’
though you might want to strip them if they otherwise deanonymise the user
-
jonas’
oh that's fun
-
jonas’
I should not have said that out loud
-
pep.
heh
-
pep.
k. Well I guess it doesn't hurt removing them anyway if they're making look like I did it, even though clients should check disco
-
pep.
Ah that means I need to have access to state in this method.. to know my own domain :x
-
pep.
https://xmpp.org/extensions/xep-0045.html#security doesn't seem to mention this. /me adds it to the TODO
-
jonas’
what exactly?
-
pep.
A list of stuff not to broadcast.
-
jonas’
ah, stripping <{muc}x/>
-
pep.
https://code.bouah.net/pep/hanabi-muc/commit/ce640864cae80f4456ac67c0914e745a8215f658 that's about all I can do without knowledge of my own domain I guess :x
-
pep.
Next step.
-
jonas’
I see an "and" in the commit message :<
-
jonas’
(it's disguised as ;, but still)
-
pep.
ssshhh
-
pep.
You have seen nothing.
-
pep.
https://xmpp.org/extensions/xep-0203.html#security < This mentions stripping fwiw
-
pep.
https://xmpp.org/extensions/xep-0359.html < This only says validating entities should check whether @from actually supports it.
-
nicoco
MattJ: I don't grasp it all tbh, but I'll just respect the MUST I guess ;)
-
MattJ
nicoco: I think you've got it, apart from that you implicitly made an assumption that the previous version of the XEP used mam:1
-
MattJ
We've had mam:1, mam:2, and mam:2+mam:2#extended
-
pep.
(mam:3! :p)
-
pep.
Just that it didn't want to be named mam:3
-
MattJ
The new changes were all additions, so switching the whole lot to mam:3 would have needlessly broken every existing implementation
-
pep.
I think it's exactly the same really. I hope implementations don't check 'urn:xmpp:mam:2' but '^uxn:xmpp:mam:2$'. So in any case they'd have multiple checks to do (or to change this regex)
-
pep.
(if they even use regex)
-
MattJ
I hope you're joking 🙂
-
pep.
What do you mean
-
MattJ
Maybe I misunderstood your statement
-
jonas’
excusemewhat
-
MattJ
I'm not sure what regex has to do with anything
-
MattJ
It has zero applicability here
-
pep.
regex or whatever. I hope they check for the full string to equal the namespace they're looking for
-
pep.
Not some partial equality
-
MattJ
They shouldn't be doing regex, no
-
pep.
That's not the point
-
nicoco
we all know what happens when you try to parse XML with regular expressions, don't we?✎ -
nicoco
we all know what happens when we try to parse XML with regular expressions, don't we? ✏
-
pep.
Forget I even said anything about regex, it doesn't matter to make this point
-
MattJ
I'm happy to say I've never used regex to parse XML, and I would never do so
-
MattJ
The sender retracted their previous message but your client does not support that
-
jonas’
I cannot say the same.
-
MattJ
I did it yesterday, with sed, in a bash script (it was a config file though, not XMPP)
-
pep.
So I stand by my comment above, I see no different in using mam:2#extended or mam:3. It's exactly the same change to me for a client to support it
-
pep.
(or ignore it)
-
pep.
difference*
-
Holger
MattJ: I still believe that MUST can be slightly confusing as it may mislead the reader to believe the client can rely on those fields. Yes if you strictly follow the text then you'll probably get that you should check for the :2#extended feature. But if I see a MUST I'll always assume it was added to make a protocol/interop guarantee, which isn't the case here. So if I knew nothing about 0313 history it'd confuse me as well.
-
Holger
No big thing of course.
-
MattJ
pep. [09:17]: > So I stand by my comment above, I see no different in using mam:2#extended or mam:3. It's exactly the same change to me for a client to support it The difference isn't for clients that are updated to support it, the difference is for clients that are not updated - they continue to work
-
pep.
MattJ, they would anyway, since "mam:2" definitely isn't the same string as "mam:2#extended"
-
MattJ
Ah, so you're saying the XEP update should have said you MUST advertise mam:2 and mam:3
-
MattJ
Could have done that, yes
-
pep.
Ah, yeah
-
pep.
Ok sorry if that was the confusion
-
MattJ
That would probably have been confusing, as people might have tried using mam:3 in stanzas
-
MattJ
#extended is just a disco feature
-
MattJ
The core namespace remained mam:2
-
pep.
It's just names and conventions. You make them do whatever you want to :)
-
MattJ
And we did 🙂
-
pep.
I'm not saying it doesn't make sense, it's fine this way. Just that there wasn't much more differences to me to use mam2/3
-
MattJ
But it wasn't done only to make upgrading easy, it was done to keep compatibility with existing clients (given that 98% of the protocol is the same)
-
pep.
It looks like you're saying mam2/3 wouldn't keep compatibility and I'm saying that's not true. In both cases a server zould advertise both features right?
-
pep.
(if it supported both)
-
MattJ
It's not as simple as that, unfortunately
-
pep.
Ok I don't understand then
-
pep.
But it's fine, we can drop it if it gets too long :P
-
MattJ
We had implementation experience of supporting multiple namespaces concurrently with e.g. XEP-0198
-
MattJ
It's certainly possible, and perfectly fine if it's necessary. But it's not as trivial as it seems on the surface, and there was no need to do it in this case.
-
pep.
I actually don't get how it differs technically. 'mam:2#extended' is another version, same as 'mam:3'
-
MattJ
No, note how in the XEP, all the elements are still in the mam:2 namespace
-
pep.
Ok, it just makes slightly more sense to name it mam:2#foo, I guess, but that doesn't change that you could have advertised anything to say "we support this" and still used mam:2 in the spec
-
pep.
It's just a flag, the name doesn't really matter. It makes it easier for humans for sure (even though some may wonder why you're not using #extended then as the ns :p)
-
MattJ
Yes, you're totally right
-
MattJ
Given the choice though, I think #extended was a better fit because advertising mam:3 would lead people to assume that's the namespace of the elements too. They might assume that with #extended too of course, but less likely I'd hope.
-
MattJ
All we'd need was one server accidentally supporting mam:3 elements and then we'd soon have one or more clients doing it too 🙂
-
pep.
Holger, that "if .. MUST" doesn't disturb me that much. As an analogy I think I prefer !(foo && bar) rather than !foo || !bar. Plus in a spec it can quickly get messy to know what goes with what :/
-
Holger
Not sure we mean the same thing? My point is, for me a MUST implies a guarantee, i.e. protocol breakage if it's ignored. In this case there's no breakage if it's ignored, the guarantee only depends on the XEP revision, which is not a protocol thing. So the MUST may be confusing while trying to understand the protocol.
-
pep.
Well the MUST is behind a predicate here
-
jonas’
Holger, no it doesn't depend on the XEP revisino, it depends on a feature.
-
Holger
jonas’: The text says "servers MUST support (and announce) the feature".
-
pep.
The text says "Servers supporting"
-
Holger
It doesn't say "if the feature is announced, it MUST be supported".
-
pep.
It could be made slightly more explicit maybe, with an if
-
pep.
Maybe we should stick to simple english in specs, to make that less confusing and more accessible :/
-
jonas’
Holger, > Servers supporting fields marked with an asterisk (*) MUST advertise the disco feature 'urn:xmpp:mam:2#extended' and clients that depend on these fields MUST verify that the server advertises this feature before attempting to use them.
-
pep.
(That'd definitely help me too)
-
Holger
> Six further fields are defined by this XEP and MUST be supported by servers, though all of them are optional for the client.
-
Holger
nicoco pointed that out yesterday. I didn't remember it. Which is probably also due to the fact that it makes no sense to me 🙂
-
jonas’
Holger, https://xmpp.org/extensions/attic/xep-0313-0.6.html#filter
-
jonas’
the non-asterisk fields were always part of MAM
-
Holger
Sure, but those are only three?
-
jonas’
yes
-
jonas’
ohhh
-
jonas’
that does indeed sound like a weird thing then
-
jonas’
MattJ, wording issue or intentional? :)
-
jonas’
("Six fields MUST be supported; servers supporting the last three MUST announce feature X". I would've expected: "Three fields MUST be supported, the other three fields MUST be supported iff the feature is announced")
-
Holger
👍 If the real intention simply was to encourage servers to offer them, it could just be stated that way, in my book.
-
MattJ
It's not encouragement, they are mandatory
-
MattJ
As is #extended
-
pep.
It's there because.. some previous version of mam:2 didn't have them?✎ -
jonas’
MattJ, but that's a weird thing to say if "clients that depend on these fields MUST verify that the server advertises this feature before attempting to use them"
-
jonas’
but I get your perspective now
-
jonas’
with mam:2 as one version, and mam:2+mam:2#extended as the next version.
-
jonas’
interesting way to look at things, I shall keep that in mind.
-
pep.
It's there because.. some previous version of 313 didn't have them? ✏
-
MattJ
The alternative was mam:3 and breaking the world, or making these fields perpetually optional
-
Holger
Effectively they _are_ optional until we bump to mam:3.
-
MattJ
How so?
-
MattJ
It's a federated open ecosystem. By that logic, everything ever is always optional 🙂
-
pep.
And even the bump to mam:3 would be optional :P
-
MattJ
This is no different from bumping to mam:3 really, as pep said... they are just strings
-
Holger
Adhering to a MUST is not "optional".
-
Holger
In my book.
-
MattJ
The only difference is backwards compatibility is gained for legacy implementations
-
MattJ
Agreed, but you just wrote that it is
-
Holger
Well I expressed my concerns above, guess I won't bring them over better by adding even more text 🙂 Call it a day.
-
pep.
Holger, it is. It depends on time, motivation, money, [insert something here]. And while that's not implemented you're just "not compliant"(tm), or half-compliant
-
pep.
(or something-compliant)
-
MattJ
A server cannot be compliant with the latest XEP version without supporting those things
-
MattJ
They are not optional
-
MattJ
However implementations may not be up to date with the latest version of the XEP (whether it uses mam:3 or #extended)
-
Holger
Yes I get that idea. I just think it's confusing to do offer a guarantee that depends on a XEP revision, as the latter isn't a protocol thing.
-
MattJ
We do that all the time though
-
MattJ
I think you're only confused because we didn't call it mam:3
-
Holger
We may disagree but that's definitely not my confusion, no.
-
MattJ
Ok
-
MattJ
So a XEP can never mandate stuff in a new revision?
-
pep.
Or from which point of its lifecycle (knowing that MAM was experimental for a really long time while being implemented in many places)
-
Holger
MattJ: If ignoring the new MUST breaks nothing then my answer would be no.
-
pep.
no it can't mandate stuff in newer revisions?
-
MattJ
I think that's where we disagree then
-
Holger
Not without actually bumping the namespace, no.
-
pep.
Well it has been bumped here
-
MattJ
This happens widely in protocols
-
MattJ
Like how TLS 1.3 has many new MUSTs, but still advertises as TLS 1.2 in the handshake for compatibility purposes
-
MattJ
In practice many things still accept 1.2 right now, but that won't always be the case
-
flow
doesn't tls 1.3 still negotiate that it's 1.3 despite the fact that the TLS version announced in the hello message is still 1.2?
-
kapad
as an somehow 'outsider', means that dont deal in developing xmpp server/clients, but also have read some of the xeps around, i dare to say that i feel a xep never give the choice how you gonna advertise a feature, but how you must. xep describes both the implementation and the language between xmpp entities. and because that lang spoken by machines must follow syntax at any cost
-
flow
> MattJ> So a XEP can never mandate stuff in a new revision? It is less about the revision, but about mandating new stuff without a namespace bump. And even there, it depends on the "new stuff"
-
flow
IMHO the mam xep could have simply strongly encouraged mam:2#extended, instead of mandating it with a new revision under the same namespace
-
pep.
There is a namespace bump
-
flow
I obviously don't see it so feel free to enlighten me, it appears the namespace is still mam:2
-
pep.
The bump is #extended
-
flow
that's one way to view it, but certainly not the tranditional namespace bump kind
-
pep.
As I said earlier, it's as if the spec had said "advertize both mam:3 and mam:2 to stay compatible"
-
flow
pep., fwiw, I agree with MattJ that bumping to mam:3 just because there fields where added whould have been destructive
-
flow
because then suddenly all mam:2-only implementations could no longer use mam:3-only services, for no good reason
-
pep.
I think you've missed my point then, but it's fine
-
flow
if you "extend" a spec, by basically optional fields, then the mam:2#extended approach is sensible
-
flow
pep., very well possible, it was a large backlog to read ;)
-
pep.
You can "extend" a spec naming the extended ns whatever you like, here it's "mam:2#extended", it could have been "mam:3"
-
pep.
In both cases the spec says you advertize "mam:2" as well to stay compatible, and you're done
-
pep.
Anyway, there is a bump to me here
-
pep.
Just not the kind we're used to see (numbers) but it's the exact same thing to me. The string changes
-
pep.
Just that 313 defines "Compatibility with the spec" as "You have implemented all-the-things". I doubt generally we see this wording in other specs? But I don't really see an issue with it.
-
pep.
It "only" prevents you to call your impl fully compatible if it doesn't implement everything
-
jonas’
I don't think that sticking to the notion of "one namespace per document version" is a good thing
-
jonas’
it's counter to the extensibility idea
-
pep.
I agree
-
jonas’
as long as elements don't change incompatibly, we should stick with the same identifier
-
jonas’
(this has been done in MIX IIRC)
-
jonas’
so this step, with adding mam:2#extended *and* mandating it in the current XEP version (which was, by the way, before stabilization as 1.0), is very sensible to me✎ -
jonas’
so this step, with adding mam:2#extended *and* mandating it in the current XEP version (which was, by the way, before stabilization as 1.0) without changing anything existing as mam:2, is very sensible to me ✏
-
jonas’
and using mam:2#extended instead of mam:3 is also good because it clearly shows that this is not the traditional way we've handled namespace versioning
-
jonas’
so all fine by me
-
flow
mandating #extended in the current MAM version gains you nothing compared to just strongly encouraging it, instead it just causes confusion.
-
flow
simply because there is no gurantee that there isn't an implementation out there that never has seen the new XEP revision and just announces mam:2
-
jonas’
flow, I think quite the contrary
-
jonas’
to someone coming newly to the spec, it's better to mandate it.
-
jonas’
that's ok, clients are clearly told they need to check for #extended
-
flow
why? the spec could simply say "if you are working on an mam implementation, then you really should also implement #extended"
-
flow
that would have the same effect
-
jonas’
that are more words than "MUST implement this"
-
jonas’
less words good
-
jonas’
you'd still need the notice that client devs need to check for the feature explicitly
-
flow
instead, what we have causes confusion, as can be seen by the question that started this discussion
-
Holger
"Why on earth would I need to check for #extended if it's mandatory?"
-
flow
what Holger says
-
pep.
Because specs are living documents :x
-
pep.
Things change
-
flow
pep., that shouldn't be an excuse that they can be confusing
-
flow
or even cause interoperability issues
-
jonas’
we can add "because it was added as an extension to mam:2"
-
jonas’
but I'm not sure that helps anyone
-
MattJ
Holger [11:21]: > "Why on earth would I need to check for #extended if it's mandatory?" Why check for mam:3 if it's mandatory?
-
MattJ
It's the same difference
-
pep.
s/ for mam:3/ even :P
-
flow
I don't see how mam:3 compares to the mam:2 + #extended situation
-
jonas’
imagine those form fields had been a separate document
-
MattJ
I don't see how it differs
-
jonas’
and it'll become clear
-
flow
mam:2 can work without #extended, it is a prime example of an extension that does not break backwards compatibility on the protcol layer
-
MattJ
mam:1 still works if implemented, too
-
Holger
MattJ: Are you talking mam:3 instead of #extended or mam:3 instead of mam:2+#extended?
-
MattJ
Either works
-
flow
mam:3 instead of #extended would be pretty strange and not idomatic
-
Holger
MattJ: The former breaks compat, obviously.
-
MattJ
A client that depends on the stuff in the latest revision can check only for #extended
-
flow
and mam:3 instead of mam:2+#extended would be destructive, as in, cause interoperability issues for no reason
-
Holger
MattJ: Yes. I'm not arguing that something doesn't work. Just that it's slightly confusing when trying to grasp the spec.
-
jonas’
Holger, I guess PRs welcome then
-
pep.
"jonas’> imagine those form fields had been a separate document" < this really
-
flow
I am not sure what would change if those form fields would be specified separate document?
-
Holger
Would 0313 mandate to implement it?
-
flow
but I view them as something that could have been specified in an extra document (like the various pubsub extensions)
-
flow
not sure how this is relevant for the discussion, though
-
jonas’
flow, if they had been a separate document, nobody would be confused about the "mandatory fields and check for this feature and don't rely on it unless you see that feature" thing
-
flow
if they where in a separte document, then it would also be strange if the mam xep says that you also need to implement the mam:2#extended xep…
-
jonas’
right, it wouldn't say that
-
jonas’
instead we'd have an indiscoverable mess
-
flow
why would it be indiscoverable?
-
jonas’
how would you learn that XEP-5555 is relevant to you, when just looking at XEP-0313?
-
flow
forward pointer in the mam xep?
-
MattJ
Because MAM would no longer mean "XEP-0313"
-
flow
similar to pubsub does not just mean xep60
-
jonas’
which isn't exactly great
-
MattJ
Once you add the pointer and say it's mandatory, might as well put it in the same doc 🙂
-
flow
right, but I am arguing that saying it is mandatory gains you nothing and only causes confusion, compared to just strongly recommending it✎ -
pep.
(Also how MUC doesn't just mean 0045. /me looks at https://wiki.xmpp.org/web/MUC_Extensions )
-
pep.
"Once you add the pointer and say it's mandatory, might as well put it in the same doc" < "It depends"(tm)
-
pep.
The doc gets huge and messy, you might want to split it up
-
pep.
Just that the xep discoverability story is meh
-
flow
right, but I am arguing that saying mam:2#extended is mandatory gains you nothing and only causes confusion, compared to just strongly recommending mam:2#extended ✏
-
Zash
flow, if you don't implement #extended then what you have is https://xmpp.org/extensions/attic/xep-0313-0.6.3.html
-
Holger
I (we?) are arguing this MUST to be confusing, at least to newcomers. There's an 'extended' feature that servers MUST implement and announce. To me, the obvious question would be "why do clients need to check for it if it's mandatory?" Even if I'm on the right track and assume spec evolution to be the culprit, I'd go on "then why is it mandatory now, am I missing something?". I may be wrong about this potential confusion. But I'm confused how you guys keep insisting on explaining the status quo, rather than explaining why you see no potential for confusion 🙂
-
Holger
I mean this entire topic came up due to the confusion I have in mind.
-
jonas’
propose wording changes
-
jonas’
best way to discuss stuff is when looking at a diff
-
Holger
Yeah, except that I'm not sure how to word the reason for it being mandatory now.
-
Holger
But I can propose wording that explains how the additional feature exists due to spec evolution, yes.
-
MattJ
> rather than explaining why you see no potential for confusion I can't say I don't see potential for confusion, rather I see potential for confusion everywhere I look for it. I'm happy to find ways to reduce confusion, but you have to draw a line somewhere
-
MattJ
Adding more words about a couple of simple MUSTs also has potential to cause confusion