XSF Discussion - 2021-07-20

  1. sadest

    ben, ben:

  2. goffi

    Hi, can anybody confirmS my understanding of https://xmpp.org/extensions/xep-0060.html#owner-purge: "If a service persists published items, a node owner may want to purge the node of all published items (thus removing all items from the persistent store, with the exception of the last published item, which MAY be cached)." . So the last item (as in most recently published) must be kept right? Because on one hand we have "purge the node of all published items" and on the other "with the exception of the last published item, which MAY be cached". (ralphm ?)

  3. Zash

    Sounds like it imagines a tiered storage system, where you have the persistence layer that stores `max_items`, and then a cache of the last item for re-sending on subscribe or presence. Purge would primarily operate on the persistence layer, but why not both?

  4. goffi

    and the caching is an implementation detail anyway. For me purge should remove all items from the node, not keeping the last one.

  5. Zash

    That seems like the sensible way, yes. But the spec allows keeping it 🤷️

  6. goffi

    I can deal with that, but I'm not sure about the spec, it just allows to keep it, or we HAVE to keep the last one? I hope it's the later, because otherwise when we have a purge event, we don't know if the last one is still there or not.

  7. Zash


  8. Zash


  9. Zash

    > mean that an item is truly optional

  10. goffi

    I understood it as the MAY is for "cached", "MAY be cached".

  11. goffi

    so the way I get it, is it MAY be cached, but we remove all but last item.

  12. Zash


  13. ralphm

    I read it as, if you need to keep the last published item, e.g. for PEP use cases, a service MAY keep the last item when a node is purged.

  14. ralphm

    Also, I don't remember this changing. I agree it is confusing.

  15. goffi

    OK, so we don't really know if last item it kept or not when we receive the purge event?

  16. Zash

    Speaking of pubsub, I wonder if I've misunderstood the whole persistence thing at some point. In Prosody, the `persist_items` setting toggles between persistence to storage and persistence to an in-memory cache. So `persist_items=false` just means that data is forgotten on reboot, but otherwise it looks the same from the outside.

  17. Zash

    Now I'm wondering if it's actually supposed to forget stuff after broadcast. And then maybe a different cache with 1 item for rebroadcast of last item.

  18. goffi

    Zash I understood it as the former too, but the later would make sense, and after checking the spec, it's not really sure which one it is.

  19. MattJ

    Yeah, I'm not sure whether the items are stored in memory or on disk is really useful (or should be controlled by) a client

  20. MattJ

    Time for Pubsub 2.0 :)

  21. Zash

    Before or after The Split happens?

  22. ralphm

    Zash: yes, that's wrong indeed. With persist_items false, you don't store them in any way and also if the item doesn't have an id you don't have to generate one for the notifications.

  23. ralphm

    The method of storage, or any other type of guarantees on the retention cannot currently be controlled by the client in a standard way. Other than maybe the pubsub#item_expire config option.

  24. ralphm

    goffi: to be honest, I think the exception for the last item feels weird to me and I wouldn't object to removing that.

  25. goffi

    ralphm: it would be possible on a draft XEP ?

  26. ralphm

    Let's see what the council thinks after you PR the change :D

  27. Zash

    Don't forget to discuss it on the standards list :)

  28. ralphm

    This author doesn't object, if that helps.

  29. goffi

    ralphm: I'm finishing a protoXEP to indicate extra data giving details on Pubsub implementation, so we can safely cache a node, I'll propose it later this week. I've added this case, so maybe it's not necessary.

  30. Zash

    s/MAY/SHOULD NOT/ ought to be safe from a interop perspective, but then who'd rely on a cache anyway?

  31. ralphm

    I'm not even sure what the difference is between caching and storage, from a protocol perspective.

  32. Zash


  33. goffi

    ralphm: I mean for caching a pubsub service from a client (local cache)

  34. goffi

    but I guess I'll propose the protoXEP first, we can discuss after.

  35. ralphm

    Unfortunately when Kev converted the repo from Subversion, we didn't keep the older history, so I can't check how it got in.

  36. ralphm

    In any case somewhere between 1.6 and 1.9

  37. ralphm

    goffi: by the way, you *can* detect this. There's a `cache-last-item` feature. Apparently it doesn't appear in the list of features in section 10 nor is it registered with the Registrar.

  38. ralphm

    That feels like a bug

  39. goffi

    ralphm: one of the problems is that those features may only be accessible to node owner (even to only read the value).

  40. ralphm

    wait huh?

  41. goffi

    those features are in node configuration right? Node configuration may not be accessible from a random user.

  42. ralphm

    Why not? They should be available via disco

  43. ralphm

    If they are not, all bets are off. You wouldn't even know if a node persists items in the first place. Also, the purge action is for the node owner, who should definitely be able to see the node config.

  44. lovetox

    ralphm, its not a give that every configuration is avaliable as disco info

  45. lovetox

    we had similar issues with the MUC XEP

  46. MattJ

    Yeah, those things are ugly :)

  47. lovetox

    but this is i guess in part because it was never intended for that or ?

  48. lovetox

    its about what features a service supports

  49. lovetox

    not neccessarily how the service is configured currently

  50. MattJ

    Yeah, I think it probably began as "it's useful to see some of the important configuration info" + "we don't want to dump the entire config into disco"

  51. MattJ

    and they don't necessarily map 1->1

  52. MattJ

    e.g. config option 'roomconfig_whois' maps to the features 'muc_nonanonymous'/'muc_semianonymous'

  53. MattJ

    I don't think it's a bad design given those constraints, but it could do with a bit more consistency

  54. ralphm

    lovetox: I meant node meta data discovery (section 5.4), where the node's configuration is passed as a Data Form in the disco#info response.

  55. ralphm

    This is the only true way to find out if, say, a node supports persistent storage. I know the section is a MAY, but not supporting this is meh.

  56. goffi

    ralphm: it can be configured per node, and it's not in node disco (at least it's not enforced). That's exactly what my XEP is doing: making sure it's in metadata. Also purge is done by node owner, but if I cache a node as simple subscriber member, I want to be aware of the purge.

  57. ralphm

    You'd get a purge notification?

  58. goffi

    if I'm a subscriber, yes

  59. goffi

    why would not I get it?

  60. ralphm

    That's the deal, I guess. If you are a subscriber, you get notifications.

  61. goffi

    yes, but not all. There are lot of corner case. For the purge I get a "purge" notification, but if I don't know if the last item is kept or not, I can't replicate the command in my local cache.

  62. ralphm

    Sure, I understand that. I agree that implementations should expose node meta data through disco. FWIW, an implementation that does should present that in a regular disco#info feature. Also this feature is RECOMMENDED.

  63. ralphm

    What silly implementations don't support this, really?

  64. ralphm

    (pubsub#metadata, I mean)

  65. goffi

    I haven't seen it a lot in the wild to be honest.

  66. Zash

    did you mean `pubsub#meta-data` :D

  67. ralphm


  68. ralphm

    Zash: oh, crap. Well, I guess we can add a ticket against Prosody :-(

  69. goffi

    and ejabberd

  70. goffi

    and propably most pubsub implementations

  71. ralphm

    goffi: I mean that Prosody presents this with http://jabber.org/protocol/pubsub#meta-data instead of #metadata.

  72. goffi

    oh prosody does it, OK.

  73. ralphm

    Oh, cool. Even idavoll has #meta-data. /me hides in shame

  74. ralphm

    I can't tell if ejabberd announces support for Node Metadata, but it does implement it as far as I can see.

  75. goffi

    yes, and non owner can't get config

  76. goffi

    how it does sorry, I've misread

  77. goffi

    it does? I have nothing on a ejabberd PEP node

  78. ralphm


  79. ralphm

    Although I wouldn't be surprised if it behaves differently for PEP nodes.

  80. goffi

    weird, I'm actually testing on edhelas blog node I don't see any metadata.

  81. Zash


  82. goffi


  83. Zash

    How did this happen?

  84. edhelas


  85. ralphm

    Zash: don't worry, if all actual implementations use #meta-data, we can change it in XEP-0060 :D

  86. edhelas

    what are you looking for ?

  87. ralphm

    edhelas: Pubsub Node Metadata (see section 5.4)

  88. edhelas

    I see

  89. Zash

    ``` <field type='hidden' var='FORM_TYPE'> <value>http://jabber.org/protocol/pubsub#meta-data</value> </field> ```

  90. ralphm

    Zash: well, yes. But that is kinda useless. It should be in a <feature/>, cause otherwise, how'd you know to expect the form?

  91. ralphm

    oh never mind

  92. ralphm

    It looks like tigase has http://jabber.org/protocol/pubsub#metadata.

  93. ralphm

    And basically anyone else: http://jabber.org/protocol/pubsub#meta-data

  94. goffi

    alright I see it on a non PEP node with ejabberd (with meta-data :-/)

  95. goffi

    but not in the PEP node

  96. ralphm

    goffi: happy for you to push ejabberd to start supporting this in PEP nodes.

  97. Zash

    And no more info on the change that added this than "mod_pubsub: Advertise title and description in disco#info"

  98. ralphm

    Zash: I'm not surprised. Idavoll was one of the first implementations, so I wouldn't be surprised if that had a knock-on effect.

  99. Zash

    So no idea if _someone_ copypasted from ejabberd, or from an example shown by whoever requested this, whatever it was

  100. goffi

    well 2 wolves found with Pubsub specs/implementations (not sure if this expression translates to English, but whatever), we have not lost our day.

  101. Zash

    Write longer commit messages plz!

  102. Zash

    (it was me)

  103. ralphm

    My commit to add support was on 2005-01-02.

  104. Zash

    this was just 2018

  105. ralphm

    The version of XEP-0060 supporting this was published on 2004-07-13.

  106. ralphm

    (the disco feature)

  107. Zash

    Heh, nodes only have a leaf identity and a form in disco#info

  108. ralphm


  109. Zash

    Someone pointed out recently that disco#info itself is required (*hint* if someone wanna send a trivial patch)

  110. ralphm

    And the service, because that's where the feature should be published

  111. Zash

    Whatabout individual nodes?

  112. MattJ

    That disco#info thing makes no sense to me :)

  113. Zash

    Mmmm, yeah, getting a non-error response kinda implies that it's supported :)

  114. ralphm

    Zash: I think nodes should only have the http://jabber.org/protocol/pubsub feature

  115. ralphm

    But I'm sure we can clarify if nodes should have disco features other than that.

  116. ralphm

    I _think_ the features listed in Section 10 are service wide, though.

  117. ralphm

    Also, I don't there's anything particularly wrong with announcing all the features on each node, but not needed.

  118. Zash

    Did you already notice that there's no #meta(-?)data feature? Only the form

  119. ralphm

    Zash: in Prosody you mean?

  120. Zash

    In XEP-0060

  121. Zash

    Orrr, wait

  122. Zash

    https://xmpp.org/extensions/xep-0060.html#registrar-features does list it

  123. ralphm

    Section 10 does, too.

  124. Zash

    Ah, because I searched for "pubsub#meta"

  125. Zash

    But then the FORM_TYPE is a bit like a feature if you squint at it

  126. Zash

    Oh, there, now I see pubsub#meta-data on the service itself

  127. ralphm


  128. ralphm

    except for the dash, of course

  129. ralphm

    I'm not sure how to treat this, spec-wise. Curious if there are other implementations that have #metadata, besides Tigase.

  130. ralphm

    intosi: what does MLink have?

  131. Zash

    What do clients expect?

  132. intosi

    M-Link doesn't do PubSub meta-data.

  133. ralphm

    At all?

  134. intosi

    At all.

  135. ralphm

    Didn't expect that, but at least it doesn't use the wrong namespace then.

  136. ralphm

    Zash: Clients might not actually check

  137. Zash


  138. Zash

    https://xmpp.org/extensions/attic/xep-0060-1.13.2.html#entity-metadata here it is #meta-data ??

  139. Zash searches vcs log for meta-data

  140. Zash

    https://github.com/xsf/xeps/commit/660ff16274cebad556115f94b6ce7550c17b32e5 https://github.com/xsf/xeps/commit/37c7b852f302fca26a885e9d2c2aea5de019fc3e https://github.com/xsf/xeps/commit/6842676930122aaa2988cf3595175849fd7fa4f8

  141. MattJ


  142. Zash

    jonas’, editors, something for you to look at? ↑

  143. ralphm


  144. ralphm

    pep. strikes with search/replace :-D

  145. ralphm

    Also https://tdan.com/the-meta-data-fiasco/18502

  146. jonas’

    why the rerevert though

  147. jonas’

    I recall the issue, but not the resolution

  148. Zash

    It's very nice when you track down an issue and find a commit with a long commit message that explains the reasoning.

  149. ralphm

    I'm not sure the assertion that "meta-data" being wrong is true.

  150. debacle

    ralphm Michael Brackett has a wrong understanding on what a tautology is. "metadata" (= data about data) is not a tautology, a "discussion about discussions" is not a tautology etc.

  151. ralphm

    Sure, but the bit about trademarks is interesting

  152. ralphm

    I also don't think that linguistically, putting a hyphen into a compound word is wrong.

  153. debacle

    ralphm I agree, that the hyphen were wrong in English, but we Germans *love* to put wrong hyphens in English compound nouns!

  154. ralphm

    Well, I know that in Dutch, compound words generally cannot have a space in them, and hyphens can be inserted for clarity.

  155. ralphm

    If I were to talk about "Service Discovery" as a concept in Dutch, borrowing the English term, I would have to write "service-discovery".

  156. ralphm

    Simply because "service" is not (used as) an adjective.

  157. ralphm

    (and Dutch also doesn't capitalize nouns like German does, so you can't tell the difference)

  158. ralphm

    jonas’: shall I put in a PR?

  159. Holger

    > ralphm I agree, that the hyphen were wrong in English, but we Germans *love* to put wrong hyphens in English compound nouns! Simply concatenating two words is certainly wrong in English, though. (Except when it isn't.)

  160. debacle

    I find it convincing in Bracketts text, that "meta" is a prefix word and therefore does not have a hyphen, so "metadata" is correct in the prosa. However, the XML namespace is a different thing.

  161. Holger


  162. debacle

    Like Bracketts example "metaphysics" etc.

  163. Zash

    Something something "sär skrivning"

  164. ralphm

    Holger: well, hyphenated compound words tend to become closed compound words with time. E.g. if a compound word is used "enough" the hyphen tends to be dropped eventually.

  165. jonas’

    the heck

  166. jonas’


  167. jonas’

    so either this was a typo in some commit id or I don't know

  168. jonas’

    I apparently merged it without re-checking closely...

  169. jonas’

    ralphm, go ahead please!

  170. ralphm


  171. jonas’

    Kev, repoke: https://github.com/xsf/xeps/pull/1047

  172. jonas’

    also, sorry everyone ;)

  173. ralphm

    jonas’: for what? The meta-data thing? Maybe you are 'just' human?

  174. Zash

    Errare humanum est.

  175. goffi

    FYI I've just proposed a protoXEP to fix some issues mentioned earlier today: https://xmpp.org/extensions/inbox/pubsub-caching-hints.html .

  176. ralphm

    jonas’: https://github.com/xsf/xeps/pull/1092

  177. jonas’

    ralphm, thanks!

  178. ralphm

    You're welcome. Zash: thanks for the find!

  179. Zash


  180. ralphm

    Also put in a drive-by comment over at Tigase.

  181. ralphm

    Look like they only put that in in May this year: https://github.com/tigase/tigase-pubsub/pull/5

  182. ralphm


  183. Zash

    The addition in Prosody was only a few months before, ejabberd seems to have had it a couple years more.

  184. mathieui

    jonas’, that was a lot of emails! (thanks for the work :D)

  185. jonas’

    well, automation did the job

  186. jonas’

    thanks to Sam for pointing out that I should trigger the automation more often :)

  187. ralphm

    yay scripting

  188. mathieui

    pushing a button to run a script is still work

  189. ralphm

    It sure is. This is what pays bills (in other places)

  190. ralphm

    Wojtek: fast turn-around :-D

  191. Wojtek

    owning to Andrzej :-)

  192. goffi

    We don't have a standard way to indicate in disco that a feature is implemented for a something but not something, do we? For instance on a server, we want to indicate if RSM support is for PEP/Pubsub and/or MAM and/or disco items. There is https://xmpp.org/extensions/xep-0060.html#subscriber-retrieve-returnsome with its "http://jabber.org/protocol/pubsub#rsm" but I have the feeling that we need something more flexible, this looks like a piece of sticky tape.

  193. goffi

    something but not something else*

  194. flow

    goffi, no, cross products of features are an issue. the similar is similar with SCE: if someone announced support for SCE and, say, chatstates, does that mean that SCE-encrypted chatstates are also supported?

  195. flow

    Personally I would lean to the safe side and usually ask for an explicit announcement of those things

  196. goffi

    Because I'm updating the Order-By XEP following feedbacks, and I need a way to indicate if it applies to Pubsub, MAM, or anything else.

  197. goffi

    What do you think of `urn:xmpp:order-by:0#urn:xmpp:mam:2` (so `feature_supported#where_it_applies`)? Do you see any problem with this kind of notation? It would be flexible and I could publish a new XEP to formalise it.

  198. goffi

    maybe something else that "#" as "#" is used by some protocols to indicate sub-features (e.g. pubsub)

  199. goffi

    or two "##"? I can try with that and see what happens on standard@

  200. flow

    goffi, hmm, what about urn:xmpp:order-by:0@urn:xmpp:mam:2

  201. flow

    but, tbh, it all looks rather ugly

  202. goffi

    if you have a better options, I'll be happy

  203. flow

    otoh, something like this would also been my first attempt at this :)

  204. goffi

    If have against "@" and it make sens as it means "at", I just find it slighly more difficult to read ("@" is close to "0"), but that's a question fo taste, not really relevant.

  205. goffi

    I have nothing against "@"*

  206. goffi

    makes sense*

  207. emus

    Why do we actually not have a nice XEP bot here? ☺ Alex: anything possible? 😃

  208. Alex

    In the old days we had one. But I can't remember who wrote and maintained it.

  209. jonas’

    emus, what would it do?

  210. emus

    point you to the xep on request or other details

  211. jonas’

    on which request?

  212. emus

    what I saw today was: !xep XEP Number -- recieve information about the specified XEP

  213. mathieui

    emus, why a bot when you can just ask Link Mauve ;)

  214. emus

    😀 work that a machine can do should be done by a machine 🙂

  215. deuill

    Is it a coincidence that Link Mauve means Purple Link, which is what followed links are usually colored as?

  216. Ellenor Malik

    uh, wouldn't they be Lien Mauve?

  217. emus

    Alex, jonas' https://github.com/mightyBroccoli/xmpp-chatbot