XSF Discussion - 2023-03-09

  1. Daniel

    what does an extend MAM metadata response look like for empty archives?

  2. Daniel

    and is it different from disabled archived?

  3. MattJ

    It's on my specs to-do to clarify this: https://logs.xmpp.org/jdev/2023-01-16?p=h#2023-01-16-3eedf318e3ba95f6

  4. Zash

    MUST? Prosody violates this then.

  5. MattJ

    In which cases?

  6. Zash

    Empty archive

  7. MattJ


  8. Zash

    It would return just <metadata/>

  9. MattJ

    I think we just need to clarify "if the archive is not empty"

  10. Daniel

    empty metadata sounds good to me though

  11. MattJ

    I'll write that email now

  12. MattJ

    or... just get back into XEP mode and write a PR

  13. Daniel

    I think it should also spec that specific case too. instead of returning an error or so

  14. Zash

    one could possibly argue for item-not-found as well

  15. Daniel

    not merly allow empty metadata but say thats how it's supposed to be i mean

  16. MattJ

    As I noted in the above chat, I prefer empty to error, partly because <metadata> is also used in bind2

  17. Daniel

    yes. and we should make it distinct from archive is broken or not enabled

  18. MattJ

    Right, what does it mean for an archive to be "disabled"?

  19. MattJ

    Wouldn't you just have service-unavailable then?

  20. MattJ

    and no disco feature

  21. MattJ

    and no <metadata> in bind2

  22. Daniel

    mhhh I don’t know. server supports it but user has it disabled

  23. Zash

    broken? internal-server-error?

  24. Daniel

    that would probably still have the feature?

  25. Zash

    disabled in user preferences?

  26. MattJ

    internal-server-error is wrong if it's intentional (internal-server-error could imply that trying again might work)

  27. Daniel

    anyway. i haven’t full thought through all the edge cases yet. but I think we already concluded that empty element is the right way to go; independently of how corner cases are handled

  28. Zash

    > archive is broken or not enabled clarify what those terms mean here?

  29. Daniel

    but I was just thinking that on empty metadata will start my MAM-id tracking on the client side. and any sort of error would lead me to try again on the next login

  30. Daniel

    yes i mean disabled by user preference

  31. MattJ

    I think it should just be missing from disco features then

  32. MattJ

    Those are per-user already

  33. Zash

    What would be missing?

  34. MattJ


  35. Zash

    Does that mean disco changes on preferences?

  36. MattJ

    Otherwise I'm not sure what the difference is (or how to signal) between "not supported so it doesn't work" or "disabled so it doesn't work"

  37. MattJ

    or why that distinction is useful to have

  38. Zash

    Buuut doesn't disabled just mean no _new_ messages are added to the archive, but there still could be some that you could query for?

  39. Zash

    I don't think we defined that disabling archiving would wipe it (yet)

  40. MattJ

    I don't think we want to go down that road

  41. MattJ

    We could, but I don't see a use-case for it

  42. MattJ

    Removing disco features when MAM is disabled is reasonable and backwards-compatible

  43. MattJ

    Because claiming to support MAM when actually no MAM operations will succeed is silly

  44. Zash

    So disabled := default==never AND always:empty() AND never:empty() ?

  45. MattJ


  46. MattJ

    I don't like that options protocol though, and I'd like to retire it

  47. Zash

    in XEP-0441 terms

  48. pep.

    if disco is removed from my account, how do I know I can reenable it? Because it's supported by the server?

  49. pep.

    if disco is removed from my account, how do I know I can reenable it? Because it's enabled on the server?

  50. MattJ

    You re-enable it through whatever protocol you disabled it

  51. pep.

    who is "you"

  52. MattJ

    the account owner

  53. MattJ

    The same "my" from your message

  54. pep.

    I mean which client. How does the client know

  55. MattJ

    The same way it knew how to disable it

  56. pep.

    Ok I'm missing something that seems obvious to you. The disco feature is different?

  57. Zash

    But if that way goes away if you disable it, how does it know it's still there?

  58. MattJ

    What goes away?

  59. MattJ

    The configuration protocol?

  60. pep.

    The disco feature

  61. MattJ

    Only the MAM disco feature would go away

  62. pep.

    https://xmpp.org/extensions/xep-0441.html there doesn't seem to be a disco feature here

  63. MattJ

    Oh, XEP-0441

  64. MattJ

    Two things:

  65. MattJ

    1) > I don't like that options protocol though, and I'd like to retire it

  66. MattJ

    2) It's not clear that setting it to 'Never' and having nobody in your 'always' list is the same as actually disabling MAM entirely

  67. MattJ

    Nothing has changed in that regard

  68. pep.

    But that doesn't conflict with what I'm asking :p

  69. MattJ

    I don't think you can fully "disable MAM" via XEP-0441, I don't think it can/should remove the disco feature (because then yes, you lose the ability to configure MAM)

  70. pep.


  71. MattJ

    I don't think XEP-0441 should continue to exist, and I intend to remove it from Prosody in the future

  72. pep.

    So what is the "configuration protocol" you're talking about

  73. MattJ

    It doesn't yet exist

  74. pep.


  75. MattJ

    Prosody doesn't allow you to disable MAM on a per-account basis, or remove the disco feature

  76. pep.

    No wonder I was configured

  77. MattJ

    As far as I'm concerned, all this is hypothetical

  78. MattJ

    Rather than the options XEP-0441 gives you, I'd rather switch to something that allows someone to control the retention settings, and remove support for the per-JID stuff (which has the ability to break things in unexpected ways)

  79. MattJ

    We also need to add an in-band way for someone to purge their archive

  80. Zash

    Also might let you do funky things like include MUC PMs in MAM :D

  81. Daniel

    MattJ, just for my understanding, mam’s before-id and after-id are inclusive (the stanza referenced as after-id is included in the results) buf rsm after/before are exclusive, no?

  82. Zash


  83. Zash

    I thought before-id and after-id was a trick to let you have both after and before in RSM

  84. Zash

    Thus same semantics

  85. Zash

    Except for the ordering implications of before

  86. Daniel

    the XEP says they are (in some scenarios) interchangable

  87. Daniel

    but what i'm trying to get at is that i'm suspecting they maybe aren’t

  88. Daniel

    after-id: A stanza ID indicating the first message in the query results.

  89. Daniel

    this means inclusive, no?

  90. MattJ

    Umm, I think that is wrong

  91. Daniel

    i mean they kinda have to be inclusive. because when I retrieve a metadata and then do a query for after-id=metadata.start and before-id=metadata.end I want start and end in the result

  92. Daniel


  93. Zash

    How did you come to that kind of reasoning?

  94. Daniel

    but rsm because it is a paging thing needs to be exclusive or otherwise page A and page b would have overlapping entries

  95. MattJ

    To fetch everything you don't specify any before/after

  96. Daniel

    so the intent is that they are exclusive?

  97. MattJ


  98. Daniel

    ok; suppose I’m catching up. I know my last local MAM id is `23`. I retrieve the metadata (or get it via band2) and learn the the current end is `42`. then I’m doing a query for anything between 23-42

  99. Daniel

    i want the 42 to be included, no?

  100. MattJ

    You do. You run a query for after-id=23

  101. MattJ

    You don't include before-id

  102. Daniel

    what does that metadata thing do then?

  103. Zash

    But then you may run into live messages?

  104. MattJ

    The metadata thing is informational, for example it can be used to determine when there are gaps

  105. Zash

    Can it?

  106. MattJ

    Since start/end are implicit in any query, you don't need to actually insert those things into filters

  107. Zash

    Due to being away for a long time, so the server expired previous overlap?

  108. MattJ


  109. Daniel

    ok... I thought it could be used to put an upper bound to my catch-up query

  110. MattJ

    You would use your last local id for that, no?

  111. Daniel

    that’s the lower bound

  112. Zash

    So, If the <start> is *not* in your local database, then you have a gap. Probably.

  113. tmolitor

    Daniel is ordering the bounds by timestamp in ascending order ;)

  114. Zash

    That is *wrong*. Don't do that.

  115. Zash

    Timestamps are bad, mkay!

  116. MattJ

    I think it was a comment on Daniel's use of "upper" and "lower" in this conversation (which is logical), hopefully not on database schema

  117. Daniel

    ok forget upper and lower. but I thought i could limit it on both sides

  118. tmolitor

    yes, like MattJ said :)

  119. Zash

    You could query with after-id=your last local message, <{rsm}before/> and page backwards until you don't want to anymore

  120. Zash

    That would get you the latest new messages first

  121. Zash

    Except thanks to OMEMO, that's totally useless, innit?

  122. Daniel

    my question isn’t about the order of messages

  123. Zash

    Daniel, upper bound on number of messages to retrieve? that's what I answered. query backwards from most recent. Then you can bound the result at any point by stopping.

  124. MattJ

    No, that's also not what Daniel is asking for

  125. Zash

    Then I have no idea what anyone is talking about

  126. Zash

    And I probably won't understand without a whiteboard

  127. MattJ

    It's just that there is no way to form a MAM query that says "get me the archive contents as they were when you sent me the <metadata>"

  128. Zash

    No inclusive id fields no

  129. Zash

    except ids

  130. MattJ

    You have to have an open-ended query, and as you put it earlier, deal with the possibility that you might "run into live messages"

  131. MattJ

    But you can just stop querying after you find the message with the id

  132. MattJ

    The end id, I mean

  133. Zash

    or do a query based on the meta, then ids={start, end}

  134. MattJ

    Yes, that's one way :)

  135. MattJ

    Pity we didn't define 'ids' to be additive

  136. MattJ

    That would solve this relatively cleanly

  137. singpolyma

    Sortable uuids!!

  138. MattJ

    and it doesn't generally make sense to combine it with other filters

  139. Zash

    singpolyma, orthogonal, but nice

  140. MattJ

    Yes, if designing from scratch I probably would have mandated sortable ids

  141. Guus

    https://xmpp.org/schemas/ does not appear to hold a schema that defines the http://jabber.org/protocol/compress/exi namespace, although https://xmpp.org/extensions/xep-0322.html links to http://xmpp.org/schemas/exi.xsd (and even has a byte size and hash for that file). Does anyone know where that file disappeared to?

  142. Guus

    I have a project that contains a XEP-0322-01.xsd, but that's clearly outdated.

  143. Ge0rG

    something something xmpp protocol registry

  144. Guus


  145. Guus

    https://github.com/joachimlindborg/XMPP-EXI/blob/master/exi.xsd seems more up to date - but I'm not sure if it is the most recent version

  146. Guus

    It's not of a size that's equal to what's listed in the XEP. :/

  147. Peter Waher

    The contents of the file would be what is available in section 7 of the XEP.

  148. Peter Waher

    Perhaps white-space might be different however

  149. Guus

    Thanks Peter. Sucky for hashing, but at least i've got the namespace and element names right this time ... :)

  150. Guus

    Peter Waher: am I right to assume that https://xmpp.org/extensions/xep-0322.html#defaultSchema should not be referencing 'xep-0322-01.xsd' but 'exi.xsd' instead (as that's the name used in the XSD listing)?

  151. Alex

    sorry guys I was stuck in traffic, will start the member meeting shortly

  152. Alex

    just need to update the agenda quickly

  153. Alex

    okay, anyone ready?

  154. Guus


  155. neox is here '-'

  156. nicola

    Here I am

  157. Alex


  158. Alex bangs the gavel

  159. Alex

    here is our agenda for today: https://wiki.xmpp.org/web/Meeting-Minutes-2023-03-09

  160. Alex

    1) Call for Quorum

  161. Alex

    as you can see 41 members voted via memberbot. So we have a quorum

  162. Peter Waher

    Guus: It should reference a snapshot directory with agreed-upon default versions of schemas. In the current version, the -01.xsd corresponds to the exi.xsd file. The extension takes into consideration the gradual evolution of extensions and corresponding schemas. But you’re correct: We can remove the -01 from the file name in the schemaLocation specification.

  163. Alex

    2) Items Subject to a Vote

  164. Alex

    new and returning members, you can see all the applications here: https://wiki.xmpp.org/web/Membership_Applications_Q1_2023

  165. Alex

    3) Opportunity for XSF Members to Vote in the Meeting

  166. Alex

    any XSF member here who has not voted yet and wants to do so now?

  167. Alex

    Memberbot is still online

  168. Alex

    looks like there is none

  169. Alex

    then I will shutdown memberbot and start counting

  170. Alex

    still counting ...

  171. Zash


  172. neox


  173. Alex

    getting there, final check ;-)

  174. Arne


  175. Alex

    4) Announcement of Voting Results

  176. Alex

    when you reload the page at: https://wiki.xmpp.org/web/Meeting-Minutes-2023-03-09 you can see the results

  177. Alex

    except of 1 candidate everyone got accepted

  178. Alex

    congrats to everyone, big list of candidates this quarter

  179. Seve

    Congratulations, I count on you when my reapplication time comes! Thank you Alex!

  180. Arne

    Great, thanks Alex!

  181. nicola

    Thank you very much @Alex and all. I am honored

  182. neox

    Congrats to every newcomers and reappliers 😊

  183. Alex

    welcome nicola

  184. neox

    nicola: welcome !

  185. Alex

    5) Any Other Business?

  186. Alex

    looks like thre is none

  187. Alex

    6) Formal Adjournment

  188. Alex

    I motion that we adjourn

  189. Alex

    someone needs to second

  190. Seve does

  191. neox

    Alex: I second '-'

  192. Alex gangs the gavel

  193. Alex

    thanks everyone

  194. neox

    Thank you Alex !

  195. nicola

    Thank you @Alex !!

  196. Arne

    Thanks everyone!

  197. moparisthebest

    Thanks all