jdev - 2020-04-28


  1. lovetox

    they dont need to be closed

  2. lovetox

    they are restarted

  3. marc0s

    ah, that was the concept, "restart" https://xmpp.org/rfcs/rfc6120.html#streams-negotiation-restart I was probably falling asleep already

  4. marc0s

    thanks 🙂

  5. lovetox

    but this is mentioned again under https://tools.ietf.org/html/rfc6120#section-5.4.3.3

  6. lovetox

    explicitly client MUST not send </close>

  7. lovetox

    explicitly client MUST not send </stream>

  8. Zash

    How do other servers behave is there's some database error when adding archiving a message? (context: MAM)

  9. flow

    Zash: should be configurable and default to retry-panic

  10. flow

    question is, what the 'ignore' behavior should be? send once message (per day?) to the user stating that his messages are not archived?

  11. Ge0rG

    send a message error?

  12. flow

    question is, what the 'ignore' behavior should be? send one message (per day?) to the user stating that his messages are not archived?

  13. Ge0rG

    panic = kill the server?

  14. Zash

    I didn't ask what they should do. I asked what they *do*.

  15. flow

    good question if the message type should be error, but probably the answer is yes

  16. flow

    as long as there is a <body/>, and not just some error code that is often not shown to the user

  17. flow

    Zash, I know. But I expect some (most?) devs don't know, as error code paths are often not well tested

  18. flow

    or even not well known to the devs

  19. flow

    hmm another nice settings to have would probably be retry-notify-admin-ignore

  20. Zash

    I know what Prosody does as I've written a bunch of that code. No idea what ejabberd or Openfire or others do.

  21. flow

    Holger, any idea about ejabberd?

  22. flow

    Guus, any idea about openfire?

  23. Guus

    Unsure. Probably silently ignore after writing the error to the logfile

  24. Holger

    So it's about a DB error on writing the MAM message? Then it's the same for ejabberd.

  25. Zash

    And Prosody too, ignore and carry on trying to deliver the message.

  26. Zash

    and write something to the error log

  27. flow

    Holger, DB error is the DB returning an error and/or the DB connection lost?

  28. Holger

    Both (if re-establishing the connection fails).

  29. lovetox

    i find it weird that you want to inform a user if something database wise happens on the server

  30. lovetox

    why not send a message to the admin?

  31. lovetox

    User cant do anything about it, but informing the administrator

  32. flow

    well the users UX is degraded, doesn't strike me as the badest idea to inform him

  33. flow

    well the user's experience is degraded, doesn't strike me as the badest idea to inform him

  34. flow

    so that at least he knows that there is a reason his other clients do not show the same messages as he is used too

  35. lovetox

    A Adminstrator can also inform the users?

  36. flow

    note that I am talking about MAM archival not working, for whatever reason

  37. lovetox

    is this not his job if the service is degraded?

  38. flow

    lovetox, sure, that is why is also considered retry-notify-admin-ignore as setting

  39. Holger

    I would not send messages to users they can do nothing about other than notifying the admin.

  40. flow

    often database errors are simply returned to the requesting entity. for example, think if pubsub publish: if the item could not be persistet, then a service would probably return an IQ error

  41. Holger

    I think the options are "just notify the admin" vs. "bounce the message with an error".

  42. flow

    Holger, bounce the message to whom?

  43. Holger

    The sender.

  44. flow

    but the message stil gets delivered?

  45. flow

    why should the sender care that the recipients MAM archive does not work?

  46. Zash

    I think this all depends on whether you want to have a store-and-forward system or not.

  47. jonas’

    bouncing an error to the sender if the message got delivered is going to be confusing

  48. Holger

    flow, well the question is whether MAM is considered critical or not.

  49. jonas’

    although, an excellent use-case of @type='continue'

  50. Zash

    I've assumed that delivery is the priority.

  51. jonas’

    but still, the sender can use that info even less

  52. pep.

    also careful with notifying the admin on a busy server. don't do that for every message :p

  53. flow

    Zash, the alternative to store-and-forward is potentially-store-and-forward?

  54. flow

    If so, I lean towards potentially-store-and-forward

  55. lovetox

    for all server errors, inform admin, admin can decide if he informs his users

  56. jonas’

    elitistly, I may say: logging as error should be enough. You should have monitoring which picks up on that.

  57. Holger

    jonas’, if a message is not delivered the sender does need an error message to be aware of the issue.

  58. jonas’

    but all those log monitoring stacks are insane

  59. jonas’

    Holger, 100% agree

  60. jonas’

    I’m reading backlog and replying ;)

  61. flow

    jonas’, true, but it's kinda right there to have your xmpp server notify the admin about issues ;)

  62. Zash

    Users (other devices) do get a hint that messages aren't stored, at least Prosody only adds the stanza-id if it successfully archived the message, which shows up in carbons.

  63. jonas’

    flow, sure

  64. jonas’

    mod_error_to_message which hooks into logging and rate-limitedly forwards error log messages to the admin?

  65. jonas’

    ideally rate-limited by type

  66. jonas’

    ideally rate-limited by message template

  67. jonas’

    "error: %s (and %d more instances)" | format("foo %s bar %d baz" | format(original_logging_arguments...), number_of_logentries_in_ratelimit_bucket)

  68. flow

    I see why service with thousands of users do not like the users to be automatically notified about a MAM outage, but I could imagine I would run my personal family service with such a setting enabled

  69. flow

    I see why a service with thousands of users may not be keen on users to be automatically notified about a MAM outage, but I could imagine I would run my personal family service with such a setting enabled

  70. jonas’

    do servers still assign a stanza-id on inbound messages if archiving fails?

  71. jonas’

    if not, a client could detect that passively and display an inline warning about that

  72. flow

    i'd assume that you can not assign an archive ID if your archive is unavaible

  73. jonas’

    it’s perfectly possible

  74. flow

    well unless you do the ejabberd thingy

  75. jonas’

    if you uuidgen all your IDs, you can do that before storing and only later realize that you did it

  76. jonas’

    I think '313 should specifically forbid that; it’ll cause pain once the archive service is restored

  77. flow

    it is still kind of odd that you assign an archive ID without being sure that this message will ever hit the archive

  78. Holger

    flow: But you're just adding a `<stanza-id/>` which you may well want to do independently of MAM, no?

  79. Holger

    flow: Didn't the XEP-0359 author consciously make this an independent XEP?

  80. jonas’

    uff, but then you’re in fun trouble

  81. jonas’

    because a client may have to do a full archive download because MAM will tell it that it doesn’t know that ID

  82. Zash

    maybe set the hostname as issuer of the id instead of the account?

  83. Holger

    ejabberd doesn't add a `<stanza-id/>` if the message wasn't archived, but I would've thought doing so would be perfectly fine. And I considered doing so in the past as some client dev(s) had some non-MAM related use case.

  84. flow

    Holger, true, but in this case the 'by' attribute of <stanza-id/> is typically known to be an address of an archive too

  85. jonas’

    cc @ MattJ

  86. flow

    but yes, there is nothing in xep359 which would prevent you from adding a stanza-id even if the archive is currentlyd own

  87. flow

    and I would not want it any other way

  88. Holger

    If that's not fine I don't see why 0359 would be a separate XEP at all. Given that 0313 hard-depends on it.

  89. jonas’

    Holger, there are other use-cases for stanza-id besides MAM, aren’t there?

  90. Holger

    So if clients need to know whether a message was archived we need a separate flag besides the <stanza-id/>.

  91. flow

    yeah, there is also the possibility that an alternative to MAM emerges that could also re-use xep359

  92. flow

    Holger, or, maybe better, a flat that the message was not archived

  93. flow

    Holger, or, maybe better, a flag that the message was not archived

  94. Holger

    Or both.

  95. flow

    is there an advantage in having both? seems unnecessary to me, compared to simply flag the unexpected/unusual case

  96. jonas’

    flow, both allows a client to passively detect the support for setting the flags on the server

  97. flow

    i mean, we could have a stream feature that this "i-will-archive-messages-unless-flaged"

  98. jonas’

    no flag present -> no support; one flag present -> support && clear statement; both flags present -> ????

  99. Holger

    The client can't deduce a message was archived from the absence of a wasn't-archived flag, no? Except we add another feature.

  100. Holger

    That.

  101. flow

    i mean, we could have a stream feature that says "i-will-archive-messages-unless-flaged"

  102. Holger

    Yes that would work as well.

  103. MattJ

    History: XEP-0313 originally had <archived id='...'/> and that certainly conveyed meaning that we have lost with the current protocol

  104. MattJ

    I have considered bringing it back (we really have to make the decision now)

  105. Kev

    I can't commit any brainpower to this today, but if the question is "do clients need to be told that a stanza was archived, and by whom", I think the answer is 'yes'.

  106. Zash

    What about the inverse?

  107. MattJ

    What's the inverse?

  108. Zash

    Do clients need to be told about failure to archive?

  109. Zash

    Kev, what does M-Link do if archiving fails?

  110. MattJ

    Part of the reason I was hesitant, is because with the IM-NG approach, I'm not sure there is much need for the additional signalling

  111. jonas’

    MattJ, signalling is useful in case of errors

  112. jonas’

    like the other day on yaxim

  113. MattJ

    It's questionable what should happen in that case

  114. jonas’

    if a client knows that a stanza was, contrary to expectations, not archived, then you can avoid querying that stanza-id on next connect, leading to fun issues

  115. MattJ

    Should the stanza even be delivered? :)

  116. Ge0rG

    Great, let's re-add `<archived id='...'/>` into MAM and kill 0359.

  117. jonas’

    that’s a good question and should probably be configurable, but orthogonal, I think.

  118. MattJ

    Pre-MAM, failure to store an offline message would generate an error bounce

  119. jonas’

    if you ask me, the server should assign IDs and queue messages for archival and try it’s best to do it later

  120. MattJ

    MAM is basically per-device offline, and what we're saying is that partial delivery (i.e. only to currently online devices) counts as delivery, so it's ok

  121. Ge0rG

    what if the message was delivered to one of three clients, stored in offline store, but not MAM-archived?

  122. jonas’

    while havnig the message delivered with a flag indicating that it may not have been archived

  123. jonas’

    but that’s expensive to do, resource wise

  124. Ge0rG

    jonas’: your suggestion would lead to an OOM crash with all the messages getting lost eventually

  125. Zash

    What if you can't archive the archiving failure notification?!!??!!!!?‽!!?

  126. Ge0rG

    Zash: what if you generate an archiving failure notification for each archiving failure notification?

  127. Zash

    🤯️