XSF Discussion - 2020-01-05

  1. pep.

    > on a more general point of view, I don't see the point of having this configuration field mandatory or set at all by default. We don't have such thing for mam. I'm also curious if something like this can be done.

  2. Zash

    edhelas: You can discover node configuration defaults like this: https://xmpp.org/extensions/xep-0060.html#owner-default

  3. Zash

    Why don't we use XEP-0122 ranges to advertise the accepted range? I.e. <field type="text-single" var="pubsub#max_items" label="Max # of items to persist"> <validate xmlns="http://jabber.org/protocol/xdata-validate" datatype="xs:integer"> <range min="0" max="255"/> </validate> <value>1</value> </field>

  4. lovetox

    hm it does not matter what the XEP says, it will never be able to prevent that clients change that settings

  5. lovetox

    and i thought we have fixed it already

  6. lovetox

    there is now a setting for unlimited which was not previously

  7. lovetox

    so now that there is "unlimited" there is no need for clients to set it anymore, except they want it limited intentionally like for PEP nodes

  8. lovetox

    or even better they set it all to unlimited in there publish options

  9. lovetox

    or even better they set it all to unlimited in their publish options

  10. lovetox

    its not clear to me how a client that is like movim and allows to publish microblog content would set the node to max_items=1

  11. lovetox

    that must be clearly a bug as it makes no sense

  12. lovetox

    the max_items is a per node setting

  13. lovetox

    so a client that does not support microblogging will *never* change anything on the microblogging node

  14. lovetox

    it does not matter if it sets max_items for all other nodes, like omemo, tune, activity etc

  15. lovetox

    so i dont see a problem their

  16. lovetox

    in the microblogging case there is also no need for discovery of max_items, the only thing the server must support is max_items="max"

  17. lovetox

    and if it supports it you will see if you put it in the publish options and it does not fail

  18. MattJ

    Agree, that all makes sense

  19. MattJ

    max_items is very useful in some cases

  20. MattJ

    Not usually in microblogging, sure

  21. lovetox

    previously without "max" setting for max_items, it was a problem, because on node creation you had to make sure as client the node exists with good defaults, so you put in some number for max_items, of course other clients needed to do this also, but they didnt put in the same number

  22. lovetox

    and you didnt even know what number to put their, what the server supports and what not

  23. pep.

    > its not clear to me how a client that is like movim and allows to publish microblog content would set the node to max_items=1 > that must be clearly a bug as it makes no sense I don't think Movim ever did that. There was a gajim bug though at some point that would reset microblog's max_items to 1 :p

  24. pep.

    Also "max" doesn't seem to get much love from people from what I can see

  25. Zash

    A rushed hack IMO

  26. Daniel

    I think the fastening / mam summary debate brings an inner conflict between smart servers and dumb message routing servers to the surface that has been boiling for years now. It's super hard (I'm afraid impossible) to have smart servers in a generic form

  27. pep.

    If you agree there's a need for such a feature maybe that can be fixed

  28. pep.

    Zash ^

  29. pep.

    (It's not like anybody was using it already)

  30. lovetox

    how is this a rushed hack?

  31. lovetox

    i dont see another solution, and i heard not of any argument why max_items=max does not what it intends to do

  32. lovetox

    pep., why would you say it gets no love from people?

  33. lovetox

    because its not implemented instantly?

  34. lovetox

    its only needed for very few cases right now in xmpp

  35. lovetox

    bookmarks2 and mircoblogging

  36. lovetox

    thats it

  37. MattJ

    lovetox, the main complaint we have is shoving a text string into a field that was previously an integer

  38. lovetox

    ok but that has nothing to do with the intention of the feature

  39. MattJ

    Sure, no, I agree with the intent

  40. lovetox

    yeah -1 would maybe be better

  41. lovetox

    but i have no experience how much problems that generate on the server side if its now a string/int

  42. lovetox

    i know at client level i dont care

  43. lovetox

    hm actually ...

  44. MattJ

    It depends on the server

  45. lovetox

    what if we put that into a form like a node config

  46. lovetox

    but we only have text fields there

  47. MattJ

    In previous versions of Prosody it would have been ok, we had custom validation code everywhere and we could have tweaked that to be even more custom

  48. lovetox

    there are no int / number fields

  49. lovetox

    so it does not care either

  50. MattJ

    But we cleaned up the custom stuff and implemented standard validation of fields where it made sense

  51. MattJ

    But there is no standard data type for "sometime int, sometimes a string 'max'"

  52. lovetox

    no but its always a string isnt it

  53. lovetox

    you have to take the string, and convert it to an int

  54. lovetox

    and this works or it doesnt

  55. MattJ

    Correct, so it must be something convertible to an int

  56. lovetox

    and if it doesnt, you test if its "max"

  57. MattJ

    Sure, as I said, we can litter the code with custom hacks like that

  58. MattJ

    But we try not to

  59. lovetox

    yeah i see how a generic validation code cant handle that

  60. lovetox

    its not even int/string is allowed

  61. lovetox

    its int and one single string is allowed

  62. MattJ

    and I'll bet this isn't the only option where this feature makes sense

  63. MattJ

    It's just the one that is being focused on right now

  64. lovetox

    the proposal from Daniel was actually some unicode char, then this was discussed and it was changed to max

  65. lovetox

    i dont know if -1 came up in the discussion

  66. MattJ

    -1 would have been fine with me from an implementation perspective

  67. MattJ

    From a protocol perspective it would probably be cleaner to define a type xmpp:integer-or-max or something at https://xmpp.org/registrar/xdv-datatypes.html

  68. MattJ

    which can still be done

  69. MattJ

    But I have been working on other things

  70. MattJ

    Also I don't think this allows the client to actually discover the limit, does it? Zash's proposal does

  71. MattJ

    Even if the client can't do anything about the server limit, it might be good to inform the user that posting a new post will remove the old one

  72. MattJ

    We might also want an option to disable that behaviour (reject the new item if the node is full instead)

  73. pep.

    yeah the form thing above looks nice

  74. pep.

    That forces the client to fetch the config though (kinda moot for publish-options no?)

  75. MattJ

    Yeah, maybe

  76. Zash

    integer-or-max (throw in "min" too for good measure) would be fine

  77. Zash

    also, limits could be advertised somewhere, like the limit on http upload size

  78. pep.

    in disco?

  79. Zash

    A form in disco#info at your account seems appropriate

  80. lovetox

    i would expect the server to reject it

  81. lovetox

    and not silently start deleting stuff

  82. MattJ

    That means PEP will be permanently broken :)

  83. MattJ

    unless you issue a delete before every publish

  84. MattJ

    tune change == delete old tune, publish new tune

  85. lovetox

    no, it should only reject new items

  86. lovetox

    it should allow overwriting existing

  87. lovetox

    thats why everyone should use a SingletonNode

  88. lovetox

    so max_items should actually mean, max different ids

  89. lovetox

    so a server rejecting all new ids except the existing one

  90. MattJ

    I'd be happy with that

  91. MattJ

    There are still many use-cases where deleting the oldest item does make sense

  92. MattJ

    But I think it needs to be configurable

  93. lovetox

    is this written in the xep that the server has to delete old items when max_items=1?

  94. lovetox

    or is it not defined

  95. MattJ

    > Note: If the service or node is configured so that there is a maximum number of items cached at the node and the maximum is reached when an item is published, the service MUST delete one of the existing items. It is RECOMMENDED for the service to follow the "first in, first out" rule and delete the oldest item. Depending on node configuration, deletion of an existing item MAY result in sending of a delete notification to the subscribers.

  96. MattJ


  97. lovetox

    ok damn ..

  98. MattJ

    So we COULD delete a random item :D

  99. Zash

    We're using a publish-subscribe mechanism for storage. It's weird.

  100. MattJ

    Zash, it's more a key->value database with change notifications

  101. Zash

    We've turned it into that, yes.

  102. MattJ

    XEP-0060 made sense when I realised that

  103. MattJ

    No, I think it always has been

  104. lovetox

    find it weird that FIFO behavior is the default case in pubsub

  105. MattJ

    At least since 2003

  106. lovetox

    would expect this as a special config of the node

  107. MattJ

    I agree that it should be, especially for all the things we are using it for today

  108. Zash

    Still, storing things at all is optional

  109. MattJ

    Quietly deleting user data is not ideal

  110. lovetox

    i would argue for a microblogging service this is a no-go

  111. lovetox

    even on "max" server would delete user data once it reaches the max

  112. MattJ


  113. lovetox

    so there is actually no possibility right now in pubsub to retain userdata

  114. lovetox

    so its a very bad storage

  115. lovetox

    hm does MIX not use pubsub as storage?

  116. Zash

    nothing prevents you from implementing pubsub/pep that keeps everything forever

  117. Zash

    same with MAM

  118. lovetox

    Zash you mean making a server that has no max

  119. lovetox