XSF Discussion - 2020-01-05

> on a more general point of view, I don't see the point of having this configuration field mandatory or set at all by default. We don't have such thing for mam. I'm also curious if something like this can be done.

edhelas: You can discover node configuration defaults like this: https://xmpp.org/extensions/xep-0060.html#owner-default

Why don't we use XEP-0122 ranges to advertise the accepted range? I.e. <field type="text-single" var="pubsub#max_items" label="Max # of items to persist"> <validate xmlns="http://jabber.org/protocol/xdata-validate" datatype="xs:integer"> <range min="0" max="255"/> </validate> <value>1</value> </field>

hm it does not matter what the XEP says, it will never be able to prevent that clients change that settings

and i thought we have fixed it already

there is now a setting for unlimited which was not previously

so now that there is "unlimited" there is no need for clients to set it anymore, except they want it limited intentionally like for PEP nodes

or even better they set it all to unlimited in their publish options ✏

its not clear to me how a client that is like movim and allows to publish microblog content would set the node to max_items=1

that must be clearly a bug as it makes no sense

the max_items is a per node setting

so a client that does not support microblogging will *never* change anything on the microblogging node

it does not matter if it sets max_items for all other nodes, like omemo, tune, activity etc

so i dont see a problem their

in the microblogging case there is also no need for discovery of max_items, the only thing the server must support is max_items="max"

and if it supports it you will see if you put it in the publish options and it does not fail

Agree, that all makes sense

max_items is very useful in some cases

Not usually in microblogging, sure

previously without "max" setting for max_items, it was a problem, because on node creation you had to make sure as client the node exists with good defaults, so you put in some number for max_items, of course other clients needed to do this also, but they didnt put in the same number

and you didnt even know what number to put their, what the server supports and what not

> its not clear to me how a client that is like movim and allows to publish microblog content would set the node to max_items=1 > that must be clearly a bug as it makes no sense I don't think Movim ever did that. There was a gajim bug though at some point that would reset microblog's max_items to 1 :p

Also "max" doesn't seem to get much love from people from what I can see

A rushed hack IMO

I think the fastening / mam summary debate brings an inner conflict between smart servers and dumb message routing servers to the surface that has been boiling for years now. It's super hard (I'm afraid impossible) to have smart servers in a generic form

If you agree there's a need for such a feature maybe that can be fixed

Zash ^

(It's not like anybody was using it already)

how is this a rushed hack?

i dont see another solution, and i heard not of any argument why max_items=max does not what it intends to do

pep., why would you say it gets no love from people?

because its not implemented instantly?

its only needed for very few cases right now in xmpp

bookmarks2 and mircoblogging

thats it

lovetox, the main complaint we have is shoving a text string into a field that was previously an integer

ok but that has nothing to do with the intention of the feature

Sure, no, I agree with the intent

yeah -1 would maybe be better

but i have no experience how much problems that generate on the server side if its now a string/int

i know at client level i dont care

hm actually ...

It depends on the server

what if we put that into a form like a node config

but we only have text fields there

In previous versions of Prosody it would have been ok, we had custom validation code everywhere and we could have tweaked that to be even more custom

there are no int / number fields

so it does not care either

But we cleaned up the custom stuff and implemented standard validation of fields where it made sense

But there is no standard data type for "sometime int, sometimes a string 'max'"

no but its always a string isnt it

you have to take the string, and convert it to an int

and this works or it doesnt

Correct, so it must be something convertible to an int

and if it doesnt, you test if its "max"

Sure, as I said, we can litter the code with custom hacks like that

But we try not to

yeah i see how a generic validation code cant handle that

its not even int/string is allowed

its int and one single string is allowed

and I'll bet this isn't the only option where this feature makes sense

It's just the one that is being focused on right now

the proposal from Daniel was actually some unicode char, then this was discussed and it was changed to max

i dont know if -1 came up in the discussion

-1 would have been fine with me from an implementation perspective

From a protocol perspective it would probably be cleaner to define a type xmpp:integer-or-max or something at https://xmpp.org/registrar/xdv-datatypes.html

which can still be done

But I have been working on other things

Also I don't think this allows the client to actually discover the limit, does it? Zash's proposal does

Even if the client can't do anything about the server limit, it might be good to inform the user that posting a new post will remove the old one

We might also want an option to disable that behaviour (reject the new item if the node is full instead)

yeah the form thing above looks nice

That forces the client to fetch the config though (kinda moot for publish-options no?)

Yeah, maybe

integer-or-max (throw in "min" too for good measure) would be fine

also, limits could be advertised somewhere, like the limit on http upload size

in disco?

A form in disco#info at your account seems appropriate

i would expect the server to reject it

and not silently start deleting stuff

That means PEP will be permanently broken :)

unless you issue a delete before every publish

tune change == delete old tune, publish new tune

no, it should only reject new items

it should allow overwriting existing

thats why everyone should use a SingletonNode

so max_items should actually mean, max different ids

so a server rejecting all new ids except the existing one

I'd be happy with that

There are still many use-cases where deleting the oldest item does make sense

But I think it needs to be configurable

is this written in the xep that the server has to delete old items when max_items=1?

or is it not defined

> Note: If the service or node is configured so that there is a maximum number of items cached at the node and the maximum is reached when an item is published, the service MUST delete one of the existing items. It is RECOMMENDED for the service to follow the "first in, first out" rule and delete the oldest item. Depending on node configuration, deletion of an existing item MAY result in sending of a delete notification to the subscribers.

https://xmpp.org/extensions/xep-0060.html#publisher-publish-success

ok damn ..

So we COULD delete a random item :D

We're using a publish-subscribe mechanism for storage. It's weird.

Zash, it's more a key->value database with change notifications

We've turned it into that, yes.

XEP-0060 made sense when I realised that

No, I think it always has been

find it weird that FIFO behavior is the default case in pubsub

At least since 2003

would expect this as a special config of the node

I agree that it should be, especially for all the things we are using it for today

Still, storing things at all is optional

Quietly deleting user data is not ideal

i would argue for a microblogging service this is a no-go

even on "max" server would delete user data once it reaches the max

Yep

so there is actually no possibility right now in pubsub to retain userdata

so its a very bad storage

hm does MIX not use pubsub as storage?

nothing prevents you from implementing pubsub/pep that keeps everything forever

same with MAM

Zash you mean making a server that has no max

yes