XSF Discussion - 2018-06-20

labdsf, caps still work with routing-ng (or why shouldn't they) and you are still able to send messages to the full jid

labdsf, please close PRs which shouldn’t be merged

you can make a new one later

open PRs clutter the view

ok, closed

I can even reopen it

yupp

thank you d)

flow: i mean, when I send messages to bare JID, I cannot know which capabilities the receiving client has

labdsf, there are use-cases where you want to send to a single device, for example for IQ-based protocols and stuff like Jingle

So ok, I can simply send message to all full JIDs that I think have required capabilities

And attach no-copy hint

except that they’re not archived

Not archived by MAM is ok

awful

For ephemeral messages that is exactly what I want

I am strictly against introducing another reason why messages don’t appear on all devices.

OMEMO itself is bad enough already.

If I send to all full JIDs, then it will appear on all online devices

I think we need a better way to deliver messages to offline devices

We have it, it's MAM.

It does not deliver messages to full JID

labdsf, except that the device may be going offlnie exactly in that moment you send the message

and come back online a secodn later

without the user noticing

(badly timed SM resumption can cause this behaviour)

How about updating XEP 0160 to say that messages send with no-copy should be postponed and delivered to exactly that full jid, not some random with positive priority?

sent*

Or return service-unavailable at least

labdsf, also, what if: 1. you check device support 2. you send a message to all full JIDs you want to 3. your network is slow and the message is delayed 4. one of your target devices goes offline 5. another device from that user comes online which does *not* have the feature you were looking for, but uses the same resource 6. your message was delivered to the wrong device

jonasw: do not use the same resource

you can’t force your peer to not do that

using the resource for that kind of stuff is inherently racy because it’s an inherently temporary thing

I think using unique resources is the best practice

And remember that clients don't get to choose their resources, the server does.

Otherwise we cannot do anything but say that capabilities are totally unreliable

labdsf, that’s what we’ve been doing for a while now :)

for IQ based stuff it’s okay, and for things like jingle where there’s constant feedback, but using it for messaging is not a good idea

It's generally ok, but not foolproof.

Except that capabilities XEP mentions XHTML-IM in the introduction, which is messaging use case

labdsf, it’s also old

I would rather slowly improve offline message delivery then declare that nothing is reliable in XMPP and we should just send everything to bare JID with store hint and body and save it in MAM forever

labdsf, why not?

Because it is impossible to implement any extension that relies on client support on top of that

Lets say, replies, editing messages other than the last etc.

OMEMO has to put encryption keys for *all* devices except the one it is intended for

Adding body to OMEMO messages is a kludge also

There are two problems with full JIDs: 1. No way to get list of offline resources 2. No reliable delivery to offline resources

Both are fixable on server side

(1) That's what a presence subscription gives you (2) That's because there's no such thing as an offline resource.

Kev, how does presence subscription give you a list of *offline* resources?

Sorry, I read *online* resources. It also gives you a full list of offline resources - it's an empty list because they don't exist :)

heh

We do need to track offline clients for assorted reasons, but those are offline clients, not offline resources.

Ok, "offline resource" means a resource that offline client will use when it reconnect

There is no way to know what resource a client will use when it reconnects

A resource is a session identifier for an online session only

As much as it would be nice to imagine it is a device identifier, it is not - there is no standard for that, and it's not what most clients do

(yet)

I would like to change both of those things

If it randomizes resource every time, it should be fixed IMO, or don't expect to get messages other than those sent to bare JID

(I don’t expect messages except those sent to the bare JID.)

(and I don’t think we should send normal IM to anything *but* the bare JID.)

Agreed

labdsf, the right way to handle stuff like XHTML-Im etc. is to provide a sensible fallback for clients which do not support it

the protocol must be designed in such a way that legacy clients do the right thing

MattJ: agreed to me or jonasw?

jonasw, but I do agree that clients should not request a random resource for every session

I also agree that a client cannot assume it will get *every* message sent to a full JID, but that one is far less clear

Instead of MAM, server can list resources used by offline client for a week or so, with cached capabilities, and store offline messages sent to them for some time

and sending to full JID is, like random client-requested resources, something we now want to discourage

I already have most of an implementation of that, FWIW

But I still don't think that means what you think it means

Just because the server can track devices, doesn't necessarily mean they will always have the same resource

It's potentially something you don't want to leak to third parties

yet we do it as soon as a client replies with a message

We leak resources, we don't necessarily leak devices, because resources and devices aren't a 1:1 mapping over time.

true, I still wonder if we shouldn't try to address that by adding a way to send messages from the bare JID. although I fear that this would probably break some (receiving) clients

flow: or just introduce a virtual "proxy" resource maintained for you by the servet

server*

MattJ: what are the drawbacks of "leaking" a list of resources. Third party will know how many devices you have, anything else?

Not that I can immediately think of

It's not just how many, it's when you use certain devices also

e.g. someone can track you between work and home if you have a device in each location

People used to explicitly set resources like /Home and /Work (well, some clients defaulted to things like that also), but I think we know better than to share things like that by default in 2018

labdsf, the resource string is the only thing between a remote entity and your device. if a remote does know it, they can send stanzas which will very likely reach your device (compared to "probably reach your device"). For example, someone knowing the resource of your mobile could write a script which sends "silent" messages every minute or so, in an attempt to drain your battery without you noticing it

MattJ, did you or someone else publish something regarding the newsletter? I've been not able to catch up lately and I wonder if I missed something

sorry, not the newsletter, where do I have my braind today...

The survey

Seve/SouL, not yet

flow, itym "reach that specific device" instead of "reach your device"

because sending to the bare JID would normally cause the device to receive the message for sure at some point

(in the ideal IM-NG world)

Ok ok, no rush MattJ, just worried about myself missing stuff :)

jonasw, yes, if we did im-ng, but I deliberatly did not take it into condsideration because there are still many open questions

flow, help to solve them :)

jonasw, if it only where so easy ;)

hmm "if it only would be that easy"

flow, what if someone sents silent messages to bare JID?

MattJ, where can I find your implementation of offline message delivery?

labdsf: on my laptop

so it is not in https://hg.prosody.im/trunk/ yet?

No

It will probably be in prosody-modules first

Too much of this is new and experimental

is there any description of it, maybe in ML?

But I am writing it with the full intention of it replacing the current code we have

No, and to be honest I have mostly avoided talking about it until recently

There is not really anything to discuss

For the initial implementation it will need no protocol changes, and no client changes

but still, you will keep disconnected resources around, they will be seen in presence and it will be possible to send messages to them, right?

No, as I said earlier a resource is not a device

I am not trying to change that

Prosody will track devices, but that does not necessarily mean sessions will have predictable resource strings

if some client uses the same resource string on every connection, and no other client uses the same resource string, the message will be delivered to that device only?

Send a message to a full JID, use no-copy and Prosody will deliver to that device only

and if there is no no-copy, it is delivered to some other resource, as per https://xmpp.org/extensions/xep-0160.html ?

As I wrote, I am not trying to change any protocols or behaviour beyond what should reasonably be expected

If the device is online when you send the message, it is not an offline message

It will go to that device

If you send to a resource that is not online, core routing rules apply as always, it will be redirected

If all devices are offline it's an offline message

what happens if there is some device online, message has no no-copy and resource it is sent to 1) was used before 2) was never used before

and what if there is no-copy and resource was never used before, message is lost?

You're over-thinking it... the answer to all the questions is essentially: "exactly the same as happens now"

Whether a resource was used before is completely irrelevant - resources are temporary, and they belong to a session

Prosody is not tracking resources, it is not making resources persistent in any way

Prosody is tracking devices. Devices != resources

how does it track devices, besides the resource string they request on connection?

XEP-0198 session token is another way, for example

A client could try to resume a session, fail (because the session timed out), and then it goes on to bind a completely random resource

Same device, different resource

Except not, because the device was killed by the OS and all that was lost

the device can try to save SM id in some persistent database

Zash, tangential to the devices != resources point :)

Possibly

labdsf, I would eventually like bind2 or some alternative to be implemented, and allow the client to unambigously send a unique id

But as I mentioned, my current scope involves no protocol changes

After it's done, bind2/something will become more of a priority

saving the id only wont help you. you'd have to save the entire state

Yeah

but on mobile the push target is another nice id for the device

clients will enable that in every session

Ah, good point, thanks :)

but maybe too late depending on what you are trying to do

No, it's easily extended to that

Though I would like the push target to become part of bind as well

Kinda "this is another way to reach me" metadata associated with the session

MattJ, how is it different from mod_smacks_offline?

it just places message for offline delivery, and you try to deliver it to the correct device instead?

mod_smacks_offline is not ideal, it doesn't work with multiple resources online

it just delivers to the resource with highest priority?

If deviceA and deviceB are both online, if deviceB goes offline and does not resume, and there is an unacked message in the queue, there is a question about what to do

It can't save it to the offline message store, because there is another device still online

it could send it to that other device

but that device possibly already received it

via bare-JID forking or through carbons

etc.

so you end up with duplicated messages or bounced messages. In the case of a single resource going offline, with no other resources, it's simple to just save it to the offline store

Next resource to log in receives it

ok, current default is that it is just lost

and what will you do instead with the new code?

Current default (mod_smacks) is to send an error reply to the sender

Which works fine, and is the safe option

...except when the sender has gone offline

or ignore all of that and Just Use MAM™

labdsf, new code tracks devices and what stanzas/messages they need to receive

which means the mod_smacks problem is solved, because the code can focus on delivery to a single device, and not worry about whether other resources are online/offline/etc.

so at the time of forking/carbons/etc. that is when it gets decided what device receives what stanza

Instead of 5-15min later "Oops, that failed to deliver, now where should we send it?"

MattJ, so what will be different, you will try to identify deviceB even after timeout?