jdev - 2024-12-25

I just saw that if you send the same media twice on Telegram, it basically doesn't reupload but reuse the same media, it seems that there's a simple hash done client side that allow reusing the previous one. Is there already XMPP clients that does that ? Basically it means that you can reuse the HTTP Upload GET URL several times during the upload, pretty simple but I was wondering about hypothetical negative impacts (security ?).

generally the way its done more privacy respecting is youll upload it again and the server deduplicates it on the backend and multiple urls point at the same file

idk if thats how xmpp servers do it per se but thats the general approach

it would be nice to have an xmpp client with msg fowarding though so you didnt have to reupload

this also doesnt work well in e2ee chats

Yeah public group would be nice but might break e2e

If it is the same user doing the upload, would the GET url not be that hash?

If its not the same server, then not needing to upload tells you something about others having already uploaded that bit of data. Could be a security/privacy issue.

> If it is the same user doing the upload, would the GET url not be that hash? As in: you can distribute the GET URL more than once, without needing to upload the data again anyways

> > If it is the same user doing the upload, would the GET url not be that hash? > As in: you can distribute the GET URL more than once, without needing to upload the data again anyways I don't really see a real privacy issue regarding that ↺

Basically my use case if if you would like to share the same picture to a few of your contacts at the same time, upload once and reuse the URL if you want to "reupload"

> Basically my use case if if you would like to share the same picture to a few of your contacts at the same time, upload once and reuse the URL if you want to "reupload" what the client could do to save this would to be locally save a map of file hash to file url and just reuse that ↺

no fetching from the server needed then

Omemo yes. Pgp may be not.

I was only wondering about a security issue when multiple difficulty people intending to send the same file would share the same URL / hash. I don't immediately see an issue with the same user reusing the same GET URL.

Alice encrypt a file to Bob and use the same url to send to Charlie. Poor Chalie can't decrypt and read the file i think

Im not sure how omemo work tho but i think pgp need to encrypt again ?

Trung, OMEMO isn’t used for XEP-0454, which is what you’re referring to.

edhelas: poezio has /embed for this

in principle any client could let you re-enter the oob url

> I just saw that if you send the same media twice on Telegram, it basically doesn't reupload but reuse the same media, it seems that there's a simple hash done client side that allow reusing the previous one. > Is there already XMPP clients that does that ? Basically it means that you can reuse the HTTP Upload GET URL several times during the upload, pretty simple but I was wondering about hypothetical negative impacts (security ?). I have all the client mechanics for this but to really work we need an extension to the http upload xep to send a hash along with the slot request so the server can send back only GET and no PUT to indicate you already uploaded this file before. If you don't just remember the GET client side then you won't know if it has already expired or similar and the URL might be dead. Sending hash in slot request the server can extend expiry as if uploading etc ↺

Yup

> Alice encrypt a file to Bob and use the same url to send to Charlie. Poor Chalie can't decrypt and read the file i think The hash would not match between encryption to different targets so that's no issue ↺

I think you can do everything fully retrocompatible and add the hash mechanism on top

Yes for sure you can

Add it as a child to the slot request. I almost just started adding it even though no server looks for it yet

is a content hash part of oob or sims? or is this an implementation-first xep-next scenario

sims yes

And content hash has its own special xep seperate from anything else that all other xeps reuse ✏

aha

but we all know the real question for today is how much beer can Santa encrypt down his tummy

We all should be notified if he resolves his own promise.

>> I just saw that if you send the same media twice on Telegram, it basically doesn't reupload but reuse the same media, it seems that there's a simple hash done client side that allow reusing the previous one. >> Is there already XMPP clients that does that ? Basically it means that you can reuse the HTTP Upload GET URL several times during the upload, pretty simple but I was wondering about hypothetical negative impacts (security ?). > I have all the client mechanics for this but to really work we need an extension to the http upload xep to send a hash along with the slot request so the server can send back only GET and no PUT to indicate you already uploaded this file before. If you don't just remember the GET client side then you won't know if it has already expired or similar and the URL might be dead. Sending hash in slot request the server can extend expiry as if uploading etc Why bother with all this, teaching the server hash mechanisms and all that jazz, instead of just "hey I uploaded this file before and would like the expiration time extended" ? ↺

You can send the same aesgcm:// encrypted link to unlimited contacts too

That's what sending the hash is for, to indicate what file this is

Why not make the hash the filename on server

That's an internal choice by the server sure

Server can use any filename it wants. Nothing even says the server needs to store the data as files per se

I mean on the url

just update the current spec for it to be http://.../$md5.mp4

URL can be anything the server wants. Fixing it to a specific format wouldn't help anyone

> That's what sending the hash is for, to indicate what file this is Why can't you just use the uploaded file's URL as an identifier? You could just tell the server "Make this URL available for longer". We are lacking the "Manage uploaded files" XEP anyway, that could specify: listing uploaded files including expiry information for then, deleting them and extending them. ✏

Clients might want to intentionally upload two files with the same hash, your concept would block that.

singpolyma, why teach the server about hashes when you can just send it the URL

yes, what larma said ^^^

I mean I guess you could. Then you have to teach the server about the URI which seems more complex for no gain to me, but I guess it would work out the same in the end

Yes, we need an XEP for managing uploaded files. I really want to permanently store some files, such as those used for avatars.

> Clients might want to intentionally upload two files with the same hash, your concept would block that. If you have two files with the same hash you need to use a different hash, heh ↺

singpolyma, unless your entire point is to demonstrate a hash collision

like, to your hacker friends

send them samples

Right but if you've found an example of that Very Bad Things are happening and we would be urgently upgrading everything

the server always has to know about the URI, because it has to serve the file there

Some web server needs to be able to decode it at request time, sure ✏

Yea, and that's what you need to talk to