XSF Discussion - 2018-06-27

Sam is bringing up a point. How do you protect a web service from files uploaded by users?

context?

I upload a html+js file via http-upload, share the link with you, it's executed in your browser, I get your webchat / account login session cookie

right, this should probably be in the security considerations

but I don’t see where he brings that up

the c.im cookie is only valid for account.c.im, so this specific issue doesn't apply

standards, three days ago, re Council Minutes 2018-06-20

(the solutions I’ve seen so far is to (a) be careful with the domain you choose for your upload service and (b) use Content-Disposition: attachment with anything which may contain executable code)

sure, I see that mail, but not what you’re saying in it :)

maybe I extended the focus of the question slightly. the general question was about "executable content"

whatever that is.

right

there is nothing about content-disposition in the XEP

somehow I didn’t get at all what that bullet point was trying to say

but what you interpret into it kinda makes sense

this is actually something I didn't have on my agenda on the last LC.

or maybe not, I’m not sure. I think when I first read it I thought something along the lines of "okay, a client really should not download and run an .exe, but that’s obvious"

I think those are mostly operational concerns which will probably evolve with how the web treats content in the future.

(nothing specific in mind)

they’re also not directly related to XMPP

so I would probably not block based on that

they also don’t require a namespace bump or otherwise interaction with the entities implementing the protocol, AFAICT

yes

we are only giving server operators a huge, back-facing gun.

nothing wrong with that.

that’s not what I wanted to s ay

I wanted to say that it should be expected that we have to modify this in the future anyways, so modifying it right when it goes to draft should be ok too (regarding your "+1 but").

yeah, also don't want to be an asshole for security's sake. I played that card last time already.

this should be fixed ASAP, but blocking advancement based on that might not help with that

(something something demotivating authors something)

yeah, that.

Hi, nobody answered my remarks on expiration date for HTTP upload, does anybody care? It seems a major point to me.

goffi, I can’t recall your feedback. I think it would help the discussion if you would provide a link to your feedback.

jonasw: https://mail.jabber.org/pipermail/standards/2018-June/035181.html

goffi, I now recall that email. I do remember that you wrote it, but I also remember not seeing this as particularly important

you don't think expiration date is important ?

maybe you should clarify on the mailing list why you think this is particularly important

we upload file on a server, we don't know for how long, and we have no access to it then.

I’m not sure it’s extremely important to have this on the protocol level, it should be included in the Privacy Policy for sure. ✏

jonasw: OK I'll try to explain more, thanks for feedback, I had the feeling to be ignored which is quite frustrating as it is a major point to me (even if it's to say that it's not that important for whatever reason).

sure

I fully understand that

to me, it really did not come across to me that you thought that this is an important point, mostly because it lacked any rationale

(typically when I try to raise a point I think is important, I’ll put a rationale on it *why* I think its important.)

goffi: in parts that was due to me being extremely busy with work these days. And in parts I was taking the easy way out and was waiting for someone to write a me too or something

daniel, FWIW, I think this is a good idea and supplements the work I was aiming to do with ToS.

I also think that this can easily be added later because it’s a data form \o/

yeah i’m ok with just putting it in there as an optional data form field

but I don’t see a technical use case for it because it won’t be reliable anyway

i see it as a pure TOS informational thing

maybe

jonasw: indeed, I've made a new message. daniel: OK, thanks to come in the discussion here, I've sent a new message to explain. I know it's not reliable, but it's informational, and there is a relation of trust with a server anyway (if I can trust retention date, I can't trust anything). It's the same with ToS.

Did you just add data forms to http upload?

the think with putting it in the HTTP Upload XEP is to make it mandatory. If it's not, nobody will set the data. Also the HTTP Upoad Component probably knows when the files are deleted so it can be set automatically.

Ge0rG, they were always there

Ge0rG, in disco#info

goffi, i see and agree with most of your points. and don’t think the suggested solution is a very good one or a solution at all

Ah

if anything we would need the client to mark a file as permanent (for avatars or blogs posts) or shortlived

and maybe a deletion command

daniel, +1

just putting the information in there on how long a server might store it doesn't do anything

or very very little

i mean what i'm a supposed to do? reupload the avatar every 30 days?

having a way to specify retention time in the request would be better, my email is not discarding this option.

Permanent storage requirements will break my GDPR compliance script... 🤔

Or we need to have multiple upload components for temporary and permanent storage

ew

daniel: is is possible to write your comments on standard@ even in a short email, so we can have other people input?

I imagine it's generally down to local policy.

Ge0rG, your component could implement this trivially by putting permanant files in a different directory (and reflecting that in the URL)

But avatars are an exception as they are 'always' permanent.

jonasw, yeah. the problem with that is that you still need some policy on the server that prevents stupid clients from always using the permanent store

and cluttering your file system

Do you?

I mean, you can always have usage limits.

daniel, quotas? and explicit request for permanent storage so that stupid clients need to be actively stupid at least

then it blows up in their face for reaching quotas and you’re done :)

quotas can be really small for permanent storage on IM deployments, 10 MiB or something

that’ll be reached extremely quickly

jonasw, yes and yes. but it makes implementations a bit more complicated then 'just put it in a different folder' is what i'm saying

(when abused for sending gifs to MUCs)

daniel, you need to have some type of quota anyways

right. but then you need different quotas

that’s true

i'm not saying it can’t be done. but right now most http upload implementations are very very simple

indeed

and I think if one went around and started abusing that, many servers would fail very quickly with ENOSPC

this sounds like a quick escalation from "file sharing" to "cloud storage provider"

yes

especially if you start doing delete

and list files

just implement WebDAV with XMPP auth

if you want to provide permanent storage, you need both listing and deletion. and quotas

actually, that doesn’t sound *too* bad for permanent HTTP-based storage.

that’s why i see a general desire to have those features but am hesitent to put them in the xep

that looks like the component I'm doing with Jingle.

jonasw: but custom http headers for obscure aws servers!11!

you’ll probably find a WebDAV library and servers which support WebDAV. just make the client pass some token in the HTTP Auth header and done.

in this case why not keeping HTTP Upload for simple temporary storage, and specifying date of expiration, and having more elaborated components for stuff like avatar or blogs ?

goffi, this sounds very reasonable

and explicitly forbidding permanent storage with HTTP Upload

I wouldn’t go that far

what’s the use case for the expiration then?

*the expiration date

if we want to keep the implementation simple, which is the main interest of this XEP I think, permanent storage doesn't seem an option.

daniel: knowing when the file is deleted, how long we can still use the link.

I'm not confortable in uploading a picture if I don't know how long it will stay

I can encrypt, but I then have the feeling to waster resources.

apperently you are comfortable with sending messages without knowing how long they will stay in archive

that's what the TOS are for…

I know how long they stay

on my server at least

How?

you also know how long files are stored on your server ;-)

right, but I have no MAM at the moment, so my server is not keeping anything. But the same question is valid for MAM I've never said the opposite.

yeah i’m not sure that every feature in xmpp that has the ability to store something needs to signal it's own retention period

and about resource, it's not the same about a couple of message, and one or several MB files

my vcard service never report how long it will store that either

goffi, not your department; the server is responsible for managing the resources it offers to you.

it must not rely on your goodwill

daniel: in this case in ToS, but it may be mentioned in the XEP

jonasw: I usually have different usage if I know I'm wasting resources

you shouldn’t

why ?

Sure you should. Wasteful is wasteful.

it’s the servers responsibility to take care of that.

Whether it's you paying the bill or someone else doesn't change that.

I mean right, uploading a screenshot of each message instead of the message itself *is* wasteful and you shouldn’t do that

jonasw: put on an analogy, if I can choose between plastic and something else, and I know plastic can't be recycled correctly and something else can, I'll take something else.

but I don’t think you should base your individual upload decisions on whether and which retention policy the server implements. you might base your decision which server to use on that, though (that makes sense to me for resource use and privacy reasons)

and don't say "once it's in the trash, it's not my problem"

goffi, I’m not arguing against that, it makes sense. and choosing a server based on those factors may make a lot of sense.

to get back to initial subject, it make sense to put this on ToS, but is there anyway to make it mandatory ? Or at least a SHOULD ?

having this information in the disco#info is good, I think

but we need to make the semantics clear

yes disco#info make sense

a service could have a dynamic retention time based on resource use for example. something like "at least 7 days, at most 14 days, and everything in between depends on how full our disk is". how would that be reflected? should the sevrice say 7 days (this will confuse users who expect their data to be gone after the threshold) or 14 days (which will confuse users who expect the data to be available up to the threshold)?

do we need a min-retention and max-retention field?

And how do you signal policy changes?

that's what framadrop is doing

large file => short rentention

You uploaded when it was a policy to keep for 30 days, and now the server is changed to 15.

in this case it need to be returned in the HTTP Upload <IQ> result

goffi, Expires header on the HTTP response would be my suggestion

(which gives a min-retention)

jonasw: I'm fine with that as long as it is explained in a XEP, either HTTP Upload itself, or a separated one mentioned in HTTP Upload.

yupp

if it's not specified anywher, nobody will do it.

since we're in disco#info topic, a good chunk of, if not all of, the examples in https://xmpp.org/extensions/xep-0045.html don't have the disco#info feature set (and apparently that's a MUST)

Maranda, I think it is generally understood that examples showing disco#info query replies are always exceprts

is it?

🤔

😁

all the XEPs do that

I see nothing that makes me assume that, but okay.

re: HTTP Upload, would a del URL as suggested by Tess Sterr be a big deal? I don't think it complicates too much on server side, and after it's up to the client to keep it or not.

Maranda, it’s easy. disco#info has the disco#info feature as MUST. it is not included. ergo, the examples are excerpts :)

goffi, it’s not very useful IMO

jonasw: why ?

goffi, I'm not strictly against it (especially if optional for the server), but just because there's a delete URL doesn't mean the file will stay until it's deleted

it would require the client to save the URL somewhere, and the multi-client story isn’t tight either

jonasw, if it was an excerpts I would expect some ... or around in 'em, https://xmpp.org/extensions/xep-0045.html#disco-rooms don't look like excerpts (to myself at least).

it would solve the "oops I've uploaded nuclear codes" case, or at least reduce the problem.

Maranda, that’s disco#items, not disco#info :)

jonasw, disco#info examples are in there as well.

brb

Maranda, feel free to submit a patch which adds ... to all the disco#info examples

MattJ: I'm not sure to understand your sentence: "just because there's a delete URL doesn't mean the file will stay until it's deleted"

you mean if we want to upload it forever ?

goffi, I was referring to the second part of your message, "and after it's up to the client to keep it or not"

MattJ, that was referring to the delete URL I think

Since earlier we were discussing retention times

Oh, I see

Ok, ignore me then :)

:)

but I think delete URL and retention time are complementary, and both are trivial to implement.

I can certainly see value in DELETE, even if it only works from a single client it solves the "oops" problem

(where retention time can be "forever as long as you don't kill your quota")

daniel: ^

Though considering the "oops" problem, if your contact is online, they are probably already downloading your file, or downloaded it

Is that not an UX issue? That clients send stuff without any confirmation

Does it ask for confirmation every time you press enter after typing a message?

http://alistapart.com/article/neveruseawarning

MattJ: you may not have shared the link already (think about blogging for instance)

it's not solving 100% the oops, as long at it's gone in the wild, it's not possible anyway. But it does mitigate it for sure.

MattJ: embedding the chosen file as a preview in the input box with an explicit [send] action might improve the UX

I'd prefer a DELETE as well, rather than a delete link that's not deterministic, so clients don't have to store anything

pep., that can't work, because it needs auth

Right, maybe if we had a solution within xmpp already to upload files.. ah wait

MattJ, not an HTTP verb then

Just send an iq or sth, with the generated url and a delete action? :/

pep.: and the iq response yields an URL that you can run HTTP DELETE on?

pep.: but then you have to keep generated URL, it's the same as a delete URL.

How to get rid of the inadvertently uploaded file in 12 easy steps.

Ge0rG, no, just tells you "yes it's deleted"

goffi, that's in the message already

pep.: but that doesn't work with external upload servers

Ge0rG, the answer of that external upload server doesn't go back through the xmpp server?

Ah wait no it's http..

pff

pep.: only if you are in a chat use case.

goffi, what's the other use case?

Avatar use for one

:(

seriously.. don't we already have stuff for that

yep, I was about to say blog, but you have the URL there too

Well in any case you already have the url in the message

and for avatar we can find it

Or the blog post, or the vcard, or..

so it's not a bad suggestion actually.

external upload server is going to be annoying

is there a way to predict that url, from the xmpp server, or is that handled by the upload server all the way down

Turtles

pep., not without storing something

Do we care really? The xmpp server can tell the upload thing, "YOU SHALL USE THIS ID"

(I'm a big fan of servers shouting stuff to things)

:)

Well either the servers stores, or the client stores

You decide

pep., to be clear, there is no communication between Prosody and an external upload server, other than the admin giving them a shared secret

MattJ, yeah I got that bit

:/

So, server devs vs client devs fight?

Has been going on for nearly two decades :)

Good luck

Is it not just a branch of the still ongoing mainfraimes vs fat workstations war?

now we have both, big cloud-mainframes and fat terminals to run big JS apps

the current round is serverless clown infrastructure

But then again, Metal-as-a-Service is a thing too

Maybe editor/iteam-related... but what if the XEP revision history was also made into an RSS feed?

Zash we could even use a feed system built in XMPP, with publications or subscriptions 🤔 wait

Yes, put them into pubsub.xmpp.org!!

I think I suggested this ~10 years ago

We could just http upload an atom xml

(When I say RSS, I mean Atom)

Zash https://github.com/edhelas/atomtopubsub :)

yes yes and https://modules.prosody.im/mod_pubsub_feeds.html

:)

and https://modules.prosody.im/mod_pubsub_post.html

https://news.ycombinator.com/item?id=17408041

that reminds me that i still have feedback for activity pub

daniel: could you add a subsection about properly securing a http upload service in a way that won't allow uploading of html/js to compromise other web applications on the same infrastructure? e.g. an account login portal, or a webchat client

Hmm, was there a standard for adding custom non-standard items in e.g. a MUC configuration form?

I vaguely recall something, but I can't remember if it was just a discussion or actually a standard

Aha, found in XEP-0068

-xep 68

Zash: Field Standardization for Data Forms (Informational, Active, 2012-05-28) See: https://xmpp.org/extensions/xep-0068.html

https://xmpp.org/extensions/xep-0068.html#approach-fieldnames

Kev, Steve Kille: you are currently pursuing user#channel@domain(/resource) as MIX JID, is that still correct?

flow: yes. The spec of this is now in XEP-403, as this encoding is not needed for the basic message distribution specified in MIX core. This encoding is used for sharing presence status of MIX participants and to address Mix participants through the channel. The discussion concluded that it was desirable that each participant bare JID was unique to the participant.