-
Ge0rG
Sam is bringing up a point. How do you protect a web service from files uploaded by users?
-
jonasw
context?
-
Ge0rG
I upload a html+js file via http-upload, share the link with you, it's executed in your browser, I get your webchat / account login session cookie
-
jonasw
right, this should probably be in the security considerations
-
jonasw
but I don’t see where he brings that up
-
Ge0rG
the c.im cookie is only valid for account.c.im, so this specific issue doesn't apply
-
Ge0rG
standards, three days ago, re Council Minutes 2018-06-20
-
jonasw
(the solutions I’ve seen so far is to (a) be careful with the domain you choose for your upload service and (b) use Content-Disposition: attachment with anything which may contain executable code)
-
jonasw
sure, I see that mail, but not what you’re saying in it :)
-
Ge0rG
maybe I extended the focus of the question slightly. the general question was about "executable content"
-
Ge0rG
whatever that is.
-
jonasw
right
-
Ge0rG
there is nothing about content-disposition in the XEP
-
jonasw
somehow I didn’t get at all what that bullet point was trying to say
-
jonasw
but what you interpret into it kinda makes sense
-
Ge0rG
this is actually something I didn't have on my agenda on the last LC.
-
jonasw
or maybe not, I’m not sure. I think when I first read it I thought something along the lines of "okay, a client really should not download and run an .exe, but that’s obvious"
- Ge0rG ponders between "+1 but" and "-1 because"
-
jonasw
I think those are mostly operational concerns which will probably evolve with how the web treats content in the future.
-
jonasw
(nothing specific in mind)
-
jonasw
they’re also not directly related to XMPP
-
jonasw
so I would probably not block based on that
-
jonasw
they also don’t require a namespace bump or otherwise interaction with the entities implementing the protocol, AFAICT
-
Ge0rG
yes
-
Ge0rG
we are only giving server operators a huge, back-facing gun.
-
Ge0rG
nothing wrong with that.
-
jonasw
that’s not what I wanted to s ay
-
jonasw
I wanted to say that it should be expected that we have to modify this in the future anyways, so modifying it right when it goes to draft should be ok too (regarding your "+1 but").
-
Ge0rG
yeah, also don't want to be an asshole for security's sake. I played that card last time already.
-
jonasw
this should be fixed ASAP, but blocking advancement based on that might not help with that
-
jonasw
(something something demotivating authors something)
-
Ge0rG
yeah, that.
-
goffi
Hi, nobody answered my remarks on expiration date for HTTP upload, does anybody care? It seems a major point to me.
-
jonasw
goffi, I can’t recall your feedback. I think it would help the discussion if you would provide a link to your feedback.
-
goffi
jonasw: https://mail.jabber.org/pipermail/standards/2018-June/035181.html
-
jonasw
goffi, I now recall that email. I do remember that you wrote it, but I also remember not seeing this as particularly important
-
goffi
you don't think expiration date is important ?
-
jonasw
maybe you should clarify on the mailing list why you think this is particularly important
-
jonasw
I’m not sure it’s extremely important to have this on the protocol.✎ -
goffi
we upload file on a server, we don't know for how long, and we have no access to it then.
-
jonasw
I’m not sure it’s extremely important to have this on the protocol level, it should be included in the Privacy Policy for sure. ✏
-
goffi
jonasw: OK I'll try to explain more, thanks for feedback, I had the feeling to be ignored which is quite frustrating as it is a major point to me (even if it's to say that it's not that important for whatever reason).
-
jonasw
sure
-
jonasw
I fully understand that
-
jonasw
to me, it really did not come across to me that you thought that this is an important point, mostly because it lacked any rationale
-
jonasw
(typically when I try to raise a point I think is important, I’ll put a rationale on it *why* I think its important.)
-
daniel
goffi: in parts that was due to me being extremely busy with work these days. And in parts I was taking the easy way out and was waiting for someone to write a me too or something
-
jonasw
daniel, FWIW, I think this is a good idea and supplements the work I was aiming to do with ToS.
-
jonasw
I also think that this can easily be added later because it’s a data form \o/
-
daniel
yeah i’m ok with just putting it in there as an optional data form field
-
daniel
but I don’t see a technical use case for it because it won’t be reliable anyway
-
daniel
i see it as a pure TOS informational thing
-
jonasw
maybe
-
goffi
jonasw: indeed, I've made a new message. daniel: OK, thanks to come in the discussion here, I've sent a new message to explain. I know it's not reliable, but it's informational, and there is a relation of trust with a server anyway (if I can trust retention date, I can't trust anything). It's the same with ToS.
-
Ge0rG
Did you just add data forms to http upload?
-
goffi
the think with putting it in the HTTP Upload XEP is to make it mandatory. If it's not, nobody will set the data. Also the HTTP Upoad Component probably knows when the files are deleted so it can be set automatically.
-
jonasw
Ge0rG, they were always there
-
jonasw
Ge0rG, in disco#info
-
daniel
goffi, i see and agree with most of your points. and don’t think the suggested solution is a very good one or a solution at all
-
Ge0rG
Ah
-
daniel
if anything we would need the client to mark a file as permanent (for avatars or blogs posts) or shortlived
-
daniel
and maybe a deletion command
-
jonasw
daniel, +1
-
daniel
just putting the information in there on how long a server might store it doesn't do anything
-
daniel
or very very little
-
daniel
i mean what i'm a supposed to do? reupload the avatar every 30 days?
-
goffi
having a way to specify retention time in the request would be better, my email is not discarding this option.
-
Ge0rG
Permanent storage requirements will break my GDPR compliance script... 🤔
-
Ge0rG
Or we need to have multiple upload components for temporary and permanent storage
-
jonasw
ew
-
goffi
daniel: is is possible to write your comments on standard@ even in a short email, so we can have other people input?
-
Kev
I imagine it's generally down to local policy.
-
jonasw
Ge0rG, your component could implement this trivially by putting permanant files in a different directory (and reflecting that in the URL)
-
Kev
But avatars are an exception as they are 'always' permanent.
-
daniel
jonasw, yeah. the problem with that is that you still need some policy on the server that prevents stupid clients from always using the permanent store
-
daniel
and cluttering your file system
-
Kev
Do you?
-
Kev
I mean, you can always have usage limits.
-
jonasw
daniel, quotas? and explicit request for permanent storage so that stupid clients need to be actively stupid at least
-
jonasw
then it blows up in their face for reaching quotas and you’re done :)
-
jonasw
quotas can be really small for permanent storage on IM deployments, 10 MiB or something
-
jonasw
that’ll be reached extremely quickly
-
daniel
jonasw, yes and yes. but it makes implementations a bit more complicated then 'just put it in a different folder' is what i'm saying
-
jonasw
(when abused for sending gifs to MUCs)
-
jonasw
daniel, you need to have some type of quota anyways
-
daniel
right. but then you need different quotas
-
jonasw
that’s true
-
daniel
i'm not saying it can’t be done. but right now most http upload implementations are very very simple
-
jonasw
indeed
-
jonasw
and I think if one went around and started abusing that, many servers would fail very quickly with ENOSPC
-
Ge0rG
this sounds like a quick escalation from "file sharing" to "cloud storage provider"
-
daniel
yes
-
daniel
especially if you start doing delete
-
daniel
and list files
-
jonasw
just implement WebDAV with XMPP auth
-
Ge0rG
if you want to provide permanent storage, you need both listing and deletion. and quotas
-
jonasw
actually, that doesn’t sound *too* bad for permanent HTTP-based storage.
-
daniel
that’s why i see a general desire to have those features but am hesitent to put them in the xep
-
goffi
that looks like the component I'm doing with Jingle.
-
Ge0rG
jonasw: but custom http headers for obscure aws servers!11!
-
jonasw
you’ll probably find a WebDAV library and servers which support WebDAV. just make the client pass some token in the HTTP Auth header and done.
-
goffi
in this case why not keeping HTTP Upload for simple temporary storage, and specifying date of expiration, and having more elaborated components for stuff like avatar or blogs ?
-
jonasw
goffi, this sounds very reasonable
-
goffi
and explicitly forbidding permanent storage with HTTP Upload
-
jonasw
I wouldn’t go that far
-
daniel
what’s the use case for the expiration then?
-
daniel
*the expiration date
-
goffi
if we want to keep the implementation simple, which is the main interest of this XEP I think, permanent storage doesn't seem an option.
-
goffi
daniel: knowing when the file is deleted, how long we can still use the link.
-
goffi
I'm not confortable in uploading a picture if I don't know how long it will stay
-
goffi
I can encrypt, but I then have the feeling to waster resources.
-
daniel
apperently you are comfortable with sending messages without knowing how long they will stay in archive
-
daniel
that's what the TOS are for…
-
goffi
I know how long they stay
-
goffi
on my server at least
-
Kev
How?
-
jonasw
you also know how long files are stored on your server ;-)
-
goffi
right, but I have no MAM at the moment, so my server is not keeping anything. But the same question is valid for MAM I've never said the opposite.
-
daniel
yeah i’m not sure that every feature in xmpp that has the ability to store something needs to signal it's own retention period
-
goffi
and about resource, it's not the same about a couple of message, and one or several MB files
-
daniel
my vcard service never report how long it will store that either
-
jonasw
goffi, not your department; the server is responsible for managing the resources it offers to you.
-
jonasw
it must not rely on your goodwill
-
goffi
daniel: in this case in ToS, but it may be mentioned in the XEP
-
goffi
jonasw: I usually have different usage if I know I'm wasting resources
-
jonasw
you shouldn’t
-
goffi
why ?
-
Kev
Sure you should. Wasteful is wasteful.
-
jonasw
it’s the servers responsibility to take care of that.
-
Kev
Whether it's you paying the bill or someone else doesn't change that.
-
jonasw
I mean right, uploading a screenshot of each message instead of the message itself *is* wasteful and you shouldn’t do that
-
goffi
jonasw: put on an analogy, if I can choose between plastic and something else, and I know plastic can't be recycled correctly and something else can, I'll take something else.
-
jonasw
but I don’t think you should base your individual upload decisions on whether and which retention policy the server implements. you might base your decision which server to use on that, though (that makes sense to me for resource use and privacy reasons)
-
goffi
and don't say "once it's in the trash, it's not my problem"
-
jonasw
goffi, I’m not arguing against that, it makes sense. and choosing a server based on those factors may make a lot of sense.
-
goffi
to get back to initial subject, it make sense to put this on ToS, but is there anyway to make it mandatory ? Or at least a SHOULD ?
-
jonasw
having this information in the disco#info is good, I think
-
jonasw
but we need to make the semantics clear
-
goffi
yes disco#info make sense
-
jonasw
a service could have a dynamic retention time based on resource use for example. something like "at least 7 days, at most 14 days, and everything in between depends on how full our disk is". how would that be reflected? should the sevrice say 7 days (this will confuse users who expect their data to be gone after the threshold) or 14 days (which will confuse users who expect the data to be available up to the threshold)?
-
jonasw
do we need a min-retention and max-retention field?
-
Kev
And how do you signal policy changes?
-
goffi
that's what framadrop is doing
-
goffi
large file => short rentention
-
Kev
You uploaded when it was a policy to keep for 30 days, and now the server is changed to 15.
-
goffi
in this case it need to be returned in the HTTP Upload <IQ> result
-
jonasw
goffi, Expires header on the HTTP response would be my suggestion
-
jonasw
(which gives a min-retention)
-
goffi
jonasw: I'm fine with that as long as it is explained in a XEP, either HTTP Upload itself, or a separated one mentioned in HTTP Upload.
-
jonasw
yupp
-
goffi
if it's not specified anywher, nobody will do it.
-
Maranda
since we're in disco#info topic, a good chunk of, if not all of, the examples in https://xmpp.org/extensions/xep-0045.html don't have the disco#info feature set (and apparently that's a MUST)
-
jonasw
Maranda, I think it is generally understood that examples showing disco#info query replies are always exceprts
-
Maranda
is it?
-
Maranda
🤔
-
Maranda
😁
-
jonasw
all the XEPs do that
-
Maranda
I see nothing that makes me assume that, but okay.
-
goffi
re: HTTP Upload, would a del URL as suggested by Tess Sterr be a big deal? I don't think it complicates too much on server side, and after it's up to the client to keep it or not.
-
jonasw
Maranda, it’s easy. disco#info has the disco#info feature as MUST. it is not included. ergo, the examples are excerpts :)
-
jonasw
goffi, it’s not very useful IMO
-
goffi
jonasw: why ?
-
MattJ
goffi, I'm not strictly against it (especially if optional for the server), but just because there's a delete URL doesn't mean the file will stay until it's deleted
-
jonasw
it would require the client to save the URL somewhere, and the multi-client story isn’t tight either
-
Maranda
jonasw, if it was an excerpts I would expect some ... or around in 'em, https://xmpp.org/extensions/xep-0045.html#disco-rooms don't look like excerpts (to myself at least).
-
goffi
it would solve the "oops I've uploaded nuclear codes" case, or at least reduce the problem.
-
jonasw
Maranda, that’s disco#items, not disco#info :)
-
Maranda
jonasw, disco#info examples are in there as well.
-
Maranda
brb
-
jonasw
Maranda, feel free to submit a patch which adds ... to all the disco#info examples
-
goffi
MattJ: I'm not sure to understand your sentence: "just because there's a delete URL doesn't mean the file will stay until it's deleted"
-
goffi
you mean if we want to upload it forever ?
-
MattJ
goffi, I was referring to the second part of your message, "and after it's up to the client to keep it or not"
-
jonasw
MattJ, that was referring to the delete URL I think
-
MattJ
Since earlier we were discussing retention times
-
MattJ
Oh, I see
-
MattJ
Ok, ignore me then :)
-
goffi
:)
-
goffi
but I think delete URL and retention time are complementary, and both are trivial to implement.
-
MattJ
I can certainly see value in DELETE, even if it only works from a single client it solves the "oops" problem
-
goffi
(where retention time can be "forever as long as you don't kill your quota")
-
goffi
daniel: ^
-
MattJ
Though considering the "oops" problem, if your contact is online, they are probably already downloading your file, or downloaded it
-
Zash
Is that not an UX issue? That clients send stuff without any confirmation
-
MattJ
Does it ask for confirmation every time you press enter after typing a message?
-
MattJ
http://alistapart.com/article/neveruseawarning
-
goffi
MattJ: you may not have shared the link already (think about blogging for instance)
-
goffi
it's not solving 100% the oops, as long at it's gone in the wild, it's not possible anyway. But it does mitigate it for sure.
-
Ge0rG
MattJ: embedding the chosen file as a preview in the input box with an explicit [send] action might improve the UX
-
pep.
I'd prefer a DELETE as well, rather than a delete link that's not deterministic, so clients don't have to store anything
-
MattJ
pep., that can't work, because it needs auth
-
pep.
Right, maybe if we had a solution within xmpp already to upload files.. ah wait
-
pep.
MattJ, not an HTTP verb then
-
pep.
Just send an iq or sth, with the generated url and a delete action? :/
-
Ge0rG
pep.: and the iq response yields an URL that you can run HTTP DELETE on?
-
goffi
pep.: but then you have to keep generated URL, it's the same as a delete URL.
-
Ge0rG
How to get rid of the inadvertently uploaded file in 12 easy steps.
-
pep.
Ge0rG, no, just tells you "yes it's deleted"
-
pep.
goffi, that's in the message already
-
Ge0rG
pep.: but that doesn't work with external upload servers
-
pep.
Ge0rG, the answer of that external upload server doesn't go back through the xmpp server?
-
pep.
Ah wait no it's http..
-
pep.
pff
-
goffi
pep.: only if you are in a chat use case.
-
pep.
goffi, what's the other use case?
-
Zash
Avatar use for one
-
pep.
:(
-
pep.
seriously.. don't we already have stuff for that
-
goffi
yep, I was about to say blog, but you have the URL there too
-
pep.
Well in any case you already have the url in the message
-
goffi
and for avatar we can find it
-
pep.
Or the blog post, or the vcard, or..
-
goffi
so it's not a bad suggestion actually.
-
pep.
external upload server is going to be annoying
-
pep.
is there a way to predict that url, from the xmpp server, or is that handled by the upload server all the way down
-
Zash
Turtles
-
MattJ
pep., not without storing something
-
pep.
Do we care really? The xmpp server can tell the upload thing, "YOU SHALL USE THIS ID"
-
Guus
(I'm a big fan of servers shouting stuff to things)
-
pep.
:)
-
pep.
Well either the servers stores, or the client stores
-
pep.
You decide
-
MattJ
pep., to be clear, there is no communication between Prosody and an external upload server, other than the admin giving them a shared secret
-
pep.
MattJ, yeah I got that bit
-
pep.
:/
-
pep.
So, server devs vs client devs fight?
-
MattJ
Has been going on for nearly two decades :)
-
pep.
Good luck
-
Zash
Is it not just a branch of the still ongoing mainfraimes vs fat workstations war?
-
edhelas
now we have both, big cloud-mainframes and fat terminals to run big JS apps
-
Ge0rG
the current round is serverless clown infrastructure
-
Ge0rG
But then again, Metal-as-a-Service is a thing too
-
Zash
Maybe editor/iteam-related... but what if the XEP revision history was also made into an RSS feed?
-
edhelas
Zash we could even use a feed system built in XMPP, with publications or subscriptions 🤔 wait
-
Zash
Yes, put them into pubsub.xmpp.org!!
-
MattJ
I think I suggested this ~10 years ago
-
Ge0rG
We could just http upload an atom xml
-
Zash
(When I say RSS, I mean Atom)
-
edhelas
Zash https://github.com/edhelas/atomtopubsub :)
-
Zash
yes yes and https://modules.prosody.im/mod_pubsub_feeds.html
-
edhelas
:)
-
Zash
and https://modules.prosody.im/mod_pubsub_post.html
-
edhelas
https://news.ycombinator.com/item?id=17408041
-
daniel
that reminds me that i still have feedback for activity pub
-
Ge0rG
daniel: could you add a subsection about properly securing a http upload service in a way that won't allow uploading of html/js to compromise other web applications on the same infrastructure? e.g. an account login portal, or a webchat client
-
MattJ
Hmm, was there a standard for adding custom non-standard items in e.g. a MUC configuration form?
-
MattJ
I vaguely recall something, but I can't remember if it was just a discussion or actually a standard
-
MattJ
Aha, found in XEP-0068
-
Zash
-xep 68
-
Bunneh
Zash: Field Standardization for Data Forms (Informational, Active, 2012-05-28) See: https://xmpp.org/extensions/xep-0068.html
-
MattJ
https://xmpp.org/extensions/xep-0068.html#approach-fieldnames
-
flow
Kev, Steve Kille: you are currently pursuing user#channel@domain(/resource) as MIX JID, is that still correct?
-
Steve Kille
flow: yes. The spec of this is now in XEP-403, as this encoding is not needed for the basic message distribution specified in MIX core. This encoding is used for sharing presence status of MIX participants and to address Mix participants through the channel. The discussion concluded that it was desirable that each participant bare JID was unique to the participant.