XSF Discussion - 2018-06-27


  1. Ge0rG

    Sam is bringing up a point. How do you protect a web service from files uploaded by users?

  2. jonasw

    context?

  3. Ge0rG

    I upload a html+js file via http-upload, share the link with you, it's executed in your browser, I get your webchat / account login session cookie

  4. jonasw

    right, this should probably be in the security considerations

  5. jonasw

    but I don’t see where he brings that up

  6. Ge0rG

    the c.im cookie is only valid for account.c.im, so this specific issue doesn't apply

  7. Ge0rG

    standards, three days ago, re Council Minutes 2018-06-20

  8. jonasw

    (the solutions I’ve seen so far is to (a) be careful with the domain you choose for your upload service and (b) use Content-Disposition: attachment with anything which may contain executable code)

  9. jonasw

    sure, I see that mail, but not what you’re saying in it :)

  10. Ge0rG

    maybe I extended the focus of the question slightly. the general question was about "executable content"

  11. Ge0rG

    whatever that is.

  12. jonasw

    right

  13. Ge0rG

    there is nothing about content-disposition in the XEP

  14. jonasw

    somehow I didn’t get at all what that bullet point was trying to say

  15. jonasw

    but what you interpret into it kinda makes sense

  16. Ge0rG

    this is actually something I didn't have on my agenda on the last LC.

  17. jonasw

    or maybe not, I’m not sure. I think when I first read it I thought something along the lines of "okay, a client really should not download and run an .exe, but that’s obvious"

  18. Ge0rG ponders between "+1 but" and "-1 because"

  19. jonasw

    I think those are mostly operational concerns which will probably evolve with how the web treats content in the future.

  20. jonasw

    (nothing specific in mind)

  21. jonasw

    they’re also not directly related to XMPP

  22. jonasw

    so I would probably not block based on that

  23. jonasw

    they also don’t require a namespace bump or otherwise interaction with the entities implementing the protocol, AFAICT

  24. Ge0rG

    yes

  25. Ge0rG

    we are only giving server operators a huge, back-facing gun.

  26. Ge0rG

    nothing wrong with that.

  27. jonasw

    that’s not what I wanted to s ay

  28. jonasw

    I wanted to say that it should be expected that we have to modify this in the future anyways, so modifying it right when it goes to draft should be ok too (regarding your "+1 but").

  29. Ge0rG

    yeah, also don't want to be an asshole for security's sake. I played that card last time already.

  30. jonasw

    this should be fixed ASAP, but blocking advancement based on that might not help with that

  31. jonasw

    (something something demotivating authors something)

  32. Ge0rG

    yeah, that.

  33. goffi

    Hi, nobody answered my remarks on expiration date for HTTP upload, does anybody care? It seems a major point to me.

  34. jonasw

    goffi, I can’t recall your feedback. I think it would help the discussion if you would provide a link to your feedback.

  35. goffi

    jonasw: https://mail.jabber.org/pipermail/standards/2018-June/035181.html

  36. jonasw

    goffi, I now recall that email. I do remember that you wrote it, but I also remember not seeing this as particularly important

  37. goffi

    you don't think expiration date is important ?

  38. jonasw

    maybe you should clarify on the mailing list why you think this is particularly important

  39. jonasw

    I’m not sure it’s extremely important to have this on the protocol.

  40. goffi

    we upload file on a server, we don't know for how long, and we have no access to it then.

  41. jonasw

    I’m not sure it’s extremely important to have this on the protocol level, it should be included in the Privacy Policy for sure.

  42. goffi

    jonasw: OK I'll try to explain more, thanks for feedback, I had the feeling to be ignored which is quite frustrating as it is a major point to me (even if it's to say that it's not that important for whatever reason).

  43. jonasw

    sure

  44. jonasw

    I fully understand that

  45. jonasw

    to me, it really did not come across to me that you thought that this is an important point, mostly because it lacked any rationale

  46. jonasw

    (typically when I try to raise a point I think is important, I’ll put a rationale on it *why* I think its important.)

  47. daniel

    goffi: in parts that was due to me being extremely busy with work these days. And in parts I was taking the easy way out and was waiting for someone to write a me too or something

  48. jonasw

    daniel, FWIW, I think this is a good idea and supplements the work I was aiming to do with ToS.

  49. jonasw

    I also think that this can easily be added later because it’s a data form \o/

  50. daniel

    yeah i’m ok with just putting it in there as an optional data form field

  51. daniel

    but I don’t see a technical use case for it because it won’t be reliable anyway

  52. daniel

    i see it as a pure TOS informational thing

  53. jonasw

    maybe

  54. goffi

    jonasw: indeed, I've made a new message. daniel: OK, thanks to come in the discussion here, I've sent a new message to explain. I know it's not reliable, but it's informational, and there is a relation of trust with a server anyway (if I can trust retention date, I can't trust anything). It's the same with ToS.

  55. Ge0rG

    Did you just add data forms to http upload?

  56. goffi

    the think with putting it in the HTTP Upload XEP is to make it mandatory. If it's not, nobody will set the data. Also the HTTP Upoad Component probably knows when the files are deleted so it can be set automatically.

  57. jonasw

    Ge0rG, they were always there

  58. jonasw

    Ge0rG, in disco#info

  59. daniel

    goffi, i see and agree with most of your points. and don’t think the suggested solution is a very good one or a solution at all

  60. Ge0rG

    Ah

  61. daniel

    if anything we would need the client to mark a file as permanent (for avatars or blogs posts) or shortlived

  62. daniel

    and maybe a deletion command

  63. jonasw

    daniel, +1

  64. daniel

    just putting the information in there on how long a server might store it doesn't do anything

  65. daniel

    or very very little

  66. daniel

    i mean what i'm a supposed to do? reupload the avatar every 30 days?

  67. goffi

    having a way to specify retention time in the request would be better, my email is not discarding this option.

  68. Ge0rG

    Permanent storage requirements will break my GDPR compliance script... 🤔

  69. Ge0rG

    Or we need to have multiple upload components for temporary and permanent storage

  70. jonasw

    ew

  71. goffi

    daniel: is is possible to write your comments on standard@ even in a short email, so we can have other people input?

  72. Kev

    I imagine it's generally down to local policy.

  73. jonasw

    Ge0rG, your component could implement this trivially by putting permanant files in a different directory (and reflecting that in the URL)

  74. Kev

    But avatars are an exception as they are 'always' permanent.

  75. daniel

    jonasw, yeah. the problem with that is that you still need some policy on the server that prevents stupid clients from always using the permanent store

  76. daniel

    and cluttering your file system

  77. Kev

    Do you?

  78. Kev

    I mean, you can always have usage limits.

  79. jonasw

    daniel, quotas? and explicit request for permanent storage so that stupid clients need to be actively stupid at least

  80. jonasw

    then it blows up in their face for reaching quotas and you’re done :)

  81. jonasw

    quotas can be really small for permanent storage on IM deployments, 10 MiB or something

  82. jonasw

    that’ll be reached extremely quickly

  83. daniel

    jonasw, yes and yes. but it makes implementations a bit more complicated then 'just put it in a different folder' is what i'm saying

  84. jonasw

    (when abused for sending gifs to MUCs)

  85. jonasw

    daniel, you need to have some type of quota anyways

  86. daniel

    right. but then you need different quotas

  87. jonasw

    that’s true

  88. daniel

    i'm not saying it can’t be done. but right now most http upload implementations are very very simple

  89. jonasw

    indeed

  90. jonasw

    and I think if one went around and started abusing that, many servers would fail very quickly with ENOSPC

  91. Ge0rG

    this sounds like a quick escalation from "file sharing" to "cloud storage provider"

  92. daniel

    yes

  93. daniel

    especially if you start doing delete

  94. daniel

    and list files

  95. jonasw

    just implement WebDAV with XMPP auth

  96. Ge0rG

    if you want to provide permanent storage, you need both listing and deletion. and quotas

  97. jonasw

    actually, that doesn’t sound *too* bad for permanent HTTP-based storage.

  98. daniel

    that’s why i see a general desire to have those features but am hesitent to put them in the xep

  99. goffi

    that looks like the component I'm doing with Jingle.

  100. Ge0rG

    jonasw: but custom http headers for obscure aws servers!11!

  101. jonasw

    you’ll probably find a WebDAV library and servers which support WebDAV. just make the client pass some token in the HTTP Auth header and done.

  102. goffi

    in this case why not keeping HTTP Upload for simple temporary storage, and specifying date of expiration, and having more elaborated components for stuff like avatar or blogs ?

  103. jonasw

    goffi, this sounds very reasonable

  104. goffi

    and explicitly forbidding permanent storage with HTTP Upload

  105. jonasw

    I wouldn’t go that far

  106. daniel

    what’s the use case for the expiration then?

  107. daniel

    *the expiration date

  108. goffi

    if we want to keep the implementation simple, which is the main interest of this XEP I think, permanent storage doesn't seem an option.

  109. goffi

    daniel: knowing when the file is deleted, how long we can still use the link.

  110. goffi

    I'm not confortable in uploading a picture if I don't know how long it will stay

  111. goffi

    I can encrypt, but I then have the feeling to waster resources.

  112. daniel

    apperently you are comfortable with sending messages without knowing how long they will stay in archive

  113. daniel

    that's what the TOS are for…

  114. goffi

    I know how long they stay

  115. goffi

    on my server at least

  116. Kev

    How?

  117. jonasw

    you also know how long files are stored on your server ;-)

  118. goffi

    right, but I have no MAM at the moment, so my server is not keeping anything. But the same question is valid for MAM I've never said the opposite.

  119. daniel

    yeah i’m not sure that every feature in xmpp that has the ability to store something needs to signal it's own retention period

  120. goffi

    and about resource, it's not the same about a couple of message, and one or several MB files

  121. daniel

    my vcard service never report how long it will store that either

  122. jonasw

    goffi, not your department; the server is responsible for managing the resources it offers to you.

  123. jonasw

    it must not rely on your goodwill

  124. goffi

    daniel: in this case in ToS, but it may be mentioned in the XEP

  125. goffi

    jonasw: I usually have different usage if I know I'm wasting resources

  126. jonasw

    you shouldn’t

  127. goffi

    why ?

  128. Kev

    Sure you should. Wasteful is wasteful.

  129. jonasw

    it’s the servers responsibility to take care of that.

  130. Kev

    Whether it's you paying the bill or someone else doesn't change that.

  131. jonasw

    I mean right, uploading a screenshot of each message instead of the message itself *is* wasteful and you shouldn’t do that

  132. goffi

    jonasw: put on an analogy, if I can choose between plastic and something else, and I know plastic can't be recycled correctly and something else can, I'll take something else.

  133. jonasw

    but I don’t think you should base your individual upload decisions on whether and which retention policy the server implements. you might base your decision which server to use on that, though (that makes sense to me for resource use and privacy reasons)

  134. goffi

    and don't say "once it's in the trash, it's not my problem"

  135. jonasw

    goffi, I’m not arguing against that, it makes sense. and choosing a server based on those factors may make a lot of sense.

  136. goffi

    to get back to initial subject, it make sense to put this on ToS, but is there anyway to make it mandatory ? Or at least a SHOULD ?

  137. jonasw

    having this information in the disco#info is good, I think

  138. jonasw

    but we need to make the semantics clear

  139. goffi

    yes disco#info make sense

  140. jonasw

    a service could have a dynamic retention time based on resource use for example. something like "at least 7 days, at most 14 days, and everything in between depends on how full our disk is". how would that be reflected? should the sevrice say 7 days (this will confuse users who expect their data to be gone after the threshold) or 14 days (which will confuse users who expect the data to be available up to the threshold)?

  141. jonasw

    do we need a min-retention and max-retention field?

  142. Kev

    And how do you signal policy changes?

  143. goffi

    that's what framadrop is doing

  144. goffi

    large file => short rentention

  145. Kev

    You uploaded when it was a policy to keep for 30 days, and now the server is changed to 15.

  146. goffi

    in this case it need to be returned in the HTTP Upload <IQ> result

  147. jonasw

    goffi, Expires header on the HTTP response would be my suggestion

  148. jonasw

    (which gives a min-retention)

  149. goffi

    jonasw: I'm fine with that as long as it is explained in a XEP, either HTTP Upload itself, or a separated one mentioned in HTTP Upload.

  150. jonasw

    yupp

  151. goffi

    if it's not specified anywher, nobody will do it.

  152. Maranda

    since we're in disco#info topic, a good chunk of, if not all of, the examples in https://xmpp.org/extensions/xep-0045.html don't have the disco#info feature set (and apparently that's a MUST)

  153. jonasw

    Maranda, I think it is generally understood that examples showing disco#info query replies are always exceprts

  154. Maranda

    is it?

  155. Maranda

    🤔

  156. Maranda

    😁

  157. jonasw

    all the XEPs do that

  158. Maranda

    I see nothing that makes me assume that, but okay.

  159. goffi

    re: HTTP Upload, would a del URL as suggested by Tess Sterr be a big deal? I don't think it complicates too much on server side, and after it's up to the client to keep it or not.

  160. jonasw

    Maranda, it’s easy. disco#info has the disco#info feature as MUST. it is not included. ergo, the examples are excerpts :)

  161. jonasw

    goffi, it’s not very useful IMO

  162. goffi

    jonasw: why ?

  163. MattJ

    goffi, I'm not strictly against it (especially if optional for the server), but just because there's a delete URL doesn't mean the file will stay until it's deleted

  164. jonasw

    it would require the client to save the URL somewhere, and the multi-client story isn’t tight either

  165. Maranda

    jonasw, if it was an excerpts I would expect some ... or around in 'em, https://xmpp.org/extensions/xep-0045.html#disco-rooms don't look like excerpts (to myself at least).

  166. goffi

    it would solve the "oops I've uploaded nuclear codes" case, or at least reduce the problem.

  167. jonasw

    Maranda, that’s disco#items, not disco#info :)

  168. Maranda

    jonasw, disco#info examples are in there as well.

  169. Maranda

    brb

  170. jonasw

    Maranda, feel free to submit a patch which adds ... to all the disco#info examples

  171. goffi

    MattJ: I'm not sure to understand your sentence: "just because there's a delete URL doesn't mean the file will stay until it's deleted"

  172. goffi

    you mean if we want to upload it forever ?

  173. MattJ

    goffi, I was referring to the second part of your message, "and after it's up to the client to keep it or not"

  174. jonasw

    MattJ, that was referring to the delete URL I think

  175. MattJ

    Since earlier we were discussing retention times

  176. MattJ

    Oh, I see

  177. MattJ

    Ok, ignore me then :)

  178. goffi

    :)

  179. goffi

    but I think delete URL and retention time are complementary, and both are trivial to implement.

  180. MattJ

    I can certainly see value in DELETE, even if it only works from a single client it solves the "oops" problem

  181. goffi

    (where retention time can be "forever as long as you don't kill your quota")

  182. goffi

    daniel: ^

  183. MattJ

    Though considering the "oops" problem, if your contact is online, they are probably already downloading your file, or downloaded it

  184. Zash

    Is that not an UX issue? That clients send stuff without any confirmation

  185. MattJ

    Does it ask for confirmation every time you press enter after typing a message?

  186. MattJ

    http://alistapart.com/article/neveruseawarning

  187. goffi

    MattJ: you may not have shared the link already (think about blogging for instance)

  188. goffi

    it's not solving 100% the oops, as long at it's gone in the wild, it's not possible anyway. But it does mitigate it for sure.

  189. Ge0rG

    MattJ: embedding the chosen file as a preview in the input box with an explicit [send] action might improve the UX

  190. pep.

    I'd prefer a DELETE as well, rather than a delete link that's not deterministic, so clients don't have to store anything

  191. MattJ

    pep., that can't work, because it needs auth

  192. pep.

    Right, maybe if we had a solution within xmpp already to upload files.. ah wait

  193. pep.

    MattJ, not an HTTP verb then

  194. pep.

    Just send an iq or sth, with the generated url and a delete action? :/

  195. Ge0rG

    pep.: and the iq response yields an URL that you can run HTTP DELETE on?

  196. goffi

    pep.: but then you have to keep generated URL, it's the same as a delete URL.

  197. Ge0rG

    How to get rid of the inadvertently uploaded file in 12 easy steps.

  198. pep.

    Ge0rG, no, just tells you "yes it's deleted"

  199. pep.

    goffi, that's in the message already

  200. Ge0rG

    pep.: but that doesn't work with external upload servers

  201. pep.

    Ge0rG, the answer of that external upload server doesn't go back through the xmpp server?

  202. pep.

    Ah wait no it's http..

  203. pep.

    pff

  204. goffi

    pep.: only if you are in a chat use case.

  205. pep.

    goffi, what's the other use case?

  206. Zash

    Avatar use for one

  207. pep.

    :(

  208. pep.

    seriously.. don't we already have stuff for that

  209. goffi

    yep, I was about to say blog, but you have the URL there too

  210. pep.

    Well in any case you already have the url in the message

  211. goffi

    and for avatar we can find it

  212. pep.

    Or the blog post, or the vcard, or..

  213. goffi

    so it's not a bad suggestion actually.

  214. pep.

    external upload server is going to be annoying

  215. pep.

    is there a way to predict that url, from the xmpp server, or is that handled by the upload server all the way down

  216. Zash

    Turtles

  217. MattJ

    pep., not without storing something

  218. pep.

    Do we care really? The xmpp server can tell the upload thing, "YOU SHALL USE THIS ID"

  219. Guus

    (I'm a big fan of servers shouting stuff to things)

  220. pep.

    :)

  221. pep.

    Well either the servers stores, or the client stores

  222. pep.

    You decide

  223. MattJ

    pep., to be clear, there is no communication between Prosody and an external upload server, other than the admin giving them a shared secret

  224. pep.

    MattJ, yeah I got that bit

  225. pep.

    :/

  226. pep.

    So, server devs vs client devs fight?

  227. MattJ

    Has been going on for nearly two decades :)

  228. pep.

    Good luck

  229. Zash

    Is it not just a branch of the still ongoing mainfraimes vs fat workstations war?

  230. edhelas

    now we have both, big cloud-mainframes and fat terminals to run big JS apps

  231. Ge0rG

    the current round is serverless clown infrastructure

  232. Ge0rG

    But then again, Metal-as-a-Service is a thing too

  233. Zash

    Maybe editor/iteam-related... but what if the XEP revision history was also made into an RSS feed?

  234. edhelas

    Zash we could even use a feed system built in XMPP, with publications or subscriptions 🤔 wait

  235. Zash

    Yes, put them into pubsub.xmpp.org!!

  236. MattJ

    I think I suggested this ~10 years ago

  237. Ge0rG

    We could just http upload an atom xml

  238. Zash

    (When I say RSS, I mean Atom)

  239. edhelas

    Zash https://github.com/edhelas/atomtopubsub :)

  240. Zash

    yes yes and https://modules.prosody.im/mod_pubsub_feeds.html

  241. edhelas

    :)

  242. Zash

    and https://modules.prosody.im/mod_pubsub_post.html

  243. edhelas

    https://news.ycombinator.com/item?id=17408041

  244. daniel

    that reminds me that i still have feedback for activity pub

  245. Ge0rG

    daniel: could you add a subsection about properly securing a http upload service in a way that won't allow uploading of html/js to compromise other web applications on the same infrastructure? e.g. an account login portal, or a webchat client

  246. MattJ

    Hmm, was there a standard for adding custom non-standard items in e.g. a MUC configuration form?

  247. MattJ

    I vaguely recall something, but I can't remember if it was just a discussion or actually a standard

  248. MattJ

    Aha, found in XEP-0068

  249. Zash

    -xep 68

  250. Bunneh

    Zash: Field Standardization for Data Forms (Informational, Active, 2012-05-28) See: https://xmpp.org/extensions/xep-0068.html

  251. MattJ

    https://xmpp.org/extensions/xep-0068.html#approach-fieldnames

  252. flow

    Kev, Steve Kille: you are currently pursuing user#channel@domain(/resource) as MIX JID, is that still correct?

  253. Steve Kille

    flow: yes. The spec of this is now in XEP-403, as this encoding is not needed for the basic message distribution specified in MIX core. This encoding is used for sharing presence status of MIX participants and to address Mix participants through the channel. The discussion concluded that it was desirable that each participant bare JID was unique to the participant.