XMPP Service Operators - 2023-03-15

> opal, https://www.youtube.com/watch?v=6n3pFFPSlW4 yep and i hate it

I'm getting a question on how to configure HTTP File Upload to allow for transfers of 2 gigabyte (we're limiting the file size by default). Apart from the configuration option: are those kind of sizes practically feasible with HTTP File Upload? Does anyone have experience with those kind of sizes?

you may run into some funny issues there

Yes, we do such uploads

- some reverse proxies will buffer the request -> you need that much disk space available in wherever they buffer - some reverse proxies and application servers will also set a timeout until the entire request must be received -> if (size / user_upload_bandwidth) >= that_timeout, your uploads will fail

Two notes: 1. Upload should be done using HTTP ranges requests, and not in a single request. 2. We “prepare” the upload with a scope/purpose of the upload. This allows the server to determine place, life-time, size-limits, etc.

Also HTTP does not support upload resumption

- there is no resumption specified -> if the upload interrupts, it has to start from scratch

MattJ, it does in principle, but it's not written down in the XEP ✏

jonas’, it does, where?

range requests, as Peter Waher says

That's for the response

Thanks

also for the request.

I dare say... citation needed...

I'm fairly certain to have implemented such a thing in javascript in 2018

I can look it up later

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Range <-- suggests download, not upload.

It's not in RFC 7233

> A server MUST ignore a Range header field received with a request method that is unrecognized or for which range handling is not defined. For this specification, GET is the only method for which range handling is defined.

https://httpwg.org/specs/rfc9110.html#field.range

and I've never seen anything like this used. For example, in Amazon S3 they just break the upload slot into multiple, so you can do a normal upload to a bunch of different URLs (which the server maps to a single result)

no-one here is ruling out that jonas’ accidentally wrote his own HTTP stack and simply forgot about it. ;)

Of course :)

Guus, indeed, I wouldn't rule that out

or I might've done something with query parameters or whatever

It's certainly possible to do partial uploads in a number of ways, but there is no standard that I'm aware of

WE’ve used byte ranges with PUT for many years. Works well. POST is not logical, as it works (implicitly) with dynamic data, but PUT works fine.

probably depends on your server software a lot

But the standard Range header is about the response

So you can receive only a partial response

Ah, RFC 7231 explicitly forbids this: An origin server that allows PUT on a given target resource MUST send a 400 (Bad Request) response to a PUT request that contains a Content-Range header field (Section 4.2 of [RFC7233]), since the payload is likely to be partial content that has been mistakenly PUT as a full representation. Partial content updates are possible by targeting a separately identified resource with state that overlaps a portion of the larger resource, or by using a different method that has been specifically defined for partial updates (for example, the PATCH method defined in [RFC5789]).

ahh PATCH

PATCH makes more sense, but I'm still not aware of any standard defining how that should work

No mention of the Range/Content-Range headers in RFC 5789

(in any case, none of this is commonly supported by libraries, javascript APIs or servers AFAIK) ✏

Because PATCH with a full application/json for example is often interpreted to mean "merge this object with the existing resource"

Not "this is the full desired content, replace it"

So Range/Content-Range could just as easily mean "this is part of a patch", or something

Looks like cigaretteking@xmpp.jp is another spammer

+1, already on the RTBL

MattJ: thanks

I also once implemented gzip compression for PUT uploads, you can do whatever you want, but no standard for it