Can Django support chunked file uploads without a Content-Length header?

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Can Django support chunked file uploads without a Content-Length header?

Nick Sarbicki
I'm working on a Django API which receives audio files and some extra POST data and then processes the file. This was initially in Flask, but Django suits the overall project better so we're migrating. However we've run into a snag.

Currently the audio file is sent via node.js, with the node-fetch library. The important thing about this library is that it does not set a Content-Length header and instead chunks the file, giving us a "Transfer-Encoding: chunked" header.

It seems to me that Django cannot handle this and simply reads the wsgi.input as blank.

To be specific about how this happens Django creates a LimitedStream using the wsgi.input stream and a content-length defaulting to 0 here.

On trying to then read the stream we hit this block due to the limit (the content-length) being 0, which in turn causes this block to be executed rendering the input stream effectively blank.

The effect of the above is to stop Django being able to read chunked files. This occurs both with the development server and with gunicorn.

I did find a seemingly related bug here but it seems specific to GET requests and is applied in middleware - which occurs after we've already set the nulling limit on the LimitedStream.

So I'm left wondering if this is intentional, or should it be considered a bug? If it is intentional, is there a workaround for this? For instance in werkzeug which uses a wsgi.input_terminated flag to tell whether it should just use the raw wsgi.input stream or use its own version of a LimitedStream.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/fb74257e-5f06-4d42-a266-2e694d831187%40googlegroups.com.