The code:
# return the entire file
if 'wsgi.file_wrapper' in environ:
# Return env[wsgi.fw](file, block size)
return environ['wsgi.file_wrapper'](the_file , 1024)
else:
return iter(lambda: the_file.read(1024), '')
can be a problem for a couple of reasons.
Now although you have simply copied this from the WSGI specification itself, the use of iter/lambda is arguably bad practice as it doesn't ensure that the file object is closed promptly when consumed. Instead when the file object is closed is dependent on the particular Python interpreter implementation and so may not happen immediately and may only happen when the garbage collector reclaims it.
The end result is that one possible reason for running out of file descriptors is that the file objects aren't being closed in time before you run of file descriptors.
The better way of writing the above code is actually:
# return the entire file
if 'wsgi.file_wrapper' in environ:
return environ['wsgi.file_wrapper'](the_file , 1024)
else:
def file_wrapper(fileobj, block_size=1024):
try:
data = fileobj.read(block_size)
while data:
yield data
data = fileobj.read(block_size)
finally:
fileobj.close()
return file_wrapper(the_file, 1024)
What this is relying on for the prompt explicit closing of the file is that when the WSGI server consumes the generator, the finally block will be executed and so the file will be closed.
Even if the generator isn't completely consumed, then the WSGI server is required to call close() on the generator and in doing that, then the finally block will be executed and the file closed.
When you use:
iter(lambda: the_file.read(1024), '')
there is no explicit closing of the file and so you are at the mercy of the interpreter as to when it happens.
Now the second reason there could be a problem is that if the WSGI server does actually implement wsgi.file_wrapper, it doesn't mean that how it is implemented is correct, with it ensuring that the file is closed. It itself could have simply used the iter/lambda trick and be broken.
A further problem is that there have also been various WSGI servers that didn't always call close() properly on the iterable returned from the WSGI application when the request completed, especially in the case of an exception occurring part way through consuming the response. Examples of WSGI servers which have had this problem at certain times are the wsgiref WSGI server in the Python standard library, the Django development server and uWSGI.
I have never looked at bjoern to see whether it implements the WSGI specification correctly, but I am certainly not surprised these days when I find WSGI servers that are broken.