Hi folks,

Using Quixote on Medusa to serve up static files, I was getting really terrible
performance when the file sizes started to increase. CPU would shoot through
the roof on the server, and througput downgraded very quickly.

I've got a patch that seems to solve the problem... for me, anyway. Instead of
pushing the content of the response to Medusa as a string, I push a Medusan
file_producer, wrapped around the already-built StringIO, output_file.

Performance for me was improved significantly. Here's a snippet of my
comparison (using apachebench, with both apachebench ("ab -n 5 ...") and the
server running on localhost, Windows XP):


content         reqs/sec,             reqs/sec,
size (KB)      unpatched               patched

11                124.82                124.82
21                124.82                124.82
51                 55.48                 83.21
101                27.74                 41.61
191                14.27                 33.29
281                 6.93                  20.8
371                 3.62                 17.22
461                 2.32                 14.27
551                 1.65                  9.99
641                 1.17                  8.76
731                 0.89                  7.56
821                 0.68                  6.75
911                 0.55                  6.16
991                 0.47                  5.74

As you can see, the patched version never underperformed the unpatched, and
markedly improved as content size increased.

I'm looking for willing victims using Quixote/Medusa on other platforms to test
the patch. A quick anecdotal test would be fine, but if you're willing to set
up something more rigorous, that would be great.

For my test code, I ran a Quixote server with a simple application:

     _q_exports = []

     def _q_index(req):
         try:
             rlen = int(req.form['size'])
         except:
             rlen = 1 << 21   # (2 MB default)
         req.response.set_content_type('application/octet-stream')
         resp = '*' * rlen
         return resp

and a script to drive ab (with n=5), with increasing size values in the URL
(e.g. http://localhost:8080/?size=1024 would return 1KB of content).

Here's the patch:

--- \projects\Quixote-0.6\server\medusa_http.py Wed Jun 04 12:45:35 2003
+++ server\medusa_http.py       Wed Jun 04 12:52:36 2003
@@ -13,4 +13,5 @@
  from StringIO import StringIO
  from medusa import http_server, xmlrpc_handler
+from medusa.producers import file_producer
  from quixote.http_request import HTTPRequest
  from quixote.publish import Publisher
@@ -106,5 +107,7 @@
          if output:
              request['Content-Length'] = str(len(output))
-            request.push(output)
+            output_file.seek(0)
+            out_producer = file_producer(output_file)
+            request.push(out_producer)

          request.done()



Thanks,

-- Graham