durusmail: quixote-users: [PATCH] twisted_http.py
ANNOUNCE: Quixote 1.0b1 released
2004-04-12
2004-04-12
2004-04-12
[PATCH] twisted_http.py
2004-04-13
Jason E. Sibre (2 parts)
2004-04-14
2004-04-14
[PATCH] twisted_http.py
Jason E. Sibre
2004-04-13
> Version 1.0b1 of Quixote is now available:
Woo Hoo!

Well, I'm sorry I didn't get this in last night, but I got lazy.  Had I
known 1.0 was a few hours away, I woulda burned some midnight oil.  Well,
better now than later:

This patch for server/twisted_http does a few things:

- Wraps Quixote output (whether it's a string or Stream) with a Twisted-
  style producer, which:
        a) Improves/stabilizes (*) performance of Twisted / Quixote in
           normal usage.
        b) Improves (reduces) memory utilitzation when transfering large
           StaticFiles (because the Stream is used properly, rather than
           getting sucked into RAM)
- Updates the demo/test code at the bottom to:
        a) Use the reactor; 'Application' has apparently been deprecated
           (at or before Twisted 1.2.0)
        b) Demonstrate SSL usage of twisted.


Jason


(*) Stabilizes... (This is not required reading, but may be interesting to
some)
Under some conditions, Twisted can have very poor performance with the
current (unpatched) twisted_http.py.  It seemed to only happen on Linux, and
not all clients would demonstrate the behavior.  But in what may well be the
mos common case (Twisted/Qx on a Linux box, client is MSIE on windows),
performance could go as low at 5-10 requests per second.  I had a hell of a
time quantifying this (because many testing tools didn't exhibit the
problem), but it was blatantly obvious something was VERY wrong when viewing
a page with lots of small images on it with MSIE(6.0) or Mozilla(1.7b)
running on a W2K machine.
The reason I say stabilizes is that it doesn't always help.  It does correct
the behavior I just descibed, but on some client/OS mixes, (that don't have
the above problem) you may actually see a slight (~3%-6%) performance
degradation by using the producer... This is slight enough that it's not
noticable, unless you're already at the limit of scalability, so I'm not too
worried about it.  I've tried to get that taken care of, but I'm already way
beyond the point of diminishing returns.  I spent lots of time watching
packets with Ethereal to figure out what was going wrong, and spent lots of
time digging through Twisted's labrithine source code.  I still have one
gray area (why would the windows TCP stack sit on an ACK for over a tenth of
a second?!), but using the producer approach sidesteps the whole issue.  As
much as possible, anyway.

Jason
reply