durusmail: quixote-users: twisted performance with quixote.
twisted performance with quixote.
2004-04-09
Jason Sibre (2 parts)
2004-04-09
2004-04-09
2004-04-09
2004-04-09
2004-04-09
2004-04-09
Re: twisted performance with quixote.
2004-04-09
2004-04-09
twisted performance with quixote.
Jason Sibre
2004-04-09
Hey folks

I've been working on some problems I've been experiencing for a while with
twisted.  When I'm serving a quixote app with twisted, it 'feels' slow, but
when I test with ab, it looks just as fast as anything else (medusa, apache
w/ fastcgi).  After much testing this way and that, (I've been jacking with
this for a few weeks now, on and off) I concluded (erroneously) that
http/1.1 was the culprit.  When using HTTP/1.1, twisted drops into, what I
like to call: step-and-fetch-it mode.  Slow motion.  About 5 or 10 requests
per second.

Well, as indicated, I've since determined that HTTP/1.1 isn't really the
culprit.  My current suspicion lies with Keep-Alives, but I'm not pinning
the tail, just yet.

Anyway, Twisted doesn't have this problem on it own (it'll just about toast
your NIC, left to it's own devices!)...  It's only when Quixote is involved
that things get slow (but as I've already hinted at, I know that it isn't
Quixote, per se, that's causing the problem, because it runs fine under
Medusa, Apache, etc..), so I've been looking long and hard at
quixote.server.twisted_http, and the twisted internals.  I'm working on a
patch for that file to boost performance, but first:

I'd like to know if anyone else can duplicate this problem.  I'm attaching a
tarball that has a test script and some content which will allow someone to
extract to a dir, run the test script in it, and test three modes: twisted,
twisted w/ quixote using twisted_http, and twisted w/ quixote using what
I've come up with so far to replace what's in twisted_http.

Once the server is running, you just point a browser at
http://hostname:port/index.html  (port is currently set to 8082, but's it's
a piece of cake to change) and look at the resulting page and see how long
it takes to do a full reload*.

It's important that it be a full reload, otherwise the IMS headers will
prevent the process from fully taking place (and it'll always look really
fast).  You should be able to see the images popping up onto the screen as
they're being loaded, so you'll get a bit of an animation effect.  If you
don't notice the animation effect, either it's not a full reload, or you
have a REALLY fast setup.  I'm using a 1Ghz machine, and even at it's
fastest (twisted), this server setup takes about a half-second to fully
paint the page.  At it's slowest (oldqx) it takes about ten seconds.

Of course, for this script to work, you need to have twisted and quixote
installed.  I have twisted 1.2, but I don't know if it has to be 1.2.

I'd REALLY apprecciate it if anyone who has twisted and quixote installed,
and has a few minutes to spare to screw with this, would try it out and let
me know: a) if they see the difference in performance between 'oldqx' and
'twisted' (modes the program can run in) and b) if 'newqx' seems to give
performance more on par with 'twisted' (though it's still a bit slower).

Thanks in advance,

Jason Sibre

* Full reload:  i.e., without honoring cached content/IMS headers, etc.
This is generally accomplished with shift-f5, ctrl-f5, ctrl-shift-r, etc...
Depends on the browser.  It's the reload that takes the longest :)  I've
documented Moz and I.E. in the test script, if you have a hard time figuring
it out for one of them.




reply