durusmail: quixote-users: Thread Safety & Multiple Publishers (Re: long running requests (Re: question about twisted_http))
Thread Safety & Multiple Publishers (Re: long running requests (Re: question about twisted_http))
2004-07-16
Thread Safety & Multiple Publishers (Re: long running requests (Re: question about twisted_http))
Aaron Brady
2004-07-16
Hi,

I know I posted a hopelessly optimistic Threaded Medusa, but I'm actually
trying to make a proper go of it at the moment.

My situation is that I would like to use threads for long running
requests. I think that using Titus Brown's code from
doc/multi-threaded.txt alongside simple thread spawning as below should
work fine for that.

Also, I want to have multiple publishers in one process. I've written a
few applications which use different configs and session stores, and want
to be able to run them all under one daemon.

I use Apache mod_rewrite and mod_proxy, alongside medusa. In this case,
you get URLs like:

   http://application-server:8080/application1/...
   http://application-server:8080/application2/...

(which are rewritten into http://www.application1.com/...)

For the session storage I would consider writing my own subclass of
SessionPublisher which actually checks the path of the application etc.,
but that seems like a little bit of a hack and doesn't get me multiple
configs.

Given that, I've decided that having multiple QuixoteHandlers with medusa
should fit my needs. I can then override match with something like:

def match(self, request):
     path, params, query, fragment = request.split_uri()
     return path.startswith('/application1/')

I've looked at overriding "quixote.publish.get_publisher()", but the
approach Titus used of using thread.get_ident() wouldn't work, because
several threads will share on Publisher instance, and the main thread
would need access to all of the Publisher instances.

However, get_publisher() appears to only be used in a couple of places in
Quixote, session.py and sendmail.py.

All of the functions in session.py have access to a request object, so if
I were to make "publish.Publisher.create_request()" add the Publisher
object to the newly created HTTPRequest, then I could replace
"get_publisher().config" with "request.publisher.config"

That leaves quixote.sendmail. These functions do not get a request object
and so cannot use this approach, but they *do* take an optional config
keyword. The only place that this is called in Quixote (that I can see) is
in "Publisher.mail_error()" - and I can change that one instance to use
a config parameter of self.config.

(And I need to remove the singleton check at the start of Publish())

What I basically want to know is am I heading down a sane path?

I realise that some of the above will almost definately break existing
applications (though, not mine), and I'm willing to maintain the changes
as a separate patch-set if anyone is interested in using them.

Sorry about the rambling mail,

Aaron

http://insom.me.uk/blog/

GF>
GF> In case it got missed, Aaron Brady posted a "mini-pattern: object method to
GF> separate thread" which adapted quixote.server.medusa_http to allow for
GF> long_running requests.
GF>
GF> http://insom.me.uk/blog/2004/07/05/thread-pattern/
GF>
GF> No comment on this; just thought it should be mentioned on the list.
GF>
GF> -- Graham
GF>

reply