Greg Ward wrote:
> [me]
> > > Quixote is not currently thread-safe.
> >
> > Could you elaborate just a bit which parts you deem to be unsafe?
>
> 1) quixote.publish maintains a global variable, '_request', for the
>    current request.  This won't work in apps that have multiple
>    concurrent threads, each one processing a separate request.
>    It should be easy to fix by changing that global to a dict
>    mapping thread IDs to request objects.

sounds easy enough. For a starter i'd simply use the morethreading.py
Ian Kos has recently posted to python-dev. It has a nice threadstorage
and advanced locks.

> 2) (new in Quixote 0.5) the Publisher class has an attribute
>    'namespace_stack' that (I think) it uses to track its current
>    location in URL traversal.  This could be tricky to fix.

maybe the Publisher instance could basically be associated with
the requests thread, too?!

> There may be others, but that's all I can think of off the top of my
> head.

certainly there are issues with writing to logs etc.pp. But quixote
is not too big and nicely programmed. it shouldn't be too hard. Hope
to find the time to give it a try soon. (first i want to go for
mod_python/apache2 for other reasons).

> > i guess we have a misunderstanding here. Why would one want to persist
> > *cached* data? In fact for my case every bit of cached data is
> > extracted/computed from a 200 GB readonly reiserfs filesystem.
> > I don't see the point to persist these computations and go
> > through the hell of file-locking and/or using a persistence layer.
>
> Yes, we definitely have a misunderstanding.  We all know that caching
> works at many levels -- from a multi-gigabyte squid cache making web
> access faster to a 512 kB level-1 cache making RAM access from my CPU
> faster.  I don't understand what level you want to cache at!

I must admit that i don't have an overall design, yet.  This makes it
a bit hard to explain :-)

Basically it's a growing music database which has tons of different
logfiles (millions of lines). The web application should be long
running and have some threads sitting on the logfiles and update
statistics. These statistics can be accessed from different users
who can ask for their 'personal topten', 'overall topten of the
last month', 'current streams' and whatnot.

I want to have ONE application process that handles analysis of the logfiles
and the user's requests.  Of course not every request should repeat
the same computations again and again (analyzing all the logfiles etc.).

I think that a (possibly mt) quixote would be a very nice fit.
I'd rather go for trying to make quixote mt-safe than inventing
yet another complicated-to-handle persistence scheme [*] :-)

thanks for yours answers and for the nice software!

    holger


[*] funny enough, in another project i *am* inventing yet another
    storage scheme which aims at fully exploiting reiserfs :-)