* mario ruggier wrote on [2007-09-11 17:52:47 +0200]:
> Recently, on the Durus mailing list, Mike announced experimental sqlite and
> postres storages for durus
> [http://mail.mems-exchange.org/durusmail/durus-users/887/]. Iiuc, as they
> are these would anyway not help to meet your requirement (of not having
> another server process), but I wonder if, with some tinkering, there'd be
> any possibilities there.

I've got an improved release of the Postgresql storage /
DBAPIStorage classes - that, and a Sancho test file are attached.

After switching DBAPI layers away from psycopg2 to something that supports
prepared statements and pg's COPY statement, I rather like this version,
performance wise.

After doing some stress testing with sqlite and Postgresql, for a
(very) large storage, I think I'd probably go with Postgresql over
sqlite if ShelfStorage wasn't the right choice.

One benefit of the Postgresql storage is that pg has a very fast
bulk import mechanism (the COPY statement); if you ever need to load
many (millions) records on anything more than a one-off basis, it can be
a real help.

You also could have a Postgresql process running on another box
entirely, which opens up some other avenues for scalability although
I've not looked at latency or performance in that scenario at all.

Still for me it was primarily a learning experience; I understand
the insides of Durus slightly better (not that I really needed to
for what I do) and it was fun to do. I can't possibly imagine
writing a storage back end for ZODB ;-) so writing one for Durus was
fun in and of itself.

But for the original question, David's limit child approach makes
the most sense. I keep meaning to think more seriously about hosting
issues for QP; perhaps may offer that as a service one of these
days.

Mike