On 19/03/2007, at 10:04 PM, David Binger wrote:

> It looks like a good change to me.
>
> I'm curious about this sqlite connection.

I've been working on and off on a storage engine based on sqlite and
just recently have had need to get more progress on it.

The main benefit vs. file_storage is that it starts up instantly with
no need to build an in memory index. It currently counts records at
start up but I've had versions without that if the object count gets
above many hundreds of thousands and doing a select(*) to get_size()
takes too long.

A nice side benefit is that I was very easily able to make a sync
script that copied the contents of one database to another for backup/
redundancy. What it does is keep track of a transaction id so that it
can do a very quick attach of the backup database, then copy all
records not in the backup and close. This is safe since it is in a
transaction where copy or rsync require some outside co-ordination.
There are a couple of corner cases that the sync script tries to
detect and bail out on and overall the script is a bit rough but
confirms the idea for us.

I've attached the sqlite storage engine and sync script if you'd like
to have a look - definitely not as clean as I'd like but passes
testing. Any input would be appreciated.

Peter Wilkinson
pw@thirdfloor.com.au