On Apr 12, 2007, at 4:00 AM, Andrew Bettison wrote:

> David Binger writes:
>>
>> On Apr 11, 2007, at 5:17 PM, Andrew Bettison wrote:
>>
>>> I just check the file size on every sync and commit.  If it's
>>> increased since we last synced or committed, then other processes
>>> have committed in the meantime.  We read the new transactions from
>>> the last point we reached, up to the new eof.
>>
>> I think we should implement this now for FileStorage when it is
>> opened
>> in read-only mode.  Currently, FileStorage in read-only mode does not
>> get any updates.
>
> You could do that.  If you don't want to worry about the race
> condition
> of one process reading while another appends, then it's pretty
> straightforward.

I was thinking that a read-only storage would not update
the index it holds unless it finds a complete new transaction on
the end of the file.  The readers' view of the world might run
a moment behind the writers', but that would be okay because
they are read-only.

>
> However, I can't see the value of an implementation susceptible to
> that
> race, unless you can somehow make all reader processes quiescent while
> writing takes place (by which I mean prevent them from doing a
> FileStorage.sync() or FileStorage.commit()).

The current read-only implementation is valuable because it allows
a peek into the data of a running server without causing the server
itself to do work.  We use it for example, to scan every instance in
the database and make sure that every instance has the expected
set of attributes and the expected type of value for each attribute.
Update-awareness would just be a small improvement for this.
The purpose isn't really the same as that of SharedFileStorage.

> You'd also have to be
> prepared to terminate all reader processes before packing, and re-
> start
> them after packing, because the reader processes could not know which
> OIDs to invalidate upon reading the packed file.

That's true.  This condition would be the same as it is now for a
read-only storage.  It would be nice to have an update-aware,
read-only file storage that would survive a pack.

>
> I guess what I'm getting at is that, like all things concurrent, it
> ain't as simple as it first looks.

I concur.

>
> I'm happy to make SharedFileStorage publicly available.  It still
> suffers from the index memory overhead issue, which doesn't stop it
> from
> working, just potentially consumes a lot of memory, in proportion
> to the
> number of processes multiplied by the number of objects.  For small
> databases (few objects), it's not such an issue, so SharedFileStorage
> may still have some value to Posix-based Durus users.

I think it would.

>
> Legal question: would shared_file_storage.py be considered a
> "derivative
> version" of Durus under para. 2 of its license?  In other words,
> must it
> and its derivatives bear the CNRI copyright message?

I honestly don't know and I'm not qualified or authorized to answer.
But, it sounds to me personally like this not a derivative version
of Durus unless you are distributing it with Persistent and Connection.

>
> --
> Andrew Bettison