David (and anyone else interested),

I'd like you to nitpick this idea I have to add server-less, multi-
process concurrency to Durus.

Goal:  Allow more than one process and/or thread on a single machine
to reliably access and write to a Durus file, without having to
maintain a separate process for a Durus server, and without having to
install more Python packages.

I'll present several actions that a process or thread might take, and
how Durus would behave to support those actions.

Please let me know if you think this will work, any details or quirks
you can think of that might get in the way, etc.  In return I will put
some time into this to see if I can produce a working patch.

== Lock file ==
Kept as a separate file, "mydatabase.durus.lock" being locked only
during write operations.
This is to allow all processes to at least read up to a "known good"
EOF marker while another process is writing a transaction.

== Packing ==
I haven't thought through this part as of yet, but I'm not worried
about it at the moment.

== Initial opening of a file ==
1. seek(SEEK_END), tell(), keep as current EOF offset
2. read initial state from file, create in-memory state

== Retrieve from a file that has not been updated ==
1. seek(SEEK_END), tell(), offset is the same as the current EOF offset
2. read requested object

== Retrieve from a file that has been updated ==
1. seek(SEEK_END), tell(), offset is different from the current EOF
offset
2. read records from current EOF offset to new EOF, update in-memory
state
3. read requested object

== commit() objects to a file that has not been updated ==
1. acquire exclusive lock
2. seek(SEEK_END), tell(), offset is the same as the current EOF offset
3. write records for new objects, update current EOF to new EOF
4. upon commit or rollback, release lock

== commit() objects to a file that has been updated ==
1. acquire exclusive lock
2. seek(SEEK_END), tell(), offset is different from current EOF offset
3. read records from current EOF offset to new EOF, update in-memory
state
        a. if conflict, raise WriteConflictError
        b. otherwise, write records for new objects, update current EOF to
new EOF
4. always, release lock

== Perform a consistent read across a data set ==
1. call pause() on the Durus connection
2. seek(SEEK_END), tell(), if offset is different from current EOF
offset, read records and update in-memory state
3. read requested objects, never doing seek/tell dance
      ... some time later ...
4. call continue() on the Durus connection


Thanks!

- Matt