durusmail: durus-users: My wish-list for Durus 3.6 (20061031)
My wish-list for Durus 3.6 (20061031)
2006-10-31
2006-10-31
2006-10-31
2006-10-31
My wish-list for Durus 3.6 (20061031)
Jesus Cea
2006-10-31
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

This is my final wishlist for Durus 3.6.


As ever, I'm ready to help to implement this, if you ask.


1. Be able to play with threads in the same "safe" ways that Durus
pre-3.5. (would be nice if available in durus 3.5.1 :-). The patch sent
by David when 3.5 came out seems to work fine.

2. Fully implement "self.storage.sync()" support in
serverstorage/filestorage. This implementation (currently partially
done) would allow:

- - Notify clients of objects garbage collected, to update their cache to
a) reclaim memory early and b) to avoid "resurrecting" a dead object
(for example, keeping (incorrectly) a nonpersistent reference around).

- - Several serverstorage instances in a multithread server could share a
single backend instance (if it is thread-safe). You could use a server
instance for client, for example, allowing multiple read requests in
parallel (if the backend allows). The syncronization point would be the
backend (inside the "begin" method), so commits would be serialized
correctly.

- - Be able to "replicate" a durus storage via the backend ability to
propagate changes to other (remote) processes. For example, BerkeleyDB
supports replication natively.

- - Be able to deploy durus "cache only" storageservers: the storage
backend would be a durus client of another "real" durus server. That
backend should be able to propagate invalidations upstream to its clients.

- - Be able to implement multithreaded filestorage processes, sharing a
single backend but keeping separate caches/connections to it. Remember
that a single storage pool can be only accessed by a single filestorage
instance. This improvement would allow to open several filestorages to
the same data. Useful if you have a multithreaded apprication and can't
share a single filestorage instance (very problematic with current durus
3.5).

3. "connection.get()" does two database requests: one to get the object
type and to create the ghost object in cache, and other access to
actually populate the cache when the object is "touched". This is not
usually a problem because the only object requested two times is "root";
all the others are reachable from there, and when traversing links we
already know the next object type (in fact, that object would already be
a ghost).

Nevertheless I sometimes use "connection.get()" like a "weak reference"
between objects. That is, instead of keeping an object link, I keep an
object OID and then I try to access it just like a python weak
reference. This "weak reference" doesn't preclude the object of being
garbage collected, like usual "weak references". The usage case is the
same that python weak references: keeping references to objects without
precluding garbage collection.

I feel that needing two database accesses is an actual bug masked by
current durus usage profile, but revealed by my "weak ref" pattern.

4. "connection" objects should provide a method to query how many cache
misses we had since that connection instantiation. With this
functionality, client could tune its cache size "automagically". To be
able to see "accumulated" idle time waiting for a remote object to come
would be nice too.

5. A "mutable data changed" notificator for BTree.

6. Add a "close" method to storage backend interface. This could be able
to do things like file locks cleanup, background thread stopping, or
replication exclusion.

7. "Factorize" the durus server socket management (in particular, socket
creation for incomming connections and socket "select") to be able to
reuse the server code in other communication media, like shared memory,
intraprocess queues or mmaped files.

8. A precompiled DURUS distribution for Windows users. Please!.

9. Be able to raise a "read only" exception when a client request new
OIDs or try a commit with objects changed. This would allow for
read-only connections, without setting the entire storage "read-only".
Also, currently if an storage is read-only, clients will be disconnected
when trying to commit changes, with no real indication of the problem.

- --
Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
jcea@argo.es http://www.argo.es/~jcea/ _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea@jabber.org         _/_/    _/_/          _/_/_/_/_/
                               _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz





-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBRUeA/Zlgi5GaxT1NAQIEKgP+OFLnn6I+l3OHHXDoMTH1S3C7CWys1pw1
jPmEoHx5GlACM51Wc3t/0hhN8xQ6gjDoMGbGr0ZWk2rA6/yT3n3LaD7s5nat5EgD
iLgRabU5Cc4qL/OEpK7jW7Dk0MhtDvJXs6gOcOgx53/uZy6ArEJml7h1TvuHcOaX
ZN+ctbK8MZs=
=Z+CB
-----END PGP SIGNATURE-----
reply