durusmail: durus-users: Re: Anybody willing to try BerkeleyDB Storage?
Anybody willing to try BerkeleyDB Storage?
2006-03-16
2006-03-16
Re: Anybody willing to try BerkeleyDB Storage?
2006-03-19
2006-03-20
2006-03-20
2006-03-20
2006-03-21
2006-03-21
2006-03-21
2006-03-21
2006-03-21
2006-03-21
2006-03-21
2006-03-21
2006-03-21
2006-03-21
2006-03-21
2006-03-21
2006-03-21
2006-03-21
Re: Anybody willing to try BerkeleyDB Storage?
Jesus Cea
2006-03-21
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

David Binger wrote:
> I understand.  And, the fact that your application works is impressive.

The transparence that durus brings to the table, make it possible. More
time to properly design the application. That is unvaluable. Thanks for
such a fine product.

With current design and the berkeleydb storage backend, I have inmense
scalability, only limited by disk space and disk speed. In fact, my
backend storage could do possible to replicate a durus database using
the build in transactional replication features of Berkeley DB.

My worse problem now is the single threaded durus server. That is, if a
client does a request, durus server won't serve any other request until
it finishes the first one. When you have tens/hundreds of requests per
second, you have a problem. When you have clients writing, the world
stops :-).

My current approach is to partition users at random between different
durus storages in orden to be able to obtain some paralelism. A bit
ugly, but it works.

> I asked about references between mailboxes before because I was
> wondering if
> you would benefit from using a separate database for each user, or maybe
> some partition by groups of users would be practical.
> I was also wondering if your message bodies are ever edited.
> Is it important to keep them in the database?

Messages are NOT edited in any way. Only a few touches in headers (that
I keep aside), like read/unread, message UID, and so on.

I like to be able to keep the messages inside the database, instead on
keeping there only a reference because I had race conditions and
collitions when using previos filestorage backend and references to
external files. Things like conflicts in the durus storage are very
complicated when you have things "outside". You must manage conflicts by
hand. Ugly and error prone.

I programmed the BerkeleyDB storage to be able to keep such a huge
number of objects of big (sometimes > 50Mbytes) size in an efficient and
"unified" way. It is doing a very fine work. In fast, as I already
wrote, my current "complain" is that durus storage server is not able to
feed the backend fast enough to keep the disk busy :-)

- --
Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
jcea@argo.es http://www.argo.es/~jcea/ _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea@jabber.org         _/_/    _/_/          _/_/_/_/_/
                               _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBRCA7Nplgi5GaxT1NAQI4nwP/VL4IKKyoQZmlZn8UMhe8B9WquMtKd4TW
y5QvEN7Zp/p5pl/Q4KhMJIqSTzkcPdFKx0J5mgaiyvKMHgaMVQqn9yrnayRRKQjx
kMOVF/k6OAeJjGbxdvFu0LptQtX+GzSNtgQ+dHOuq8YMFY0UXk7mbv8RC7w1tvzw
z++LSmdXKBE=
=9vUY
-----END PGP SIGNATURE-----
reply