durusmail: durus-users: PersistentDict vs BTree
PersistentDict vs BTree
2006-10-03
2006-10-03
2006-10-03
2006-10-03
2006-10-05
2006-10-05
2006-10-05
2006-10-05
2006-10-11
2006-10-11
2006-10-11
2006-10-12
2006-10-12
2006-10-12
2006-10-13
2006-10-13
2006-10-13
2006-10-13
2006-10-13
2006-10-13
Re: PersistentDict vs BTree
2006-10-23
2006-10-31
PersistentDict vs BTree
Jesus Cea
2006-10-05
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

David Binger wrote:
> The answer depends on your objectives.

That is right. There are a lot of things Mike Orr must evaluate: For
example, if any process is modifying the PersistentDict, readers must
fetch it entire again (31 MB), while a BTree will only reload the
modified nodes. Also a PersistentDict commit will write 31 MB to
storage, for example.

> When you load a node of a BTree, you get, by default, a maximum of 16
> values.  If they are non-persistent, they will all get loaded in one
> trip to the storage.  If they are persistent, it will take 17 trips,
> but each one has a smaller load.  I think 1 big trip will *always* be
> faster, but there are other considerations.

The Durus 3.5 "bulk object loader" method would probe useful here. Also,
a future version of my storage backend will use (internally) multiple
threads to fetch multiple objects in parallel, for huge performance win
if you need to go to the disk and slight performance hit if your OS
already has the data cached.

- --
Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
jcea@argo.es http://www.argo.es/~jcea/ _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea@jabber.org         _/_/    _/_/          _/_/_/_/_/
                               _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBRSUhHplgi5GaxT1NAQKqfwP/ScmhlmIhWbsqoYdbn+UNS2vDZgGQe1lo
GEs+d3yIy9stbWEArl3i1R3K7KkIYh4ipQzzcdtEEgM81TfCwEgGYL/KE4s23QR7
9wZU45i8SUBJCYRA/6KjFeAyyEU6rEH4eW9Bw02GBQzDAv2UHyI47QNwxSfsII48
4zB2beWf3L0=
=ul7/
-----END PGP SIGNATURE-----
reply