-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
David Binger wrote:
> Would your application be crippled by dropping those
> references, as below, before releasing the lock?
The code is several thousand of lines long. Moreover, the "issue" would
be still present, ready to beat any other multithread usage of Durus.
> Better, write a lock-protected get_value()
> function that returns the value, and no references
> to persistent instances.
That was my original workaround try, but see last paragraph :-(.
> If I remember correctly, your database is cycle-free,
> so the gc garbage collector probably isn't doing anything
> for you, or causing trouble. If it were, you
> could call gc.disable().
"gc.disable()" disables the garbage collection of cycles, but this is
not an issue here. I'm talking about the reference counter when I say
"garbage collection".
I am testing a WeakValueDictionary subclass implementing my first
suggestion: take note of the deleted objects, but keep them in the
dictionary until "cache.shrink()" calls "dictionary.shrink()".
Would you be interested in the patch?. No noticeable performance hit.
We only need to add a "dictionary.shrink()" in "cache.shrink()". The
diff could be:
=====
- --- connection.py.old Thu Sep 21 21:25:39 2006
+++ connection.py Thu Sep 21 23:26:05 2006
@@ -16,6 +16,22 @@
from time import time
from weakref import WeakValueDictionary, ref
+
+class weak(WeakValueDictionary) :
+ def __init__(self, *args, **kw) :
+ WeakValueDictionary.__init__(self, *args, **kw)
+ def remove(wr, selfref=ref(self)):
+ self = selfref()
+ if self is not None:
+ self._to_delete.append(wr.key)
+ self._remove=remove
+ self._to_delete=[]
+
+ def shrink(self) :
+ while len(self._to_delete) :
+ del self.data[self._to_delete.pop()]
+
+
ROOT_OID = p64(0)
class Connection(ConnectionBase):
@@ -310,7 +326,7 @@
class Cache(object):
def __init__(self, size):
- - self.objects = WeakValueDictionary()
+ self.objects = weak()
self.recent_objects = set()
self.set_size(size)
self.finger = 0
@@ -386,6 +402,7 @@
"""(connection:Connection)
Try to reduce the size of self.objects.
"""
+ self.objects.shrink()
current = len(self.objects)
if current <= self.size:
# No excess.
=====
The nice property of this class is that it carefully coded to not
require any locking (lists are thread safe) and is as fast as
"weakref.WeakValueDictionary", so it seems a "safe drop-in", even in the
single-thread case.
If you are not sure (I dislike it, also) about using private interfaces
(the "self._remove"), the simple path would be to clone the entire
"weakref.WeakValueDictionary" source code (it is small) and creating a
"durus" version, with the previous patch. Or far more simple code
implementing only the methods used in durus. I guess that implementation
could be faster that current "weakref.WeakValueDictionary" code, and
fairly simple to do :-)
I know that you are not interested in multithreading issues, but would
you mind to integrate a patch like this in standard durus?.
PS: Now, I just think I could simply "overload" the
"WeakValueDictionary" internal callback to force it to get the lock.
Deadlocks could be avoided if I change my global lock to a "reentrant"
lock. Let me try... Argg, reentrant locks are ten times more costly...
So I keep my previous patch :-p
I add this here as "dead-end" documentation.
- --
Jesus Cea Avion _/_/ _/_/_/ _/_/_/
jcea@argo.es http://www.argo.es/~jcea/ _/_/ _/_/ _/_/ _/_/ _/_/
jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/
_/_/ _/_/ _/_/ _/_/ _/_/
"Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/
"My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iQCVAwUBRRMGwplgi5GaxT1NAQIjegP7By45iYea+DGaRQxeo2YO8Dq3yTflcktN
6Ak4IoHScHKgfTTP4+e5Nz+2I0+yNStEKcWiiuNmI8FP0yOzc6Afduzmz0eD3kim
hRbqJl8pHyJTm7cm8jpJy8jpD2PJCIYYBQ0DDmrKmMYE1TKh0g3SPDR/mPCfy5T5
/Xp3wKgogYI=
=JH+f
-----END PGP SIGNATURE-----