I think there might be a bug in load_state in connection.py - shouldn't it also catch KeyError to handle the case when working from a local FileStorage instead of a ClientStorage (as the traceback below shows)? Regardless, that's a case that shouldn't have happened in our app. In looking at the Shelf packer in Durus 3.8, it seems a bit more sophisticated than the one in 3.7. I'm guessing that an upgrade to 3.8 is likely to prevent this corruption? Thanks. Dave On Aug 14, 2011, at 11:44 AM, David Hess wrote: > > After some more investigation - this occurred right after live packing the database and changes to this BTree were made and committed during the pack. > > Afterwards, the oids seem to be jumbled up in this BTree (at least - maybe elsewhere in the database too). Unpickled Persistent objects are not what they should be - interior BNodes are sometimes application classes and stored values are sometimes BNodes rather than application classes. > > Dave > > On Aug 14, 2011, at 10:45 AM, David Hess wrote: > >> >> We have a long running process that does a lot of work with BTrees and in the middle of doing an "in" operation, we got this traceback: >> >> File "/usr/local/lib/python2.6/dist-packages/durus/btree.py", line 343, in __contains__ >> return self.root.search(key) is not None >> File "/usr/local/lib/python2.6/dist-packages/durus/btree.py", line 93, in search >> return self.nodes[position].search(key) >> File "/usr/local/lib/python2.6/dist-packages/durus/btree.py", line 93, in search >> return self.nodes[position].search(key) >> File "/usr/local/lib/python2.6/dist-packages/durus/persistent.py", line 173, in _p_load_state >> self._p_connection.load_state(self) >> File "/usr/local/lib/python2.6/dist-packages/durus/connection.py", line 182, in load_state >> pickle = self.get_stored_pickle(oid) >> File "/usr/local/lib/python2.6/dist-packages/durus/connection.py", line 111, in get_stored_pickle >> record = self.storage.load(oid) >> File "/usr/local/lib/python2.6/dist-packages/durus/file_storage.py", line 96, in load >> raise KeyError(oid) >> KeyError: '\x00\x00\x00\x00\x00\x00T\xb4' >> >> This is using shelf storage on Durus 3.7 and is a long running process. My best guess is a ghosted BNode that thought it was persisted to disk but really wasn't? I think I've seen this once before a couple of years ago (i.e. it seems to be really rare). >> >> Is this new and/or known and fixed in Durus 3.8? >> >> Dave >> > > _______________________________________________ > Durus-users mailing list > Durus-users@mems-exchange.org > http://mail.mems-exchange.org/mailman/listinfo/durus-users > ------ David K. Hess 877.343.4947 x114 dhess@fishtechnology.com