Regarding starting this Off Topic thread, I didn't expect the Spanish
Inquisition... but its been interesting just the same.
At 11:43 AM 1/7/2003 -0500, Tom Jenkins wrote:
>Anybody looked at these:
>http://slarty.polito.it:8069/~sciasbat/wiki/moin.cgi/StandaloneZodbHowto
This looks interesting because it incorporates the full text indexing
capability provided by the Zope 'product' TextIndexNG. I asked the author
of the wiki for an example of use and see he has posted an example. I
might pursue this one day but my interest is more for the text indexing
capability.
>http://www.zope.org/Members/kelcmab3/catalog_out_of_zope (old)
Nope
>http://www.async.com.br/projects/IndexedCatalog/
Yes, built some test scenarios. Easy to use. Can add a lot of overhead even
if indexing a few object attributes. Can imagine a system that relies on
it heavily _might_ have performance issues.
In my example scenario, a simple User object with three indexed attributes
(id, email, last name) / 50,000 objects, and some test data I found the
ZODB collection was approximately triple the size and retrieval performance
was poor relative to the plain ZODB approach. I don't have the numbers from
my tests saved but something like the following was observed:
The test simply accessed x number of keys, in a random fashion.
IC Plain ZODB
ZODB Data.fs size 35MB 14MB
Record retrieval- 1st pass
Retrieve 2000 25s several seconds
Retrieve 200 3s subsecond
Retrieve 20 .07 .0007
Again, going by memory on this -- but these results are examples of the
order of magnitude difference. What was more notable was that with
IndexedCatalog subsequent queries did not appear to benefit from ZODB's
client side cache. I did not however spend much time looking at this
further nor did I change the default cache size from my recollection.
Probably jacking the cache up would have made a measurable difference and I
may redo this in a spare moment.
My conclusion was that I could use IndexedCatalog if I had the need but
felt that I should try to live without such mechanisms and see how far I
could go. Plus I was keen to do so to see how far I could move my thinking
away from SQL.