On Oct 20, 2005, at 10:04 PM, David Binger wrote:
> On Oct 20, 2005, at 3:54 PM, mario ruggier wrote:
>>
>> I have added this to the processing of each item in the loop:
>>
>>             if item._p_is_saved():
>>                 item._p_set_status_ghost()
>>
>> But, even with this, the "size" in the shrink_cache()'s
>> (len(cache.objects)) grows as fast as the total number of items
>> processed, as previously reported.
>
> Ghosting objects does not remove them from the cache.
> It does, however, reduce the memory required to hold them
> in memory.  A ghost object's __dict__ is empty.

Thanks. Fwiw, i have tried running the generation process again. As the
10,000 object cylces accumulate (shrink_cache() is called at end of
each), the virtual memory of the process seems more stable, but
nevertheless continues to grow. And, on this machine with limited
resources it does eventually lose it...

Just for curiosity, here's a comparison of a few 10K object cycles of
the loop, as per the durus log output (for the cycles from 270K to 340K
objects in the container) for the loop using connection.commit() and
connection.shrink_cache() respectively.

using commit():
[667] shrink 0.897611s aged 7295 removed 9337 ghosted 6060 loaded 57404
size 86779
[667] shrink 0.812353s aged 7408 removed 8229 ghosted 6503 loaded 58099
size 89903
[667] shrink 1.354192s aged 6063 removed 11599 ghosted 4846 loaded
56537 size 89633
[667] shrink 0.636280s aged 5099 removed 15223 ghosted 3378 loaded
50931 size 85769
[667] shrink 0.361344s aged 7905 removed 15492 ghosted 741 loaded 49898
size 81597
[667] shrink 0.614141s aged 5064 removed 13516 ghosted 4384 loaded
49740 size 79434
[667] shrink 0.739988s aged 6705 removed 10252 ghosted 3785 loaded
51460 size 80566
[667] shrink 0.556780s aged 6957 removed 9608 ghosted 4299 loaded 53912
size 82314

using shrink_cache():
[4418] shrink 1.065533s aged 10596 removed 1153 ghosted 361 loaded
273261 size 274077
[4418] shrink 0.925361s aged 10390 removed 538 ghosted 131 loaded
283362 size 284226
[4418] shrink 0.877733s aged 10530 removed 220 ghosted 393 loaded
293430 size 294668
[4418] shrink 25.604914s aged 10125 removed 478 ghosted 217 loaded
303456 size 304883
[4418] shrink 6.038403s aged 11788 removed 599 ghosted 58 loaded 313710
size 314938
[4418] shrink 1.898148s aged 11488 removed 488 ghosted 192 loaded
323971 size 325166
[4418] shrink 3.095931s aged 9271 removed 669 ghosted 455 loaded 333874
size 335184
[4418] shrink 1.274971s aged 8591 removed 886 ghosted 9 loaded 343766
size 344987