durusmail: quixote-users: C stack depth problems from PTL importing.
C stack depth problems from PTL importing.
2002-04-16
Speaking of ptlc files
2002-04-16
2002-04-18
C stack depth problems from PTL importing.
Titus Brown
2002-04-16
Dear Quixote folk,

I'm running into an obnoxious problem caused by running Quixote in a
threaded environment under Linux, and I'm hoping someone has a suggestion
for a clean fix ;).

As you know, Quixote redefines the import hook to allow the import of
.ptlc files.  This means that on each successive recursive import, several
different Python functions are called.  Since each Python function call
requires several C stack frames, using Quixote automatically means a
significant increase in the amount of stack space used on package imports.

This is no problem for non-threaded applications under Linux, where the
stack grows dynamically (with an upper limit specified by ulimit).

However, it *is* a problem with threaded applications, which have an
upper limit on the stack size that is (of all things) hardcoded into
glibc.

I've been having problems with this in big packages (both of my two current
Quixote projects, in fact), and I can fix them by "pre-caching" imports, i.e.
making the imports shallower by importing deep nodes at the top level.
This is ugly and breaks easily ;(.

I wouldn't mind rewriting the Quixote PTLC import hook in C, if anyone thinks
that would make a difference...

--titus

p.s. Note that this is a known problem in Python and Java-land.  It's just
not usually an issue for Python apps.


reply