->   greg> On 17 March 2003, David Ascher said:
->   >> Has anyone come up with a way to get quixote modules to get reloaded
->   >> when they've been modified when running under mod_python?
->
->   greg> I have been down that road several times in the past, and it
->   greg> *always* ended in tears.  Module reloading is a convenient little
->   greg> hack in an interactive interpreter, but in a real-world long-lived
->   greg> process, it is always doomed to failure.  Always.
->
-> It works in -- among others, I'm sure -- Erlang, Smalltalk, and Common
-> Lisp long-running processes; I'm curious about which Python features or
-> semantics (or implementation details, for that matter) prevent it from
-> accomplishing this attainable goal. (It really is a very nice capability
-> to have, obviously.)

Hi, Kendall,

the Python module system is kind of hacked together, and (as I found out with
PyWX) isn't really designed for long-running Python systems with multiple
interpreters and processes.  It works very well, mind you, but it's hard
to munge in a consistent way.

In particular,

* there is an underlying caching layer that is per-Python-instance, not
        per-interpreter or per-thread; a cached version is initialized for
        each newly imported module.

        On initial import, and (I think) on 'reload', top-level statements
        are executed.  The results of these statements are cached and
        used to create the per-interpreter copies.

* there is a separate caching layer for each interpreter; this is why you
        need to do a 'reload'.

* modules are mutable, but changes are not passed across interpreters.

So, if you're willing to assume that modules are read-only after import,
and that functions executed on import are time-invariant, you can 'reload'
modules with impunity.

I'm not sure how this would interact with PTL stuff, but I assume it's no
different.

The situation is complicated by the multiple strategies that are used for
persistent Python interpreters:

* CGI uses non-persistent instances of the Python runtime environment;

* mod_python uses multiple, persistent instances of the Python runtime
        environment, as does SCGI, I think;

* PyWX uses one instance with multiple interpreters;

* Python itself when doing Web serving can use multiple threads, but all
        in one persistent Python runtime environment.

Because of the different levels of caching, I find it very difficult to
come up with a strategy that guarantees identical behavior across all
of these options without restricting the design of modules.

cheers,
--titus