One of the things I've wanted to get done for some time is to speed the
startup of a server process.  The LWN code has gotten reasonably large, and
it takes quite a while to import the whole thing - especially when apache
has just been started, and there are potentially hundreds of processes all
trying to start up at the same time.  The LWN server performs quite well
once it's running, but it can take quite a while to hit its stride after a
restart.

Startup would work better if the server just had to load enough code to
satisfy the requests it actually gets.  So I've been pondering, for a
while, an "autoload" functionality to pull in the server code as needed.  I
finally got around to looking at that, and found that the Quixote code
already tries to do an import if an exported symbol is missing from a
module - a nifty little undocumented feature.  Unfortunately, for me, it
doesn't quite work:

- The quixote code tries to guess the right module name, but it gets it
  wrong for my setup.

- For various reasons, my code will import several symbols from one module
  and export them in another.  In particular, much of the top-level
  namespace is done this way.

Fortunately, it's all easy to fix.  The following little patch makes the
publisher look for a function called _q_import when it thinks it's time to
try importing something.  _q_import needs to return an object for the given
name, just like _get_module does, or it should raise an exception.  A nice
side benefit is that _q_import does not *have* to return a module; if it
returns a callable object instead, Quixote is happy.  My _q_import also
stashes the symbol into the container's dictionary so that it doesn't get
called again for that particular case.

Anyway, here it is, in case it's useful.

jon

Jonathan Corbet
Executive editor, LWN.net
corbet@lwn.net


--- publish.py.0.51     2002-12-02 14:56:36.000000000 -0700
+++ publish.py  2002-12-02 14:59:57.000000000 -0700
@@ -428,12 +428,16 @@
             has_component = hasattr(container, component)
             if is_module and not has_component and component != "_q_index":
                 # get next module
-                mod_name = container.__name__ + '.' + component
-                # If we get an ImportError here we don't catch it.  It means
-                # that either someone exported something that doesn't exist or
-                # an exception was raised from deeping in the code.  A
-                # traceback should be generated in either case.
-                object = _get_module(mod_name)
+                if hasattr(container, '_q_import'):
+                    importer = getattr(container, '_q_import')
+                    object = importer(component)
+                else:
+                    mod_name = container.__name__ + '.' + component
+                    # If we get an ImportError here we don't catch it.  It
means
+                    # that either someone exported something that doesn't exist
or
+                    # an exception was raised from deeping in the code.  A
+                    # traceback should be generated in either case.
+                    object = _get_module(mod_name)

             elif has_component:
                 object = getattr(container, component)