On Wed, May 19, 2004 at 11:33:33AM -0400, Tom Jenkins wrote:
> what is the best way to handle this type of situation?  should the
> quixote class handling the request take the information, setup the long
> process and fork a child to run the long running process, while the
> parent returns the reponse?  or should we pass the information to some
> "thing" (be it database/config file/whatever) and have a separate
> process that polls for entries to that thing that then handles running
> the long running process?

  I would lean towards the second option, for two reasons:

    1. you don't burden your webserver with all the processes
    2. you can easily restart the long running job if it should exit without
       producing results

  Putting an entry in a database and then having another process that reads
  database periodically and starts the jobs.  This process could also keep
  track of which jobs are running and restart them if necessary.

> intuitively i like the first option but i have to admit i'm not
> confident that i really understand forking.  so any pointers to forking
> would be gratefully accepted also.

  Here is a pair of methods i use to start and stop processes using a fork:

    def spawn_process(self,cmd):
        """spawns a child process and returns it's pid.  returns -1 on fail

           cmd is a string of the command to run, ie:
             "/path/to/prog arg1 arg2"
        """

        pid = os.fork()
        if pid > -1:
            if pid == 0:
                cmdv = cmd.split(" ")
                os.execv(cmdv[0],cmdv)

        return pid

    def kill_process(self,cmd):
        os.kill(self.server_pid, 9)
        return os.waitpid(self.server_pid,0)

  In my case the processes I start and stop don't exit, they run until I kill
  them.  Sounds like your processes exit, so you'll need to be sure to do a
  waitpid on the processes that exit.  The return value of waitpid has
  information about the exit status of the process.  See waitpid(2) and
  pydoc os.waitpid for more info.

  Works for me.  YMMV.

Jon
--
Jon Dugan             |  Senior Network Engineer, NCSA Network Research
jdugan@ncsa.uiuc.edu  |  269 CAB, 605 E Springfield, Champaign, IL 61820
217-244-7715          |  http://www.ncsa.uiuc.edu/~jdugan/