On Sat, Jul 30, 2005 at 02:04:26PM +0800, Bo Yang wrote:
> It happened three times around the time mysql backup cron
> jobs run around 3 a.m. each day. mysqldump was configured to lock up
> each table in turn. so some of the unlucky write requests might have
> to wait for few minutes to get through. The backups took about 15
> minutes, but scgi froze right there, all the way to the morning when
> I woke up in panic.

Some ideas:

    * Use a tool like gdb or strace to see were the Apache process is
      hanging.

    * If you don't have time to use gdb or strace, kill the proces
      with SIGQUIT (should produce a core dump) and use gdb on the
      core file.

    * Try a different MPM (multi-process module) for Apache.
      Prefork should be the most robust.

    * Try removing the connection retry logic from mod_scgi (remove
      the entire "if" block that contains "goto restart").  There
      could be a bug in that code, although I don't see it.

    * If you have a test machine, perhaps you could try to reproduce
      the hang (e.g. use "ab" and add some sleeps in the Quixote
      application to simulate locked tables).


> A big thank-you to the quixote team, by the way.

I see you are using utf-8 for your site.  Have you run into any
Unicode Quixote problems?  I recently fixed a bug in the form
framework but perhaps you don't use it.

  Neil