On Tue, Sep 13, 2005 at 10:07:45AM -0700, Mike Orr wrote:
> Firefox, IE, Safari, and Netscape 7 at least.

I'm pretty sure 'Expires: -1' works properly in Firefox and Netscape
7.

> It doesn't always happen, but when it does it keeps happening
> several times in a row.  It migrates sites too, sometimes on one,
> sometimes on the other, even though they're derived from the same
> codebase.

I just fired up Ethereal and pointed it at www.incidentnews.gov.
Here's the interesting part (I left out some boring parts):

    ===>

    GET /1/search_entries_action?category=6 HTTP/1.1
    Keep-Alive: 300
    Connection: keep-alive
    Referer: http://www.incidentnews.gov/1/

    <===

    HTTP/1.1 302 Moved Temporarily
    Date: Tue, 13 Sep 2005 18:03:57 GMT
    Expires: -1
    Cache-Control: max-age=60
    Expires: Tue, 13 Sep 2005 18:04:57 GMT
    Location: http://www.incidentnews.gov/1/entries

    ===>

    GET /1/entries HTTP/1.1
    Referer: http://www.incidentnews.gov/1/

    <===

    HTTP/1.1 200 OK
    Date: Tue, 13 Sep 2005 18:03:57 GMT
    Server: Apache/1.3.33 (Darwin) mod_scgi/1.7 mod_ssl/2.8.22 [...]
    Expires: -1
    Cache-Control: max-age=60
    Expires: Tue, 13 Sep 2005 18:04:57 GMT
    Content-Length: 2805
    Connection: close

    Hurricane Katrina (LA General)

    
Browse OTHER PRODUCTS: as list [...]


Something is adding a second Expires header (maybe Apache) and
giving the expire time as 1 minute.  I think that combined with your
poor URI design is the source of the problems.

Why doesn't /1/search_entries_action?category=X just return the page
rather than redirecting back to /1/entries and then having
/1/entries return the page?  If you did do that then you could
probably allow the browser to cache the
/1/search_entries_action?category=X pages for some period of time.

Allowing caching via "Expires" is a huge performance boost because
if the page is not yet expired then browser does not have to contact
the server at all (unlike, for example, Last-Modified +
If-Modified-Since).  For users not at your site, the latency of a
HTTP request/response cycle is probably very noticeable.

> I'm going to rearrange the URLs as David suggested ( /1/categories/4 )
> and do the query for every page.

Just to be clear, you don't have to differentiate the URIs by path
(although it's often a nice design).  Using a query string is also
sufficient.  The root problem is that /1/entries is overloaded.  If
you want a quick hack, have /1/search_entries_action?category=X
redirect to /1/entries?category=X.  You would have to add a hidden
field to the paging form but that shouldn't be hard.  With that
change, clicking on 'Next' would go to /1/entries?category=X&page=1.

Hope that helps,

  Neil