Mailing List Archive

Client-side caching updates
I think I've found a help for the Internet Explorer reload problem!

I finally dug up a network dump tool and looked at the actual
communication between IE and the server. What I found was that things
went roughly like this:

First request of page: 200 return code, all text, has last-modified date
and cache-control which requires revalidation on every load.

HTTP/1.1 200 OK
Date: Sat, 22 Mar 2003 05:40:36 GMT
Server: Apache/1.3.26 (Unix) PHP/4.2.3
X-Powered-By: PHP/4.2.3
X-Accelerated-By: PHPA/1.3.3r2
-> Expires: Mon, 15 Jan 2001 00:00:00 GMT
-> Cache-Control: private, must-revalidate, max-age=0
-> Last-Modified: Fri, 21 Mar 2003 16:34:52 GMT
Content-language: en
Keep-Alive: timeout=15, max=99
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/html; charset=iso-8859-1


Second request of page: 304 return code (not modified) -- but!

HTTP/1.1 304 Not Modified
Date: Sat, 22 Mar 2003 05:40:53 GMT
Server: Apache/1.3.26 (Unix) PHP/4.2.3
X-Powered-By: PHP/4.2.3
X-Accelerated-By: PHPA/1.3.3r2
-> Expires: Sat, 22 Mar 2003 08:40:53 GMT
-> Cache-Control: public, max-age=10800
-> Last-Modified: Mon, 10 Feb 2003 14:03:33 GMT
Keep-Alive: timeout=15, max=98
Connection: Keep-Alive
Content-Type: text/html

Third request of page: didn't even bother to touch the network, because
the 304 return accidentally let apache and/or php fill in generic
settings for the last-modified and cache-control headers which claim it
was modified when the PHP script was last updated and that it doesn't
need to be revalidated for 24 hours!

I've patched OutputPage.php to include the cache-control and
last-modified when sending 304s, and it's now rechecking the page on
subsequent visits. (A 304 returns very quickly, so it's fast even on my
modem.)

This may or may not help with the "reload doesn't actually reload"
problem. Hopefully so. ;)

Next step is to add a last-touched field for pages and users to
invalidate caches when links & preferences change -- see
http://meta.wikipedia.org/wiki/Cache_control

-- brion vibber (brion @ pobox.com)
Re: Client-side caching updates [ In reply to ]
On Fri, 2003-03-21 at 22:19, Brion Vibber wrote:
> Next step is to add a last-touched field for pages and users to
> invalidate caches when links & preferences change -- see
> http://meta.wikipedia.org/wiki/Cache_control

Okay, experimental code is in CVS and running on test.wikipedia.org,
please give it a thorough tryout!

Caching is now enabled for all browsers except for Internet Explorer 5.0
and earlier (which apparently have some other dreadful problems, and
I've no means to test them).

-- brion vibber (brion @ pobox.com)


Checking in newcodebase/Article.php;
/cvsroot/wikipedia/phpwiki/newcodebase/Article.php,v <-- Article.php
new revision: 1.142; previous revision: 1.141
done
Checking in newcodebase/DatabaseFunctions.php;
/cvsroot/wikipedia/phpwiki/newcodebase/DatabaseFunctions.php,v <-- DatabaseFunctions.php
new revision: 1.27; previous revision: 1.26
done
Checking in newcodebase/DefaultSettings.php;
/cvsroot/wikipedia/phpwiki/newcodebase/DefaultSettings.php,v <-- DefaultSettings.php
new revision: 1.20; previous revision: 1.19
done
Checking in newcodebase/LinksUpdate.php;
/cvsroot/wikipedia/phpwiki/newcodebase/LinksUpdate.php,v <-- LinksUpdate.php
new revision: 1.11; previous revision: 1.10
done
Checking in newcodebase/OutputPage.php;
/cvsroot/wikipedia/phpwiki/newcodebase/OutputPage.php,v <-- OutputPage.php
new revision: 1.116; previous revision: 1.115
done
Checking in newcodebase/SpecialMovepage.php;
/cvsroot/wikipedia/phpwiki/newcodebase/SpecialMovepage.php,v <-- SpecialMovepage.php
new revision: 1.30; previous revision: 1.29
done
Checking in newcodebase/User.php;
/cvsroot/wikipedia/phpwiki/newcodebase/User.php,v <-- User.php
new revision: 1.41; previous revision: 1.40
done
Checking in newcodebase/maintenance/buildTables.inc;
/cvsroot/wikipedia/phpwiki/newcodebase/maintenance/buildTables.inc,v <-- buildTables.inc
new revision: 1.23; previous revision: 1.22
done
RCS file: /cvsroot/wikipedia/phpwiki/newcodebase/maintenance/patch-cache.sql,v
done
Checking in newcodebase/maintenance/patch-cache.sql;
/cvsroot/wikipedia/phpwiki/newcodebase/maintenance/patch-cache.sql,v <-- patch-cache.sql
initial revision: 1.1
done
Checking in newcodebase/maintenance/patch-list.txt;
/cvsroot/wikipedia/phpwiki/newcodebase/maintenance/patch-list.txt,v <-- patch-list.txt
new revision: 1.8; previous revision: 1.7
Re: Client-side caching updates [ In reply to ]
On Sat, 2003-03-22 at 03:34, I wrote in wikitech-l:
> Okay, experimental code is in CVS and running on test.wikipedia.org,
> please give it a thorough tryout!
>
> Caching is now enabled for all browsers except for Internet Explorer 5.0
> and earlier (which apparently have some other dreadful problems, and
> I've no means to test them).

Well, I've gotten no feedback, so either no one checked or no one had
any problems....

I've put the new client-side caching code on http://meta.wikipedia.org ,
which attracts at least some small traffic. Try it out and report any
problems, 'cause I'm going to install this soon on all languages, and if
your browser can't read any pages at all I'd prefer to hear about it
beforehand. :)

In summary:
* Should eliminate problems with red links continuing to appear after
the page is created (or blue links after the page is deleted)
* Should eliminate problems with page layout not changing to reflect
logging in or changing display preferences
* Should, *I think*, fix "reload doesn't actually reload without also
holding down control" problem with Internet Explorer
* Client-side caching should now work on other browsers than IE. Early
versions of IE (5.0 and before) are still blacklisted, as we had all
kinds of trouble trying to accommodate 5.0 and I don't want to mess it
up. ;)

Some things this update does *not* do:
* Cache pages on the server; this is only to allow your web browser to
cache a page once it's been generated for you (but still be able to
refetch automatically if it's changed when you revisit it). The
invalidation mechanisms will help make server-side page caching work
properly once it's added in the future, though.
* Allow for caching of anything other than viewing pages, history lists,
or the Recentchanges list. (Other things could be cached, though,
they're just not yet keyed for it.)
* Invalidate cached pages when you log out.
* Make a distinction between preference changes that will and won't
alter page display. (Overly aggressive in invalidating caches.)

See http://meta.wikipedia.org/wiki/Cache_strategy for more confusing
prose.

-- brion vibber (brion @ pobox.com)
Re: Client-side caching updates [ In reply to ]
> (Brion Vibber <brion@pobox.com>):
>
> I've put the new client-side caching code on http://meta.wikipedia.org ,
> which attracts at least some small traffic. Try it out and report any
> problems, 'cause I'm going to install this soon on all languages, and if
> your browser can't read any pages at all I'd prefer to hear about it
> beforehand. :)

Brion, if you can either (a) install it on a wiki that's OK to trash,
or (b) check in the code, perhaps with a label, so that I can install
it on a trashable wiki, then I can run the test suite against it to
make sure you didn't miss anything obvious.

--
Lee Daniel Crocker <lee@piclab.com> <http://www.piclab.com/lee/>
"All inventions or works of authorship original to me, herein and past,
are placed irrevocably in the public domain, and may be used or modified
for any purpose, without permission, attribution, or notification."--LDC
Re: Client-side caching updates [ In reply to ]
On Wed, 26 Mar 2003, Lee Daniel Crocker wrote:
> > (Brion Vibber <brion@pobox.com>):
> >
> > I've put the new client-side caching code on http://meta.wikipedia.org ,
> > which attracts at least some small traffic. Try it out and report any
> > problems, 'cause I'm going to install this soon on all languages, and if
> > your browser can't read any pages at all I'd prefer to hear about it
> > beforehand. :)
>
> Brion, if you can either (a) install it on a wiki that's OK to trash,
> or (b) check in the code, perhaps with a label, so that I can install
> it on a trashable wiki, then I can run the test suite against it to
> make sure you didn't miss anything obvious.

It's been checked in for a few days, and is running on
http://test.wikipedia.org/ (which you can feel free to trash). I guess no
one read my announcement from the other day. ;)

-- brion vibber (brion @ pobox.com)
Re: Client-side caching updates [ In reply to ]
Brion Vibber wrote:
> It's been checked in for a few days, and is running on
> http://test.wikipedia.org/ (which you can feel free to trash). I guess no
> one read my announcement from the other day. ;)

I'm not good at testing chaching. Din't you wrote meta.wikipedia.org
yesterday?

Nobody
--
------------------------- Anthill inside! ---------------------------
Re: Client-side caching updates [ In reply to ]
On Wed, 26 Mar 2003, Thomas Corell wrote:
> Brion Vibber wrote:
> > It's been checked in for a few days, and is running on
> > http://test.wikipedia.org/ (which you can feel free to trash). I guess no
> > one read my announcement from the other day. ;)
>
> I'm not good at testing chaching. Din't you wrote meta.wikipedia.org
> yesterday?

Yes, and on Saturday I wrote test.wikipedia.org. Both things are true.
However, I'd prefer that people not trash meta.wikipedia.org. :)

-- brion vibber (brion @ pobox.com)
Re: Client-side caching updates [ In reply to ]
> > Brion Vibber wrote:
> > > It's been checked in for a few days, and is running on
> > > http://test.wikipedia.org/ (which you can feel free to trash). I guess no
> > > one read my announcement from the other day. ;)

If it makes you feel better, I read your announcements. I can't say
that refreshes seemed any faster, although they certainly didn't seem
any worse (it is hard to guage). (Mozilla 1.3)

--
Nick Reinking -- eschewing obfuscation since 1981 -- Minneapolis, MN
Re: Client-side caching updates [ In reply to ]
> (Brion Vibber <vibber@sal-sun062.usc.edu>):
>
> It's been checked in for a few days, and is running on
> http://test.wikipedia.org/ (which you can feel free to trash). I guess no
> one read my announcement from the other day. ;)

For some reason it doesn't have some of my recent code updates, so the
test suite won't run against it (in particular I added name attributes
to all the HTML forms so the test suite could find them).

--
Lee Daniel Crocker <lee@piclab.com> <http://www.piclab.com/lee/>
"All inventions or works of authorship original to me, herein and past,
are placed irrevocably in the public domain, and may be used or modified
for any purpose, without permission, attribution, or notification."--LDC
Re: Client-side caching updates [ In reply to ]
On Wed, 26 Mar 2003, Lee Daniel Crocker wrote:

> > (Brion Vibber <vibber@sal-sun062.usc.edu>):
> >
> > It's been checked in for a few days, and is running on
> > http://test.wikipedia.org/ (which you can feel free to trash). I guess no
> > one read my announcement from the other day. ;)
>
> For some reason it doesn't have some of my recent code updates, so the
> test suite won't run against it (in particular I added name attributes
> to all the HTML forms so the test suite could find them).

How about now?

-- brion vibber (brion @ pobox.com)
Re: Client-side caching updates [ In reply to ]
> > For some reason it doesn't have some of my recent code updates, so the
> > test suite won't run against it (in particular I added name attributes
> > to all the HTML forms so the test suite could find them).
>
> How about now?

That's better. I'll try running the suite again after lunch.

--
Lee Daniel Crocker <lee@piclab.com> <http://www.piclab.com/lee/>
"All inventions or works of authorship original to me, herein and past,
are placed irrevocably in the public domain, and may be used or modified
for any purpose, without permission, attribution, or notification."--LDC
Re: Client-side caching updates [ In reply to ]
http://test.wikipedia.com/ passes the current incarnation of
the test suite (which is admittedly incomplete, but it will
certainly catch anything obvious--and it does make edits and
then check for them, which a broken caching system might fail).

--
Lee Daniel Crocker <lee@piclab.com> <http://www.piclab.com/lee/>
"All inventions or works of authorship original to me, herein and past,
are placed irrevocably in the public domain, and may be used or modified
for any purpose, without permission, attribution, or notification."--LDC
Re: Client-side caching updates [ In reply to ]
On Wed, 26 Mar 2003, Lee Daniel Crocker wrote:
> http://test.wikipedia.com/ passes the current incarnation of
> the test suite (which is admittedly incomplete, but it will
> certainly catch anything obvious--and it does make edits and
> then check for them, which a broken caching system might fail).

Hmm, looks like your test suite doesn't send any If-Modified-Since headers
when refetching previously visited pages, so it wouldn't trigger a 304 Not
Modified response under any circumstances.

(The logfile should now pop out a debug statement to this effect.)

What are you using to handle the http connections?

-- brion vibber (brion @ pobox.com)
Re: Client-side caching updates [ In reply to ]
> (Brion Vibber <vibber@aludra.usc.edu>):
>
> Hmm, looks like your test suite doesn't send any If-Modified-Since headers
> when refetching previously visited pages, so it wouldn't trigger a 304 Not
> Modified response under any circumstances.
>
> (The logfile should now pop out a debug statement to this effect.)
> What are you using to handle the http connections?

The virtual "browser" is a Java class from the "httpunit" package.
It can be configured in different ways--I'll look into caching
behavior.

--
Lee Daniel Crocker <lee@piclab.com> <http://www.piclab.com/lee/>
"All inventions or works of authorship original to me, herein and past,
are placed irrevocably in the public domain, and may be used or modified
for any purpose, without permission, attribution, or notification."--LDC