Mailing List Archive

Stuff in my sandbox...
My current sandbox code has a few changes in it. Most of the changes
are actually to the default config files (merging Alexei's stuff, as
modified by Florent, with my own changes for 0.8.3 --- see below for
details). I also made it spell "Required" correctly, tossed in drtr's
default-config-locations bugfix (good catch on his part), and did
*something* about the StartServers mess. The sample config file
entries for these parms currently read:

# Server-pool size regulation. Rather than making you guess how many
# server processes you need, Apache dynamically adapts to the load it
# sees --- that is, it tries to maintain enough server processes to
# handle the current load, plus a few spare servers to handle transient
# load spikes (e.g., multiple simultaneous requests from a single
# Netscape browser).

# It does this by periodically checking how many servers are waiting
# for a request. If there are fewer than MinSpareServers, it creates
# a new spare. If there are more than MaxSpareServers, some of the
# spares die off.

# NB having too few spares can measurably hurt performance.
# These values are probably OK for most sites ---

MinSpareServers 5
MaxSpareServers 10

# Number of servers to start --- should be a reasonable ballpark figure.

StartServers 5

MaxSpareServers can also be spelled "MaxServers" for NCSA server
back-combatibility (it's the best we can do with attempts to tailor
NCSA 1.4, given that it, too, would happily fork well beyond that
limit under sufficient demand), but that should be thoroughly
deprecated. I *hope* that's good enough for everybody.

The change-log entries, for this and everything else, follow:

*) Changes to server-pool management parms --- renamed current
StartServers to MinSpareServers, created separate StartServers
parameter which means what it says, and renamed MaxServers to
MaxSpareServers (though the old name still works, for NCSA 1.4
back-combatibility). The old names were generally regarded as
too confusing. Also altered "docs" in sample config files.

*) More improvements to default config files ---
sample directives (commented out) for XBitHack, BindAddress,
CacheNegotiatedDocs, VirtualHost; decent set of AddLanguage
defaults, AddTypes for send-as-is and imagemap magic types, and
improvements to samples for DirectoryIndex [Alexei Kosut]

*) Yet more improvements to default config files --- changes to
Alexei's sample AddLanguage directives, and sample LanguagePriority
[ Florent Guillaume ]

*) Set config file locations properly if not set in httpd.conf
[ David Robinson ]

*) Corrected spelling of "Required" in 401 error reports [Andrew Wilson]

rst
Re: Stuff in my sandbox... [ In reply to ]
> Hmmm... you don't want to test the actual distribution before releasing it?

I think that we should run the distribution release for at least 24 hours
before announcing it. This thing is going to create enough traffic
without having real bugs (which I am sure it does).

> On names... I *really* dislike AbsMaxServers, because (as I've said) just
> having it there invites people to set it too low, and all too many of them
> have shown that they will take up the invitation. If you don't like
> MaxClients, how about MaxConnections? (What I want is for the name to
> convey immediately, to someone who is too lazy to read anything else, that
> setting this number too low will screw the server --- MaxServers looks too
> much like things will run slower, but they'll still run).

I would vote +1 on MaxClients. It would probably have the psycological
effect of getting set too high rather than too low which would probably
be safer.

I also plan to update whatever documentation we have on imagemaping
to cover the new tags at the last minute. ie, when we release,
I'll go change it...



There is one other change that I would like to suggest. I wanted to
supply patches, but have been up to my ears.

After looking through config files this past weekend and thinking
about a config tool, it occurs to me that it would be nice to change
process_resource_config() to allow any number of files with .conf
extensions. (I personally would rather see it become 1 file that is
configured with a Tk program.) Just a thought....
Re: Stuff in my sandbox... [ In reply to ]
On Mon, 31 Jul 1995, Robert S. Thau wrote:
> I would vote +1 on MaxClients. It would probably have the psycological
> effect of getting set too high rather than too low which would probably
> be safer.
>
> That's precisely the intended effect, and this is still my own preferred
> name for this reason. Still, in the name of consensus... Brian, what do
> you think of ServerSafetyLimit? [.Robert... how long before you have to
> go home and crash? --- oh whoops, that one's for me].

MaxClients is fine.

Brian

--=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=--
brian@organic.com brian@hyperreal.com http://www.[hyperreal,organic].com/
Re: Stuff in my sandbox... [ In reply to ]
So, Rob will put 0.8.5 on hyperreal in the proper places, I'll work on
documentation tonight, and let's announce tomorrow.

Hmmm... you don't want to test the actual distribution before releasing it?

Also, Cliff's bug report does worry me a bit; it's possible that this is
just the Netscape reload brain-death (if NetScape does a reload, it sends
both If-Modified-Since and Pragma: no-cache, and expects the latter to
negate the effect of the former. With the current release, it probably
doesn't, so depending on what headers are passed along, the cache may get
a 304 reply which it wouldn't like very much. This is *probably* easy to
fix, and I could toss it in easily).

On names... I *really* dislike AbsMaxServers, because (as I've said) just
having it there invites people to set it too low, and all too many of them
have shown that they will take up the invitation. If you don't like
MaxClients, how about MaxConnections? (What I want is for the name to
convey immediately, to someone who is too lazy to read anything else, that
setting this number too low will screw the server --- MaxServers looks too
much like things will run slower, but they'll still run).

rst
Re: Stuff in my sandbox... [ In reply to ]
I would vote +1 on MaxClients. It would probably have the psycological
effect of getting set too high rather than too low which would probably
be safer.

That's precisely the intended effect, and this is still my own preferred
name for this reason. Still, in the name of consensus... Brian, what do
you think of ServerSafetyLimit? [.Robert... how long before you have to
go home and crash? --- oh whoops, that one's for me].

rst
Re: Stuff in my sandbox... [ In reply to ]
Cliff --- could you get your friend to try reloading a page with images
from what's running on www.ai.mit.edu right now? (A non-negotiated page
would be best, i.e., http://www.ai.mit.edu/index.html, not just
http://www.ai.mit.edu/). This has a one-line fix applied which makes
conditional GETs with Pragma: no-cache unconditional (as Netscape seems
to strangely expect).

rst
Re: Stuff in my sandbox... [ In reply to ]
We should deprecate ftime from the sources and use gettimeofday instead.

At least under FreeBSD 2.0.5 ftime is available only in the libcompat
library and may one day go away.

Whats the situation with other OS's.

--
Paul Richards, Bluebird Computer Systems. FreeBSD core team member.
Internet: paul@FreeBSD.org, http://www.freebsd.org/~paul
Phone: 0370 462071 (Mobile), +44 1222 457651 (home)
Re: Stuff in my sandbox... [ In reply to ]
At 1:48 PM 8/1/95, Paul Richards sez:

> We should deprecate ftime from the sources and use gettimeofday instead.
>
> At least under FreeBSD 2.0.5 ftime is available only in the libcompat
> library and may one day go away.
>
> Whats the situation with other OS's.

Doesn't exist in A/UX, either (nor in any external A/UX library that I know
of either).

-ech

______________________________________________________
Eric Hagberg Phone: 212-746-6387
User Services Manager, CUMC/OAC Fax: 212-746-8161
Re: Stuff in my sandbox... [ In reply to ]
Re: Stuff in my sandbox... [ In reply to ]
> Also, Cliff's bug report does worry me a bit; it's possible that this is
> just the Netscape reload brain-death (if NetScape does a reload, it sends
> both If-Modified-Since and Pragma: no-cache, and expects the latter to
> negate the effect of the former.

This is not a bug in Netscape, just an annoying feature, in that no
type of reload can be sent without the i-m-s. Sending both is perfectly
reasonable.. it lets the client resync with the original server.

> With the current release, it probably
> doesn't, so depending on what headers are passed along, the cache may get
> a 304 reply which it wouldn't like very much. This is *probably* easy to
> fix, and I could toss it in easily).

I believe that Roy said that end servers are not supposed to take any
notice of the Pragma: no-cache.

Earlier NCSA and Apache code certainly didn't check Pragma: no-cache, so
I think the problem must be elsewhere.

> On names... I *really* dislike AbsMaxServers, because (as I've said) just
> having it there invites people to set it too low, and all too many of them
> have shown that they will take up the invitation. If you don't like
> MaxClients, how about MaxConnections? (What I want is for the name to
> convey immediately, to someone who is too lazy to read anything else, that
> setting this number too low will screw the server --- MaxServers looks too
> much like things will run slower, but they'll still run).
>
> rst
>


rob
Re: Stuff in my sandbox... [ In reply to ]
>Cliff --- could you get your friend to try reloading a page with images
>from what's running on www.ai.mit.edu right now? (A non-negotiated page
>would be best, i.e., http://www.ai.mit.edu/index.html, not just
>http://www.ai.mit.edu/). This has a one-line fix applied which makes
>conditional GETs with Pragma: no-cache unconditional (as Netscape seems
>to strangely expect).

***********************************************************
** NO NO NO NO NO NO NO NO NO NO NO NO NO NO NO NO NO NO **
***********************************************************

Don't you dare release that code! Protecting network caching is
far more important than adding a bug to Apache just to bend over
backwards in response to a bug in this month's version of Navigator.

Screw That!

If you want to make client-specific doodads, then they had better
be predicated by matching the exact User-Agent field that is responsible
for the bug in the first place. Sending Pragma and IMS together is
a perfectly valid request in HTTP, and Apache would be broken if it
did not respond with 304 if IMS >= LM.

......Roy

p.s. -1
Re: Stuff in my sandbox... [ In reply to ]
> Date: Tue, 01 Aug 1995 17:21:02 -0400
> From: Roy Fielding <fielding@beach.w3.org>
>
> Don't you dare release that code! Protecting network caching is
> far more important than adding a bug to Apache just to bend over
> backwards in response to a bug in this month's version of Navigator.
>
> Screw That!
>
> Summary of conclusions: we're probably going to have a maintenance
> release in about two weeks, so we needn't be "broken" longer than
> that. At that point, probably the best thing to do is (as Roy
> suggests below), to check User-agent: in addition to Pragma: no-cache
> before deciding to ignore If-last-modified. However, in the meantime,
> I think Netscape-friendliness is better for everyone than strict
> conformance to the standard (just as I believed the same about Arena
> friendliness when it refused to deal with attributes on media types).

I don't want to start a flame war on this subject, so I'll say
my piece and leave it at that.

Having Apache 0.8.5 take notice of "Pragma: no-cache" is a bad thing.
Every reload to my server from Netscape users is going to cause a
retransmission of the unchanged document.
I don't know what problem Cliff's contact has, but I find it difficult to
believe that it has anything to do with how an end server handles
"Pragma: no-cache", it's more likely a proxy cache problem, a broken
client or something else broken with Apache (in order of most probable).

Has anyone had any reports of problems from their users with <= 0.8.4 ???

Cardiff has the "Pragma" check switched off, and it works fine via
the public proxy at wwwcache.hensa.ac.uk 8080. None of the hoards of
AOL and prodigy users have complained so far. I'll keep it switched
off until someone can point the finger of blame at Apache.

Bottom line: the new code is wrong (cf Roy's comments), we have one
unconfirmed problem report, but lots of confirmed satisfied customers.

tie my vote to Roy's. If he changes his -1, mine changes too.



rob
Re: Stuff in my sandbox... [ In reply to ]
Summary: This is so fucking confusing. I was trying to replicate this
behavior and see where it was happening. I am somewhat convinced it has more
to do with broken proxies than a broken netscape....

www.organic.com is still a shambhala 0.6.0 - but I installed 0.8.5 on a
higher port and it had no difference. I also tried this against an 0.6.5
server out there (www.cnet.com) but it didn't have these problems.

1) load www.organic.com locally from a fresh disk/memory cache. No
problems, looks and works like a champ.

2) hit "reload" on netscape - the main page request gets a "200"
response, the inlined images get a "304" response, and everyone is happy.

3) loaded www.organic.com through web.mit.edu port 80. First request,
everything is fine.... second request, the inlined images are fuct.

4) loaded www.organic.com through wwwcache.hensa.ac.uk 8080. First
request is fine, second request is *fine*, third request is *fine*...

I couldn't find any other public proxies - looks like Prodigy and AOL
shut theirs off. web.mit.edu is running a CERN proxy, and Hensa's
running a Netscape-Proxy.

Ugh!

This is frustrating. I'm tempted to say a bugward compatibility should
only go in with a User-Agent check (and a STERN comment in the
README's!), or not at all.

I'm really interested in hearing why going through hensa works and
web.mit doesn't.

Brian


On Tue, 1 Aug 1995, Rob Hartill wrote:
> > Date: Tue, 01 Aug 1995 17:21:02 -0400
> > From: Roy Fielding <fielding@beach.w3.org>
> >
> > Don't you dare release that code! Protecting network caching is
> > far more important than adding a bug to Apache just to bend over
> > backwards in response to a bug in this month's version of Navigator.
> >
> > Screw That!
> >
> > Summary of conclusions: we're probably going to have a maintenance
> > release in about two weeks, so we needn't be "broken" longer than
> > that. At that point, probably the best thing to do is (as Roy
> > suggests below), to check User-agent: in addition to Pragma: no-cache
> > before deciding to ignore If-last-modified. However, in the meantime,
> > I think Netscape-friendliness is better for everyone than strict
> > conformance to the standard (just as I believed the same about Arena
> > friendliness when it refused to deal with attributes on media types).
>
> I don't want to start a flame war on this subject, so I'll say
> my piece and leave it at that.
>
> Having Apache 0.8.5 take notice of "Pragma: no-cache" is a bad thing.
> Every reload to my server from Netscape users is going to cause a
> retransmission of the unchanged document.
> I don't know what problem Cliff's contact has, but I find it difficult to
> believe that it has anything to do with how an end server handles
> "Pragma: no-cache", it's more likely a proxy cache problem, a broken
> client or something else broken with Apache (in order of most probable).
>
> Has anyone had any reports of problems from their users with <= 0.8.4 ???
>
> Cardiff has the "Pragma" check switched off, and it works fine via
> the public proxy at wwwcache.hensa.ac.uk 8080. None of the hoards of
> AOL and prodigy users have complained so far. I'll keep it switched
> off until someone can point the finger of blame at Apache.
>
> Bottom line: the new code is wrong (cf Roy's comments), we have one
> unconfirmed problem report, but lots of confirmed satisfied customers.
>
> tie my vote to Roy's. If he changes his -1, mine changes too.
>
>
>
> rob
>

--=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=--
brian@organic.com brian@hyperreal.com http://www.[hyperreal,organic].com/
Re: Stuff in my sandbox... [ In reply to ]
Date: Tue, 01 Aug 1995 17:21:02 -0400
From: Roy Fielding <fielding@beach.w3.org>

Don't you dare release that code! Protecting network caching is
far more important than adding a bug to Apache just to bend over
backwards in response to a bug in this month's version of Navigator.

Screw That!

Summary of conclusions: we're probably going to have a maintenance
release in about two weeks, so we needn't be "broken" longer than
that. At that point, probably the best thing to do is (as Roy
suggests below), to check User-agent: in addition to Pragma: no-cache
before deciding to ignore If-last-modified. However, in the meantime,
I think Netscape-friendliness is better for everyone than strict
conformance to the standard (just as I believed the same about Arena
friendliness when it refused to deal with attributes on media types).




In support --- a few points:

First off, I just verified that this change *does* fix the bug
encountered by Cliff's friend (by going through the web.mit.edu public
proxy). That's in addition to the "can't reload" bug when talking
directly to the server. That's in addition to the semi-reliable
reports I've had that under certain circumstances Netscape actually
goes into loops when it gets a 304 on a request which it thought was
unconditional, repeatedly sending out requests until the user manually
intervenes --- how's *that* for conserving network bandwidth by
sending 304s? All of these are, in principle, Netscape's problem, but
only fools ignore Netscape in the current environment, and I, for one,
am not a fool.

Second, while it's true that under certain circumstances, a copy of
the document will be sent back to the proxy when it needn't be,
consuming a little extra net bandwidth, there is also a cost in net
bandwidth to doing things your way, in terms of all the people who
just won't use caches if it gets them this kind of broken behavior.
That will probably cost a hell of a lot more aggregate net bandwidth
than would be saved by strictly correct behavior in this case.

To put it another way --- I think more people will regard a server
that serves "broken windows" as broken, than one which consumes more
net bandwidth than it absolutely has to.

If you view caches as an elegant mathematical abstraction, then I
admit that catering to Netscape here mars the system. If you view
them as a tool that you want people to use, I think you're better off
living with the ugliness until fixed Netscapes are available (because
a lot of people who might use them otherwise probably won't if it gets
them behavior like this --- and, as I said, that costs bandwidth too).

If you want to make client-specific doodads, then they had better
be predicated by matching the exact User-Agent field that is responsible
for the bug in the first place. Sending Pragma and IMS together is
a perfectly valid request in HTTP, and Apache would be broken if it
did not respond with 304 if IMS >= LM.

That's actually a pretty good idea, and easy enough to add, but I'd
prefer to save it for the maintenance release which will inevitably
follow this one in (probably) two weeks or so --- however, in the
meantime, I'd rather show people images than smashed windows. (I
suppose we could take out the Arena bug workaround at the same time,
now that the latest release of your organization's own browser appears
to be finally in conformance with the spec for attributes on MIME ---
oops, media types).

FWIW, I got notice of two last-minute items this morning which are
already put off to the maintenance release, but on my TODO list:

*) Doesn't compile clean with some old versions of SunOS acc.
*) httpd -v mentions "Shambhala", and shouldn't. It's Apache now.

rst
Re: Stuff in my sandbox... [ In reply to ]
Hang on --- I'm doing a last-minute check; if I'm right about what I'm seeing,
then I can promise you something that gets a good deal weirder than what
you've seen already...

rst