Mailing List Archive

> Hmmm... this effectively implements the "bean count log" which Rob McCool
> brought up on this list ages ago as an idea which had been kicked around
> Netscape. The only quibble I have with it on that score is that a serial
> search through the entire log is going to start to drag on busy sites;

Depends on how many counters you want. It doesn't count all URLs, but
it does check them. Few people will want to count their GIFs in this
way, that's what the logfiles are there for.

> use of something like DBM might be a better bet. (I haven't ever counted
> the number of distinct URIs which this site gets requests for on an average
> day, but it wouldn't surprise me to see it in the range of a few thousand).
>
> As to making the counts available to scripts and includes... hoo boy,
> the cache fanciers on HTTP-WG are gonna love *this*.

Hmm, I suppose that if it encourages people to use counters then it
could have a negative overall effect, but if people are shown that they
can use XBITHACK-full to send cache info on counter pages, it'd help.

Unlike the SSI based counters I've seen elsewhere, you aren't forced
to have a counter shown on the page, or any other kind of include. It
could be used for accounting purposes on a subset of pages.

> One final question arises, namely whether to put this in the main
> distribution, or in the contributed modules directory.

contrib for sure.

> I guess right now, I'm a +0 on it... (the
> main problem being that it does have the potential to drag down large
> sites that try it...).

If it's used for hundreds of URLs, I'd call it "abuse" rather than "use"..
they should write their own DBM counter.


rob

Ooo, I was just thinking that a counter module can be easily
adapted into a load-cuttoff module.

When a URL reaches a particular count (also specified in the
modified counter file), the server can return a rejection response.
An external program can be used to reset the counter periodically.

So a server admin can effectively limit the number of times any
URL can be accessed.

A smarter module could reject all requests but those which leave a
count divisible by N.. so for every N requests made, only 1 is
serviced to completion.

Food for thought.



rob

Hmmm... this effectively implements the "bean count log" which Rob McCool
brought up on this list ages ago as an idea which had been kicked around
Netscape. The only quibble I have with it on that score is that a serial
search through the entire log is going to start to drag on busy sites;
use of something like DBM might be a better bet. (I haven't ever counted
the number of distinct URIs which this site gets requests for on an average
day, but it wouldn't surprise me to see it in the range of a few thousand).

As to making the counts available to scripts and includes... hoo boy,
the cache fanciers on HTTP-WG are gonna love *this*.

One final question arises, namely whether to put this in the main
distribution, or in the contributed modules directory. My personal
preference is for the latter (it would be nice to have that directory
a bit less vacant ;-), but I have no *deep* misgivings about seeing it
as a new experimental module in the main server distribution (we have a
few of those, too, already). I guess right now, I'm a +0 on it... (the
main problem being that it does have the potential to drag down large
sites that try it...).

rst

On Sun, 20 Aug 1995, Robert S. Thau wrote:
> Hmmm... this effectively implements the "bean count log" which Rob McCool
> brought up on this list ages ago as an idea which had been kicked around
> Netscape. The only quibble I have with it on that score is that a serial
> search through the entire log is going to start to drag on busy sites;
> use of something like DBM might be a better bet. (I haven't ever counted
> the number of distinct URIs which this site gets requests for on an average
> day, but it wouldn't surprise me to see it in the range of a few thousand).

Agreed, a DBM mechanism would be *much* more scalable.

> As to making the counts available to scripts and includes... hoo boy,
> the cache fanciers on HTTP-WG are gonna love *this*.

As a "cache fancier on http-wg" myself, I would say as long as the
XBITHACK options are set right and it doesn't affect the last-modified
time of the document, it's alright with me.

> One final question arises, namely whether to put this in the main
> distribution, or in the contributed modules directory. My personal
> preference is for the latter (it would be nice to have that directory
> a bit less vacant ;-), but I have no *deep* misgivings about seeing it
> as a new experimental module in the main server distribution (we have a
> few of those, too, already).

/contrib, definitely. Before the next release someone should add a
pointer to /contrib from the README and INSTALL files.

> I guess right now, I'm a +0 on it... (the
> main problem being that it does have the potential to drag down large
> sites that try it...).

I'm sure RobH could pull the relevant 10 lines from mod_auth_dbm.c to get
it to use dbm files :)

Brian

--=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=--
brian@organic.com brian@hyperreal.com http://www.[hyperreal,organic].com/

> > /contrib, definitely. Before the next release someone should add a
> > pointer to /contrib from the README and INSTALL files.
>
> Patch? (Yes, I'm serious).

Attached.

Brian

Date: Sun, 20 Aug 1995 12:49:47 -0700 (PDT)
From: Brian Behlendorf <brian@organic.com>

/contrib, definitely. Before the next release someone should add a
pointer to /contrib from the README and INSTALL files.

Patch? (Yes, I'm serious).

rst

>
>
> Ooo, I was just thinking that a counter module can be easily
> adapted into a load-cuttoff module.
>
> When a URL reaches a particular count (also specified in the
> modified counter file), the server can return a rejection response.
> An external program can be used to reset the counter periodically.
>
> So a server admin can effectively limit the number of times any
> URL can be accessed.
>
> A smarter module could reject all requests but those which leave a
> count divisible by N.. so for every N requests made, only 1 is
> serviced to completion.

Wouldn't this cause the same behavior, that was objected to earlier
where the user would just keep hitting reload because sometimes they
see a response and sometimes they don't?

-Beth Frank
efrank@ncsa.uiuc.edu
>
> Food for thought.
>
>
>
> rob
>

> > A smarter module could reject all requests but those which leave a
> > count divisible by N.. so for every N requests made, only 1 is
> > serviced to completion.
>
> Wouldn't this cause the same behavior, that was objected to earlier
> where the user would just keep hitting reload because sometimes they
> see a response and sometimes they don't?
>
> -Beth Frank
> efrank@ncsa.uiuc.edu

I'm sure it could cause that behaviour, although it should be less
likely.

I think the earlier discussion was on ways to reduce load during
peek times by rejecting all types of request to a server, based
on having a susbet of the children in a "reject everything" mode.

Counter module adaptations can concentrate on the resource
hogging requests, and shut them down for any length of "time" the
www admin chooses.

Anyway, I was just giving some ideas on how a plug in module could be
used (in different ways) to reject requests for a given resource. The
admin is free to choose her own scheme and plug it in.


rob
--
http://nqcd.lanl.gov/~hartill/