Mailing List Archive

Re: apache
On Fri, 7 Apr 1995, Mark J Cox wrote:
> I like Apache - its got lots of patches that I had also attempted! I just
> compared my patches with Apache and here are the differences: "things I'd
> like to see in Apache". I've got the patches for all of these - is there
> a mailing list for the Apache "team" yet? Some place you have your
> discussions?

Yup, I'm cc'ing them on this reply. If you want to subscribe, send mail
to majordomo@hyperreal.com with the words "subscribe new-httpd". It's
rather high volume so only subscribe if you know how to filter your mail
or whatnot :)

We're about ready to roll together a 0.5 (I think - guys?) with the aim
of using one more week to debug that without adding new features, and
then making a general release announcement. So, Wait until 0.5 comes out
before submitting patches for the below just yet... go to
http://hyperreal.com/httpd/newpatch/ to see how we're handling the patch
file submissions.

> SubAlias Patch (December 94)
> Useful for when I close off areas of my server, you can alias
> an entire sub-tree to point to a given HTML document (or CGI
> script). Adding a line SubAlias /fish /home/www/nofish.html to
> your config file would mean any accesses to /fish,
> /fish/fred.html, /fish/pike/tuna.html would all return the
> nofish.html document.

This would be very useful! I wish Alias could use real Perl-style regular
expressions, and then I forget we're not using Perl :)

> Other Patches (January 95)
> Added a couple of environoment variables for CGI scripts to
> use, DOCUMENT_ROOT and SERVER_ADMIN. Having the Server Admin
> email address is incredibly handy - if my CGI scripts detect
> errors they can mail server_admin rather than some hard-wired
> address.

We added DOCUMENT_ROOT - I definitely see a benefit in SERVER_ADMIN, too.
A little more general approach might be to allow people to specify extra,
static (meaning they are the same for every transaction) CGI variables in
srm.conf:

AddCGI SERVER_ADMIN brian@hyperreal.com

To avoid name space problems an X_ might have to be prepended....

> Indexing Patch (June 94)
> Patches to allow descriptions for files to be generated from a
> 4dos-like description file. Heres an example directory listing
> and the corresponding descript.ion and .README files. (MJC)
>
> Just like a BBS system :) Look at below for an example
> URL: http://www.telescope.org/rti/fits/

Sort of like an AddDescription but set on a per-directory basis like
.htaccess... what about allowing AddDescription in .htaccess?

> Referer Logging
> I use the field reserved for identd info (its too slow to have
> turned on) to store the Referer. Means in all my logs I can
> trace the paths of users.

We talked a lot about logging, customizable logging, etc etc. The
problem with making small changes here and there to the log file format
is that it suddenly is *not* in the Common Log Format, so log file
analysers could break. Your change would probably not break those, but
anyone who does care about identd info is fuct.

Given these constraints, and in lieu of a real configurable solution that
doesn't require a recompile with every change and that can also be fast,
I would probably support a patch that added referrer and user_agent to
the end of the log files - URL's can't have spaces so it should be safe
to keep them space-delinieated (but I don't think anyone follows the
recommended user_agent standard so I don't think we can come up with a
character to encapsulate user_agent within the same way the date is
encapsulated within [ and ]). This should be a config file switch, since
it will nearly double the size of the logfiles, and we should also
distribute a perl script that will strip away this info if requested.

I don't like NCSA 1.4's implementation at all - it logs these two bits
of info into two *separate* log files so you can't do any sort of
statistical correspondence with the accesses (NCSA correct me if I'm
wrong). Also, the default is to turn it on, and one has to explicitely
point it to /dev/null to turn it off. Ew.

> DBM Password Patch (November 94)
> Current .htpasswd and .htgroup files are inefficient. It can
> take 10 seconds to serve each page when there are more than a
> few hundred users. This patch allows a new type "BasicDBM" in
> the .htaccess file. Usernames, Passwords, Groups and other
> details are then stored in a DBM file with instant access.
>
> Now My DBM files are keyed on username with content of
>
> crypt_password:groups:anything[:...]
>
> Where groups are a comma separated list of groups. All my server
> uses the same DBM user file:
>
> AuthUserFile /extdisk/www-adm/dbmpasswd
> AuthName Submitting Jobs
> AuthType BasicDBM
>
> <Limit GET POST>
> require group rti
> </Limit>
>
> Just allows users with "rti" in their group fields in. Do you
> want the patch for this Brian? If you incorporate the group
> facility into Apache it will be worth me swapping over.

Hmm - I personally don't like this type of binding, as I can see situations
(indeed, we have them here) where those who want to be able to edit the
.htgroup file do not also have access to modify the password files. It also
puts more info in the content fields, which could bloat the size of the
password file (which is an issue for those of us with 160,000 users in our
database :)

I've documented the current implementation in 0.4 at
<URL:http://hyperreal.com/apache/docs/P14>... I feel it's as close to a
drop-in replacement for .htpasswd files as one can get.

> PS I found two things in Apache 0.4 that had to be fixed before it
> would compile
>
> in http_log.c it doesn't like the "default:" clause to be empty before the
> closing brace.
> in http_mime_db.c it doesn't like the structure definition around line 390
> it's pro

Were you compiling with -Wall, or were these actual problems that needed
to be changed before complete compile? What platform were you on?

Brian

--=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=--
brian@hotwired.com brian@hyperreal.com http://www.hotwired.com/Staff/brian/
Re: apache [ In reply to ]
On Fri, 7 Apr 1995, Brian Behlendorf wrote:
> Sort of like an AddDescription but set on a per-directory basis like
> .htaccess... what about allowing AddDescription in .htaccess?

I (and several others who run Web sites) already have these description
files in a certain (standard) format. The file consists simply of a line
per file containing the filename, any amount of whitespace, and the
description.

This format is read by quite a few GIF and JPG viewers too so its nice not
to have to convert it or add it to a .htaccess file. See URL:
http://www.telescope.org/mwc/files.html for it in action.

> analysers could break. Your change would probably not break those, but
> anyone who does care about identd info is fuct.

Agreed. This sort of patch should not make it into a general release.

> (indeed, we have them here) where those who want to be able to edit the
> .htgroup file do not also have access to modify the password files.

Well lets make a separate group DBM file for it then. I would imagine
then that your 160,000 users are not in a default group? Parsing a
standard .htgroup file with a few thousand entries takes forever.

Wish-list:

A way of storing an objects information in a separate file (things you
might find in the Head or Header like Expiry, Owner...). Like the CERN
.meta files. You could include a global realm here too (see below)
Disadvantage is you are adding another file to look for on each request.

Global realms for password authentication. You define a realm, (name, who
can access it, location of password file) and just have the realm name in
the .htaccess file. Would make updating and management easier. I think
CERN does something like this, and it allows you to say things like ".y"
extensions are only accessible by "XYZZX" realm.

Host Authentication Fall-back. If you use host authentication to limit
some documents locally you can provide a "fall back" document if the host
is not allowed. I bodged something like this with a "-fail" extension.
It the host was not allowed the -fail document was returned if it was
there. If no -fail it failed as normal. This could be something for
MultiViews

Mark
Mark J Cox ----------------------- URL:http://www.telescope.org/mark.html
University of Bradford, UK --------------- tel +44.1274.384070/fax 391333
Re: apache [ In reply to ]
On Sat, 8 Apr 1995, Mark J Cox wrote:
> On Fri, 7 Apr 1995, Brian Behlendorf wrote:
> > Sort of like an AddDescription but set on a per-directory basis like
> > .htaccess... what about allowing AddDescription in .htaccess?
>
> I (and several others who run Web sites) already have these description
> files in a certain (standard) format. The file consists simply of a line
> per file containing the filename, any amount of whitespace, and the
> description.

Ah, okay. This introduces overhead in checking for the existance of the
file, but that's only introduces when an index is being created anyways
(which has its own amount of overhead) and is only done in that one
directory (rather than checked for in every directory like .htaccess).
Okay, I'd support a patch like this in theory.

> > (indeed, we have them here) where those who want to be able to edit the
> > .htgroup file do not also have access to modify the password files.
>
> Well lets make a separate group DBM file for it then. I would imagine
> then that your 160,000 users are not in a default group? Parsing a
> standard .htgroup file with a few thousand entries takes forever.

Right, I don't have the 160K in a group at all, I simply have

<Limit GET POST>
require valid-user
</Limit>

I would support turning the htgroup files into DBM files as well, where
the key is the user and the group is the value.

> Wish-list:
>
> A way of storing an objects information in a separate file (things you
> might find in the Head or Header like Expiry, Owner...). Like the CERN
> .meta files. You could include a global realm here too (see below)
> Disadvantage is you are adding another file to look for on each request.

Have you ever looked at GN (or WN)? http://hopf.math.nwu.edu/ is the
home for it - its model is based on only serving files that exist in a
per-directory menu file, and in that menu one can specify all these
attributes.

Better yet, as suggested on new-httpd a little while ago, why not have a
new file type recognized by the server to have its own HTTP headers,
which the server then dishes out and labels text/html (a la .shtml files).
i.e.

AddType text/html-with-headers .asis

The problem is that this isn't a general solution, i.e. it won't work for
GIF's or any other non-text file format (well, it could, but the creation
and modification of a GIF file once it has text garbage stuck to the
front would be a pain). The overhead would involve a stat() for every
access and a file-read-and-interpret if a .meta file existed. If this is
a run-time configuration (a la the .htaccess check) then I might support
it...

> Global realms for password authentication. You define a realm, (name, who
> can access it, location of password file) and just have the realm name in
> the .htaccess file. Would make updating and management easier. I think
> CERN does something like this, and it allows you to say things like ".y"
> extensions are only accessible by "XYZZX" realm.

Good optimization...

> Host Authentication Fall-back. If you use host authentication to limit
> some documents locally you can provide a "fall back" document if the host
> is not allowed. I bodged something like this with a "-fail" extension.
> It the host was not allowed the -fail document was returned if it was
> there. If no -fail it failed as normal. This could be something for
> MultiViews

I'm wondering if we're going to bury ourselves with file name extensions
here :) It sound like custom error responses almost meets your needs, if
it were configurable per directory that's almost all the way.

Brian

--=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=--
brian@hotwired.com brian@hyperreal.com http://www.hotwired.com/Staff/brian/
Re: apache [ In reply to ]
> > Host Authentication Fall-back. If you use host authentication to limit
> > some documents locally you can provide a "fall back" document if the host
> > is not allowed. I bodged something like this with a "-fail" extension.
> > It the host was not allowed the -fail document was returned if it was
> > there. If no -fail it failed as normal. This could be something for
> > MultiViews
>
> I'm wondering if we're going to bury ourselves with file name extensions
> here :) It sound like custom error responses almost meets your needs, if
> it were configurable per directory that's almost all the way.
>
> Brian

You can point it at a cgi-script which is infinitely configurable.
All the info you need to decide where the rejection came from is
available in the "REDIRECT_" variables.
Re: apache [ In reply to ]
On Fri, 7 Apr 1995 11:42:39 -0800 (PST), Brian wrote:
>On Fri, 7 Apr 1995, Mark J Cox wrote:
>> SubAlias Patch (December 94)
>> Useful for when I close off areas of my server, you can alias
>> an entire sub-tree to point to a given HTML document (or CGI
>> script). Adding a line SubAlias /fish /home/www/nofish.html to
>> your config file would mean any accesses to /fish,
>> /fish/fred.html, /fish/pike/tuna.html would all return the
>> nofish.html document.
>
>This would be very useful! I wish Alias could use real Perl-style regular
>expressions, and then I forget we're not using Perl :)

Unless I'm missing something, you can already do this with Alias in apache.

>> Indexing Patch (June 94)
>> Patches to allow descriptions for files to be generated from a
>> 4dos-like description file. Heres an example directory listing
>> and the corresponding descript.ion and .README files. (MJC)
>>
>> Just like a BBS system :) Look at below for an example
>> URL: http://www.telescope.org/rti/fits/
>
>Sort of like an AddDescription but set on a per-directory basis like
>.htaccess... what about allowing AddDescription in .htaccess?

Again, AddDescription is already allowed in .htaccess.

David.