Mailing List Archive

OK...we have ftp space on hyperreal...I don't knwo what the status of
making accounts is right now...it's brian@hyperreal.com's machines.
Anyone who thinks they will be a major contributor should send brian
mail and ask for an account, be sure to mention to him to
place you in the httpd group.

As for web pages...I will start with the list rob has sent and
get something up tonight. Things are calming down for me (working
two jobs sucks, I'll be leaving one soon) and I'm ready to
rock on this thing.

I'm ftp'ing stuff in now...the location will be:

ftp://hyperreal.com/httpd

-and-
http://www.hyperreal.com/httpd

(same diretory, I've place a lame index.html there for now)

OK...who wants to send me a mission statement?

Cliff

OK, folks, here's the list. My proposal for a first-cut Apache is to
take *all* the bug-fixes and performance enhancements as a body, and
leave any functional enhancements for later. If there's some
completely non-controversial and simple functional enhancement which
would buy us a lot, perhaps we should consider it. However, many of
the items on the functional enhancements list will take some
discussion to decide if we want them, and in what form, and I don't
think those discussions should be allowed to hold up the project.

Besides, the bug-fixes are work enough by themselves.

NB this list only includes stuff for which code is *already* available
in some form; that's why items like content negotiation aren't on it,
even though a lot of people want to see them happen. Perhaps I should
add a fourth category, Wn), for these Wish-list items....

Without further ado, The List:






Bug fixes: (most available in multiple versions)

B1) The stack-scribbling security hole. Note that there are two
patches for this:

B1a) The CIAC/CERT patch, and
B1b) The official NCSA patch.

The CERT patch makes the server processes substantially
larger; the NCSA patch doesn't, but a lot of us don't trust it.
The best thing may be a compromise which includes the CIAC/CERT
patch, but allows the server to be compiled without it via
something like -DINSECURE.

B2) Don't set SO_LINGER on client sockets. (On many systems, this
leads to processes which can't be killed because they're already
trying to die, but can't close their last file descriptor (the
socket either), so they just sit around taking up swap space).

I don't know of anyone who's made this available separately as
a formal patch; this is just *deleting* two lines from httpd.c.

B3) Server always pauses 3 seconds for scripts which send a redirect,
then gratuitously kills the process (which is probably dead anyway
at that point). Again, the fix is to dike out a line of code
(first pointed out by Eric Hagberg).

B4) <!--#config timefmt --> server-side include doesn't always take.
Nicolas supplied a patch for this.

B5) XBITHACK not honored on <!--#include-->ed files [Andrew Wilson]

B6) access log files should be written with O_APPEND.
Roy Fiedling has code for this.

B7) when access control is "Limit"ed with "Order allow,deny", the
server allows by default, making any Allow directives which may be
present redundant.

rst has a patch for this.

B8) httpd can't handle numeric User specs in httpd.conf unless that
uid appears in the passwd file. If multiple usernames have the
same uid, it sometimes sets group permissions with the wrong one.

Patches for this are integrated with drtr's version of the
initgroups() fix (P1 below).

Performance enhancements: (most available in multiple versions)

P1) Don't do initgroups() once per connection. (But do redo it after
rereading the config files).

Patch available from drtr.

P2) Don't do kernel read()s of one character, when reading MIME
headers from clients, or script CGI headers.

drtr and rst have their own versions; drtr's handles POST, but
looks like it might have somewhat more overhead in the simple GET
case.

P3) Don't do open_locale() and tzset() once per connection. (These
routines are called from C library time conversion code).

rst has a patch for this.

P4) Shared-memory name server cache.

rst has something like this, though its portability can't be
guaranteed. (There are things which claim to be Unix which don't
support shared memory in any form).

Functional enhancements: (Note that many of these are still in the
process of being packaged up for submittal):

E1) DBM-based user databases for HTTP authentication. [Brian]

E2) New CGI variables (DOCUMENT_ROOT, SCRIPT_TRANSLATED_NAME). [Brian?]

E3) *.doit scripts (allows *any* URL to invoke a script, whether it
ends in a magic *.cgi suffix or not). [rst]

E4) Extended UserDir --- if server sees "UserDir /foo/bar/%u/zot",
then /~user gets translated to /foo/bar/user/zot. [rst & downstairs]

E5) Logging User-agent and Referer, at least on errors. [Roy Fielding?]

E6) Load throttling --- reject incoming connections if load is too
high. [Robert Evans]

E7) Send Last-modified header for server-side-included docs if
group XBIT is set. [Rob Hartill, by way of Andrew Wilson]

E8) Custom error messages [Webmaster at Cardiff, by way of Rob Hartill]

E9) CGI-based content negotiation [Hartill] [NOTE: controversial]

E10) Referer and User-agent logging. This exists in several variants;
there are patches on the Net which put these in separate files
(which is awkward to cross-reference, and also would have more
overhead than adding fields to the access_log). Also, Roy has
code to log these variables on separate lines after the error;
somehow, I think there may be quibbles of taste about formatting
here (some people might prefer to have everything on one line).

From: rst@ai.mit.edu (Robert S. Thau)
Date: Wed, 8 Mar 95 18:58:21 EST

E5) Logging User-agent and Referer, at least on errors. [Roy Fielding?]

E10) Referer and User-agent logging....

Oh, drat. Move that E10) be stricken, due to redundancy with E5).

Sigh...

rst

OK...I've actually started entering data and am looking for some
ideas and input about what people would like to see. The
scripts are a quick hack, and need a bit of tweeking (to escape
things before output, <'s don't show up well in an HTML document
unescaped :-))

I'm using a slightly different scheme to number patches than rst, but
it's not a big deal. The numerical id of the patch in unique accross
all types (B, E, P). So the first set of patches ar B1 -> Bn, then P(n+1)
->P(m), E(m+1) blah blah blah. I've also created a type "O" for other.
The list is identical to his. I'd like to get submiter email addresses
attached to these requests also, please speak up if they are yours.

Take a peek at http://www.hyperreal.com/httpd, it's a start. Hoepfully
will evolve into something decent in the next few days. Right now the
cgi's are on my machine (www.steam.com), but I will be moving them over
after they are cleaned up and debuged a bit more.

My plans are to enhance the subimisson form with a bunch of
instructions. Also the result will include instructions on
where to ftp your patch code.

Cliff

Date: Thu, 9 Mar 95 17:43 GMT
From: drtr@ast.cam.ac.uk (David Robinson)

B1c) I have another patch (to translate_name) which fixes another hole in a
similar manner to the NCSA patch.

Now on my list.

> B2) Don't set SO_LINGER on client sockets.
Just out of interest, why did httpd do this?

Damned if I know. NB the CERN server used to do it too by default,
but it's now disabled on most configurations (though you can turn it
on in the rule file).

>P2) Don't do kernel read()s of one character, when reading MIME
> headers from clients, or script CGI headers.
>
> drtr and rst have their own versions; drtr's handles POST, but
> looks like it might have somewhat more overhead in the simple GET
> case.

My patch isn't necessarly slower than rst's for GETs; it does 3p
system calls where p = no of data packets (select,recv,read per
packet), whereas rst's does 4 + 3p calls (4 1-byte reads +
signal,alarm,read per packet).

That's probably good to first order; I've edited the note.

David.

rst

>B1) The stack-scribbling security hole. Note that there are two
> patches for this:
>
> B1a) The CIAC/CERT patch, and
> B1b) The official NCSA patch.
>
> The CERT patch makes the server processes substantially
> larger; the NCSA patch doesn't, but a lot of us don't trust it.
> The best thing may be a compromise which includes the CIAC/CERT
> patch, but allows the server to be compiled without it via
> something like -DINSECURE.

B1c) I have another patch (to translate_name) which fixes another hole in a
similar manner to the NCSA patch.

> B2) Don't set SO_LINGER on client sockets.
Just out of interest, why did httpd do this?

>P2) Don't do kernel read()s of one character, when reading MIME
> headers from clients, or script CGI headers.
>
> drtr and rst have their own versions; drtr's handles POST, but
> looks like it might have somewhat more overhead in the simple GET
> case.
My version of the patch also includes another performace enhancement

P5) scan_script_headers should use gets() instead of getline() to read
script headers.

My patch isn't necessarly slower than rst's for GETs; it does 3p system calls
where p = no of data packets (select,recv,read per packet), whereas rst's does
4 + 3p calls (4 1-byte reads + signal,alarm,read per packet).

David.