Mailing List Archive

On Mon, 10 Apr 1995, Rob Hartill wrote:
> Patch E63 is at hyperreal, it's a 2 line patch to enable people
> to set up a file extension or list of file extensions that Apache
> will treat as not wanting server generated HTTP headers to be
> attached.

What will a HEAD request on that document generate?

Brian

--=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=--
brian@hotwired.com brian@hyperreal.com http://www.hotwired.com/Staff/brian/

>
> On Mon, 10 Apr 1995, Rob Hartill wrote:
> > Patch E63 is at hyperreal, it's a 2 line patch to enable people
> > to set up a file extension or list of file extensions that Apache
> > will treat as not wanting server generated HTTP headers to be
> > attached.
>
> What will a HEAD request on that document generate?

Nothing, there are no headers as such. The "as is" type of
document isn't necessarily a HTTP response, it's whatever the
file contains.

I've concocted a (perhaps artificial) scenario in which .asis might
have security implications:

Imagine that there's a webserver someplace on which the badguy can't
create scripts, but can have .asis'es, and that there's a
password-protected document to which the badguy wants access. Suppose
also that the badguy has access to the log file entries concerning his
own documents. (A nontrivial number of university sites are probably
like this).

One thing the badguy could do is create a document which looks like it
has a pointer to the password-protected document, but which acutally
points to a .asis document which fakes up a 401 reply. (This is
presumably a copy of some other page which actually does point to the
real thing, with nothing different about it except the URLs in the
anchors, so that victims would have to be paying fairly careful
attention to notice the difference).

The usernames the badguy gets in the log entries for the .asis
document will tell him the names of authorized users for the
password-protected page; with those in hand, dictionary-based
strategies may be adequate for getting the passwords as well.
(NB if a user supplies invalid authentication info, the access_log
shows a 401 reply with authentication username other than '-' --- my
home-grown logfile summary tool actually uses this to detect people
who tried bogus usernames/passwords without having to grub through the
error_log. Do people actually try "Anonymous" elsewhere?)

Now, this whole scenario isn't nearly as bad as letting untrusted
users run scripts, of course, but it still might be something which a
reasonable webmaster, faced with a population of untrustworthy
undergrads, might want to avoid.

Is this too paranoid?

rst

> > > What will a HEAD request on that document generate?
> >
> > Nothing, there are no headers as such. The "as is" type of
> > document isn't necessarily a HTTP response, it's whatever the
> > file contains.
>
> But this _is_ HTTP. httpd should stop sending data after a blank line
> in the .asis file in response to a HEAD request.

probably. Is this the case for "nph" ? (I don't know)

robh

>
> I've concocted a (perhaps artificial) scenario in which .asis might
> have security implications:
>
> Is this too paranoid?

I just set up a .asis to return a 401.

Unlike a real 401, this doesn't seem to log the username.
Maybe we're just lucky :-)

robh

probably. Is this the case for "nph" ? (I don't know)

NPH scripts get a direct connection back to the client --- the server
hands off the socket to the client as the script's stdout. With that
come certain responsibilities, one of which is not sending the body
on a HEAD request; an nph script which doesn't handle that right (and
I'll admit, most probably don't) is breaking the server, and probably
ought to be fixed.

The difference between that case and .asis is that a .asis file can't
elect not to send back the "body" on its own, since it's just a bucket
of bits which get passed back to the client. So, I'm in agreement that
if we're going to support this, the server should cut it off after the
headers on a HEAD request...

rst

> The difference between that case and .asis is that a .asis file can't
> elect not to send back the "body" on its own, since it's just a bucket
> of bits which get passed back to the client. So, I'm in agreement that
> if we're going to support this, the server should cut it off after the
> headers on a HEAD request...

Version 2, which I just tried and saw working (patch to follow),
treats the .asis files as script output. HEAD works as expected.

Slight change of syntax though, now you'll have to use Status: to
send a variation on the HTTP response, but that's favourable
compared to "HTTP/1.0 xxx Message" anyway.


robh

> > On Mon, 10 Apr 1995, Rob Hartill wrote:
> > > Patch E63 is at hyperreal, it's a 2 line patch to enable people
> > > to set up a file extension or list of file extensions that Apache
> > > will treat as not wanting server generated HTTP headers to be
> > > attached.
> >
> > What will a HEAD request on that document generate?
>
> Nothing, there are no headers as such. The "as is" type of
> document isn't necessarily a HTTP response, it's whatever the
> file contains.

But this _is_ HTTP. httpd should stop sending data after a blank line
in the .asis file in response to a HEAD request.

David.

Actually, this is only a gain for the Bad Guy (tm) when he has access to read
the access logs for his pages but *not* for the other, protected pages. This
would mean he'd have to be on a separate system altogether, unless the web
adminstrator kept the original log files unreadable and had scripts that sent
portions of the log files out to user (i.e., a script that sent all accesses
to /~brian/ to brian@host). Actually, they don't have to be on the same
host/port, but then you're stretching the bounds of plausibility in the
initial condition, that the BadGuy creates a phony reference.

I wouldn't worry about it. Especially when tcpdump will give you all the
info you need to crack an account like that anyways! :)

Brian

On Tue, 11 Apr 1995, Robert S. Thau wrote:
> One thing the badguy could do is create a document which looks like it
> has a pointer to the password-protected document, but which acutally
> points to a .asis document which fakes up a 401 reply. (This is
> presumably a copy of some other page which actually does point to the
> real thing, with nothing different about it except the URLs in the
> anchors, so that victims would have to be paying fairly careful
> attention to notice the difference).
>
> The usernames the badguy gets in the log entries for the .asis
> document will tell him the names of authorized users for the
> password-protected page; with those in hand, dictionary-based
> strategies may be adequate for getting the passwords as well.
> (NB if a user supplies invalid authentication info, the access_log
> shows a 401 reply with authentication username other than '-' --- my
> home-grown logfile summary tool actually uses this to detect people
> who tried bogus usernames/passwords without having to grub through the
> error_log. Do people actually try "Anonymous" elsewhere?)

--=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=--
brian@hotwired.com brian@hyperreal.com http://www.hotwired.com/Staff/brian/

>I just set up a .asis to return a 401.
>Unlike a real 401, this doesn't seem to log the username.
>Maybe we're just lucky :-)

Actually, this is how it is supposed work. But we are lucky that it works
correctly. I'm following a comment of Brian's, which was:
'Only log the user if httpd thinks the document should be protected, otherwise
debugging authentication gets too difficult' (paraphrased.)

David.

>> The difference between that case and .asis is that a .asis file can't
>> elect not to send back the "body" on its own, since it's just a bucket
>> of bits which get passed back to the client. So, I'm in agreement that
>> if we're going to support this, the server should cut it off after the
>> headers on a HEAD request...
>
> Version 2, which I just tried and saw working (patch to follow),
> treats the .asis files as script output. HEAD works as expected.
>
> Slight change of syntax though, now you'll have to use Status: to
> send a variation on the HTTP response, but that's favourable
> compared to "HTTP/1.0 xxx Message" anyway.

Good -- I was sorely tempted to veto otherwise. The server needs to
be responsible (wherever possible) for the correct usage of the protocol.

One thing to keep in mind: What does the server send if it receives
an HTTP/0.9 request for that resource?

.......Roy

answer: The body without any headers (i.e., the opposite of HEAD).

On Tue, 11 Apr 1995, Roy T. Fielding wrote:
> Good -- I was sorely tempted to veto otherwise. The server needs to
> be responsible (wherever possible) for the correct usage of the protocol.
>
> One thing to keep in mind: What does the server send if it receives
> an HTTP/0.9 request for that resource?
>
> .......Roy

> answer: The body without any headers (i.e., the opposite of HEAD).

Just so implementers get this right - a 0.9 request looks like

GET /file.html

instead of

GET /file.html HTTP/1.0

Brian

--=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=--
brian@hotwired.com brian@hyperreal.com http://www.hotwired.com/Staff/brian/

Date: Tue, 11 Apr 1995 18:41:30 -0800 (PST)
From: Brian Behlendorf <brian@wired.com>

Just so implementers get this right - a 0.9 request looks like

GET /file.html

instead of

GET /file.html HTTP/1.0

... and the variable which gets set by process_request (or one of its
subroutines) to indicate that we're handling a 0.9 request bears the
charming name "assbackwards".

rst

> I rather like this feature, but I have a few reservations about the patch.
> The most significant (almost -1) comment is about the magic mime type used:
> httpd/send-as-is. This doesn't seem in keeping with the other mime types used
> in httpd. Surely application/x-httpd-asis (cf. application/x-httpd-cgi)
> would be more appropiate?

Netsite uses "magnus/" or something equally weird :-)

I don't think that "application" conveys the true meaning of the
"mime" type, and the bottom line is that as long as the server is
intercpting them, cleints never see our silly names.

> Other concerns (a fix would get my +1):
> httpd should send the body only in response to an HTTP/0.9 request.

I'll ponder this. It's debateable where "as is" should mean without
headers in this case, since the headers are part of the document, but
it'd certainly be more consistent if it did.

> Shouldn't httpd send a Last-Modified header if the file does not already
> contain one?

Possibly. another thing to look into.


Can we at least get the existing proposal in 0.6,
and fine tune it later ?

> Yes, but are you sure httpd/send-as-is is better? I thought the - delimit
> a hierarchical structure, instead of providing punctuation for an English
> phrase 8-)

these aren't mime types, so they don't have to conform to anything
other than the server code which reads them.
They're pretend mime types.

Apache no longer lies about being Mime compatible now, so even if
one sneaked out the door, it'd be harmless.

>
> Can we at least get the existing proposal in 0.6,
> and fine tune it later ?
>
> It depends what you want to be the beta release candidate --- if
> 0.5.1 *is* beta 1, then throwing code which will need to be fine-tuned
> later into 0.6 is not a problem. On the other hand, if 0.6 is going to
> be our first beta release, then we probably shouldn't have stuff in it
> which has known problems.

that depends on whether they are "problems" or "improvements"

I'd say things like name changes were improvements, whereas
patches which broke existing features were problems.

If the objections only appear a few hours before the planned rebuild, the
patch contibutor has no time to make changes, and he ends up chasing a
moving target, as Andy put it.

I think when it comes to these improvement suggestions, progress should
outweigh a desire for perfection.

robh

I'd like to see 0.6 as the beta (will E55 of course). Betas IMHO
should not vary too much in the way of features (actuall not
at all, but that aside). Beta to FCS (first customer ship) should
be mainly hammering out bugs not feature development.

Cliff

Can we at least get the existing proposal in 0.6,
and fine tune it later ?

... just as long as at some point we've got something in which all the
code has been properly fine-tuned, and we haven't introduced any new
features that week which themselves need fine-tuning; if not, then
we'll never have anything ready to ship.

rst

Can we at least get the existing proposal in 0.6,
and fine tune it later ?

It depends what you want to be the beta release candidate --- if
0.5.1 *is* beta 1, then throwing code which will need to be fine-tuned
later into 0.6 is not a problem. On the other hand, if 0.6 is going to
be our first beta release, then we probably shouldn't have stuff in it
which has known problems.

rst

From: Rob Hartill <hartill@ooo.lanl.gov>
Date: Fri, 14 Apr 95 10:10:30 MDT

> It depends what you want to be the beta release candidate --- if
> 0.5.1 *is* beta 1, then throwing code which will need to be fine-tuned
> later into 0.6 is not a problem. On the other hand, if 0.6 is going to
> be our first beta release, then we probably shouldn't have stuff in it
> which has known problems.

that depends on whether they are "problems" or "improvements"

I'd say things like name changes were improvements, whereas
patches which broke existing features were problems.

If the objections only appear a few hours before the planned rebuild, the
patch contibutor has no time to make changes, and he ends up chasing a
moving target, as Andy put it.

I'm not sure you got the point of my objections above --- most of the
time, I wouldn't have any objections to throwing in things which seem
to be a good idea, even if they aren't quite fully baked, because most
of the time, we can just slip a fix for any problems that arise into
the next week's internal release.

However, if we are going to be *publically* releasing 0.6, then
circumstances are different --- fixing the problems means either
delaying the release another week, or shipping code with bugs. So,
it's just because we are closing to a public release that I'm
hesitant.

rst

I rather like this feature, but I have a few reservations about the patch.
The most significant (almost -1) comment is about the magic mime type used:
httpd/send-as-is. This doesn't seem in keeping with the other mime types used
in httpd. Surely application/x-httpd-asis (cf. application/x-httpd-cgi)
would be more appropiate?

Other concerns (a fix would get my +1):
httpd should send the body only in response to an HTTP/0.9 request.
Shouldn't httpd send a Last-Modified header if the file does not already
contain one?
httpd should observe any If-Modified-Since header in the request, and compare
it to any Last-Modified header in the file, or to the Last-Modified header
httpd would have added to the file if it was not present.

David.

>Can we at least get the existing proposal in 0.6,
>and fine tune it later ?
Oh yes, I wouldn't veto it on the basis of its behaviour, only on the basis
of the user interface. i.e. I _would_ veto it if we were likely to (want to)
change the mime type.

>I don't think that "application" conveys the true meaning of the
>"mime" type, and the bottom line is that as long as the server is
>intercpting them, cleints never see our silly names.

Yes, but are you sure httpd/send-as-is is better? I thought the - delimit
a hierarchical structure, instead of providing punctuation for an English
phrase 8-)

David.

Cliff wrote:
>I'd like to see 0.6 as the beta (will E55 of course). Betas IMHO
>should not vary too much in the way of features (actuall not
>at all, but that aside). Beta to FCS (first customer ship) should
>be mainly hammering out bugs not feature development.

So I'd rather 0.6 was not a beta straight away, but that we'd try it out
for a week and (almost certainly) release a 0.6.1 with _only bug fixes applied_.

David.